Dedupe based on incoming key (exit early if we've seen this key before)
•@dylburger•
•code:
•data:private•last updated:4 years ago
today
Build integrations remarkably fast!
You're viewing a public workflow template.
Sign up to customize, add steps, modify code and more.
Join 1,000,000+ developers using the Pipedream platform
steps.
trigger
HTTP API
Deploy to generate unique URL
This workflow runs on Pipedream's servers and is triggered by HTTP / Webhook requests.
steps.
nodejs
auth
to use OAuth tokens and API keys in code via the
auths objectcode
Write any Node.jscodeand use anynpm package. You can alsoexport datafor use in later steps via
return or this.key = 'value', pass input data to your code viaparams, and maintain state across executions with$checkpoint.async (event, steps) => {1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
}25
const get = require("lodash.get")
const includes = require("lodash.includes")
const MAX_LENGTH = 10000
// Retrieve emails stored in $checkpoint
const emails = get($checkpoint, 'emails', [])
// and grab the key to dedupe in, from the event body
const { email } = event.body
if (includes(emails, email)) {
$end("Email already seen. Exiting early")
}
if (emails.length >= MAX_LENGTH) {
emails.shift()
}
// Store new emails back in $checkpoint
emails.push(email)
$checkpoint = {
emails
}