How can I connect a Pipedream Source Event to trigger a workflow when a new file is added to a Dropbox folder?

[user-1]

This topic was automatically generated from Slack. You can find the original thread here.

Hi. I’ve worked with Pipedream for about 2 years, I’m using v1 workflows, and up to this point all api calls are working as designed, However, I’m having some difficulty implementing Source Events with a workflow. I created a Source event that listens for new files in a Dropbox folder, and it works because I can see the new file within the event logs. All I want this event to do is just trigger a workflow where I can process the file(s) that have been created/moved into this Dropbox folder. How to I get the Source event to trigger the workflow? Within a javascript fetch, do I await a response from an endpoint for the Source event?

[user-1]

If you’re using a Dropbox - New File source, then whenever a new file is added to the Dropbox folder, it’ll trigger the source and any connected workflows.

Are there any workflows connected to this source? You can tell by opening the Sources area in the dashboard, and then opening the dropdown on the individual source to see the connected workflows:

[user-1]

If you don’t see any connected workflows, that means your source needs to be connected to one.

You can open a new workflow, then for the trigger select My Sources to pick this Dropbox - New File source as the trigger for this particular workflow.

[user-1]

Step 1. Created the Dropbox Source for New File.
Step 2. Created a New Workflow
Step 3. Selected My Sources, and linked the Dropbox Source
Step 4. Cleared the files from Dropbox folder to be triggered for new file
Step 5. Selected Workflow that linked to Source
Step 6. Clicked on Run Now button
Step 7. Moved a file into Dropbox Folder

When I review the source logs, the transaction for the even shows up.
I’m just not sure how to access the trigger from either a webhook or any type of url.

Obviously I’m missing the concept of how to access when the source is triggered. I have been using the axios within my workflows (v1s) to get information using the Dropbox api calls. The workflows that I’m creating all start with an https:// url that triggers the workflows. Those workflows are initiated by a fetch statement within either my javascript within my app or within my Pipedream workflow to call another url. Each of the url’s return data.

However, when creating a Source event for the Dropbox add new file, there doesn’t appear to be a url associated with the Source. When I go into the Source trigger, it shows that it’s waiting for events to occur.

I’ve tried adding 2 types of triggers. 1. with the url, and 2. with the Dropbox Source trigger.

Within the logs for the Dropbox Source new file, there exists 5 lines per log transaction…

2024-01-19T21:46:19 end

2024-01-19T21:46:19 db.set
db[“dropbox_state”] ⭅ {“path”:“/SolePro_DB/sjn001tvh/~FileQ”,“recursive”:true,“cursor”:“AAHQqxfarN9wm3–uLi3goNkuS3thJh4_cx2ChtTRRB9FGMECiQJ4Q7RlJWQrT0CkZQypp4nL2nv9p64-uycK6dN9sh832_PlB2AYN18oFTPr8fZ5nfSWuVHs5UODPBiPUXbKlI6ynMaDf7TlJ5BsmEQgR1yUy18QGvY2ldUyVSbsdgmWFX47tfS8YTQrArqZ9uUojgyWtEohxKkQEp9JF9nUts0x0Yb0PD9wY6-1-bFiNwkD6sEljA9zoThaxzrDnmm2-Lu9WeVUv1e0S9J3SjB”}

2024-01-19T21:46:18 db.get
db[“dropbox_state”] ⭆ {“path”:“/SolePro_DB/sjn001tvh/~FileQ”,“recursive”:true,“cursor”:“AAF2OrFDcbC4HNKa9Chn1b7mMVPyyIu102OmcJuoj3SVM5MoRJQ3QwQx9qHWptsckIw4QZ7fpU2lmIsSR5FKdY4lbulUF4i972B2_Fe9fZV5MjHvMzM5TYRwJ1fmjLYqtnRgseq7zZ5mgdDJWnbX3tiRybZw12tT5t11iv8bF64sF8k0L8cSiM0LhjMRBLX2SBYG_UjKwhtNpTqRfCOyPqbnUqs-H7cycv1CC7aS4EwIn6JRfZJC-VSNKoJYzXwb3oq7XRomO0a9PwWx3lBqAb-c”}

2024-01-19T21:46:18 db.get db[“last_file_mod_time”] ⭆ “2024-01-20T05:45:50Z”

2024-01-19T21:46:18 start { “timestamp”: 1705729578 }

I can’t figure out what method is available to me to access these logs. And what is the url that I can use to wait for a response from this trigger to indicate that a new file has been placed in the Dropbox folder?

Thanks,

Scott.

[user-1]

I think Step #4 in your description is the issue.

Pipedream sources have deduplication built in. When you first created the source, the source will do an initial poll of the Dropbox API for matching files. Since there is no workflow attached yet, those files won’t trigger the workflow.

I believe what’s happening is even though you are deleting the files from the Dropbox folder after the source is deployed, the Pipdream source’s database is still caching those file names/IDs.

Have you tried uploading a totally unique file to the Dropbox folder that’s being monitored by the Pipedream source after the Pipedream Dropbox source has been crreated?