New to pipedream and webhooks – all I am trying to create is a system that has a URL that I can send postmark webhooks to and I want to store the payload of each of these POST requests in an s3 bucket. Is this something I can do with pipedream or should i be doing this entirely in AWS?
@kunalrye thanks for reaching out. Pipedream is perfect for this kind of use case.
Create a new workflow and select the HTTP API trigger. This will generate a unique HTTP endpoint for this workflow. Configure your Postmark webhooks to send traffic to this URL.
Then take a look at our Amazon S3 destination. Once you configure your S3 bucket to accept data from Pipedream, you can add a new Node.js code step to your workflow and save the HTTP payload like so:
Figured out the prefix situation I believe. So I could just set up multiple of these workflows to send the webhook post request into different folders of my S3 bucket by changing the prefix?
EDIT: also can I block all public access while using pipedream to send webhooks to my s3?
Re: the prefix, that’s correct. You can send the data to any prefixes you’d like, including a dynamic prefix based on data from the incoming event:
$send.s3({
bucket: "your-bucket-here",
prefix: event.body.name, // example value you want to assign to the prefix
payload: event.body,
});
Re: public access, when you add a bucket policy, I believe S3 may still complain that you’re not blocking all public access (i.e. it may show a warning), but the bucket policy from our docs does not grant anyone from the public access to the bucket. It simply allows Pipedream to put objects into the bucket and manage uploads.