This topic was automatically generated from Slack. You can find the original thread here.
Ed Johnson : Hello, is there a way to view Events in the Sources section for an entire day?
Dylan Sather (Pipedream) : Hi , is your source producing more than 100 events a day?
Ed Johnson : Hi Dylan
Ed Johnson : Yes it is
Dylan Sather (Pipedream) : Thanks. Currently, you can only view the last 100 events for a given source in the UI. Would you be open to seeing the event data in a separate data store, like a Google sheet? You can have your source trigger a workflow to route the data virtually anywhere. Happy to show you how that works
Ed Johnson : Sure
Ed Johnson : Ideally, I’d like to be able to view by day. Our client is saying that they aren’t receiving all of our records so this would help manage it on a more granular level.
Dylan Sather (Pipedream) : Are you familiar with SQL?
Ed Johnson : sorry for my delay in response, I am not getting notifications.
Ed Johnson : I am not familiar, no.
Ed Johnson : I take that back lol. Familiar? Yes. Skilled? No.
Dylan Sather (Pipedream) : haha got it. I was trying to figure out an easy way for you to ask arbitrary questions about these events, for example: show me the count of events by day, show me the details of a specific events, etc.
Have you seen SQL - Pipedream ? You can have your event source trigger a workflow, then send that event data to a table in the SQL service. If you need to just QA the event data to ask how many events were emitted by your source in a given day, you can run something like:
SELECT COUNT(*)
FROM my_data
WHERE date = '2020-10-23'
(assuming you sent the date in the event data from your workflow to the SQL service).
Lots more docs on the SQL service here: https://docs.pipedream.com/destinations/sql/#pipedream-sql-service
Dylan Sather (Pipedream) : that’s just one option, though. If your event volume is relatively low, Google Sheets may also be a good option. Airtable is another
Ed Johnson : How would I utilize Google Sheets
Dylan Sather (Pipedream) : First, when you click on your source in Sources - Pipedream, you’ll see a button to Create Workflow in the center of the screen, under the details of your source. Click that.
That’ll create a Pipedream workflow that will run each time your event source emits an event. In that workflow, add a new step, and you’ll see the action Add Single Row to Sheet in the default list that appears. That’ll let you add a row of data, with multiple columns that reference the incoming event data from your event source. See this example workflow, where I’m adding two columns of data to new rows when my workflow is triggered.
Dylan Sather (Pipedream) : let me know if that makes sense
Dylan Sather (Pipedream) : at a high level the structure looks good, but you’ll need to replace the values of the mock data / email with the references to the variables you’d like to store. If you delete the content of one of those fields and keep your cursor in the field, you’ll see a selector just below the field that allows you to select variables from event , which contains your event data.
You’ll also want to enable the trigger for your workflow in the trigger step, at the top
Dylan Sather (Pipedream) : you should also be able to select a test event from the list of events your source emitted, so you can select one of those, then press Send Test Event to send one of your events through the workflow
Ed Johnson : Please forgive my novice-ness. Could you help me select the right events if I’m looking to capture date and time?