This topic was automatically generated from Slack. You can find the original thread here.
Hello, I have made an agent that finds YouTube videos for comments. The problem is it just keeps fetching the same youtube videos and not new ones. Any ideas of how to fix this? Many Thanks
It sounds like your agent is fetching duplicate YouTube videos because it doesn’t have a way to remember which videos it has already processed. To fix this, you can use a Pipedream Data Store to keep track of the video IDs you’ve already seen. This way, each time your agent fetches new videos, it can skip any that are already in the data store. Here’s how you can do it: 1. Store processed video IDs: Each time your agent processes a video, save its video ID to a data store. 2. Check for duplicates: Before processing a new video, check if its ID is already in the data store. If it is, skip it. ### Example in Node.js ```
export default defineComponent({ props: { data: { type: “data_store” }, // Add a Data Store prop videos: { type: “string”, label: “YouTube Video IDs” }, // Array of video IDs to process }, async run({ steps, $ }) { const newVideos = ; for (const videoId of this.videos) { const alreadyProcessed = await this.data.get(videoId); if (!alreadyProcessed) { // Process the video here (e.g., fetch comments) // … // Mark as processed await this.data.set(videoId, true); newVideos.push(videoId); } } return { processed: newVideos }; }, });
I’m sorry to hear you’re having trouble. For canceling your Pipedream account and requesting a refund: - To cancel your plan: You can cancel your plan in your Billing and Usage Settings. You will have access to your paid plan through the end of your current billing period. Pipedream does not prorate plans cancelled within a billing period. - For immediate cancellation or refund requests: Please reach out to Pipedream Support for help with immediate cancellation or to discuss a refund. If you need further assistance or want to speak to someone directly, please visit Support - Pipedream.
what is this error? and how can I fix it Error
The request cannot be completed because you have exceeded your quota. Details
at Gaxios._request (/pipedream/dist/code/6099595d204a4de70b72db85d875998701aeec3cbf88988b43506b5142e6ffdc/node_modules/.pnpm/gaxios@7.1.1/node_modules/gaxios/build/cjs/src/gaxios.js:154:23)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at OAuth2Client.requestAsync (/pipedream/dist/code/6099595d204a4de70b72db85d875998701aeec3cbf88988b43506b5142e6ffdc/node_modules/.pnpm/google-auth-library@10.1.0/node_modules/google-auth-library/build/src/auth/oauth2client.js:463:20)
at null.createAPIRequestAsync (/pipedream/dist/code/6099595d204a4de70b72db85d875998701aeec3cbf88988b43506b5142e6ffdc/node_modules/.pnpm/googleapis-common@8.0.2-rc.0/node_modules/googleapis-common/build/src/apirequest.js:308:25)
at Object.listVideos (file:///pipedream/dist/code/6099595d204a4de70b72db85d875998701aeec3cbf88988b43506b5142e6ffdc/code/common-app.mjs:360:14)
at Object.run (file:///pipedream/dist/code/6099595d204a4de70b72db85d875998701aeec3cbf88988b43506b5142e6ffdc/code/actions/list-videos/common.mjs:20:30)
at null.executeComponent (/var/task/launch_worker.js:335:22)
at MessagePort.messageHandler (/var/task/launch_worker.js:835:28)
Use Case
By Video ID
id
Video IDs
[ “E0Cs2N7xWUQ,UX3PClekZAY,KOJQNWOm-a8,8_KKwVvoUYk,3WRcGZRpo9I” ]
Part
[ “snippet”, “statistics” ]
Maximum Results
20
Build from event
Replay event
I’m sorry you’re running into this issue. The error message: > The request cannot be completed because you have exceeded your quota. means that your Google (YouTube Data API) account has hit its API quota limit. This is a limit set by Google, not Pipedream, and it restricts how many API requests you can make in a given period (usually per day). ### How to Fix 1. Wait for Quota Reset: Quotas typically reset every 24 hours. You may need to wait until your quota is refreshed. 2. Check Your Google API Quota: - Go to the Google Cloud Console. - Select your project. - Go to “APIs & Services” > “Dashboard” > “YouTube Data API v3”. - Click “Quotas” to see your current usage and limits. 3. Request a Higher Quota: If you need more quota, you can request an increase from Google. In the Quotas section, click the “EDIT QUOTAS” button and follow the instructions. 4. Optimize Your Workflow: - Reduce the number of API calls your workflow makes. - Avoid unnecessary polling or duplicate requests. 5. Check for Loops or Repeated Calls: If your workflow is fetching the same videos repeatedly, it may be stuck in a loop or not deduplicating requests. Review your workflow logic to ensure you’re only making necessary API calls. — If you continue to have trouble or need help optimizing your workflow, please visit Support - Pipedream for more support options. Let me know if you need more specific help with your workflow logic or setup!