This topic was automatically generated from Slack. You can find the original thread here.
I have an rss trigger that sends telegram messages on every new entries, but it sends every day at 1.10 am even old entries without a reason, can you help me?
It seems like your RSS trigger may be firing at a specific time and sending messages for entries that it has already processed. This could be due to the way the trigger is configured or how the workflow is handling the entries. To help you further, I would need to understand a bit more about your current setup: 1. Are you using the “New Item in Feed” trigger or another RSS trigger? 2. How is your workflow configured to determine which entries are new? 3. Do you have any deduplication logic in place to prevent sending messages for old entries? Once I have this information, I can provide more targeted assistance. In the meantime, you might want to review the logic in your workflow that determines which entries are considered new and ensure that it’s only processing and sending messages for truly new entries. If you continue to experience issues, please visit Pipedream Support for further assistance.
I just created my custom source copying the original new-item-in-feed.ts so I change nothing
It seems like your question was cut off. Could you please provide more details on what you need help with regarding your custom source based on the new-item-in-feed.ts? Are you experiencing any issues, or do you need guidance on what to do next after copying the source?
I have an rss trigger that sends telegram messages on every new entries, but it starts sending everyday at around 1 am old entries without a reason, can you help me ? I just created my custom source copying the original new-item-in-feed.ts so I changed nothing
It seems like your custom RSS trigger might be reprocessing old entries and sending them as new. This could be due to the state management within the trigger, which is responsible for keeping track of already processed items. To help you further, I would need to see the code of your custom source to understand how it’s handling state. However, if you’re using a custom source based on the original new-item-in-feed.ts, make sure that you’re correctly updating the state after processing each item. The state should be used to store the timestamp or unique identifier of the last processed item, and your trigger should only process items that are new since that timestamp or identifier. Here’s a general approach on how you might handle state in a custom RSS trigger: export default defineComponent({ props: { rss: { type: "app", app: "rss", }, }, async run({ steps, $ }) { const Parser = require('rss-parser'); const parser = new Parser(); // Retrieve the last processed entry's timestamp or unique identifier from the state const lastProcessed = this.$checkpoint; // Replace with your feed URL const url = "https://pipedream.com/community/latest.rss"; const feed = await parser.parseURL(url); // Filter out items that are older than the last processed one const newItems = feed.items.filter(item => { // Assuming each item has a unique identifier or timestamp return isNewItem(item, lastProcessed); }); // Process new items and send messages to Telegram for (const item of newItems) { // Your code to send the message to Telegram } // Update the state with the timestamp or unique identifier of the last item processed this.$checkpoint = newItems.length > 0 ? getIdentifier(newItems[0]) : lastProcessed; }, }); function isNewItem(item, lastProcessed) { // Implement logic to determine if the item is new // For example, compare timestamps or unique identifiers } function getIdentifier(item) { // Implement logic to get the unique identifier or timestamp from the item } Make sure to replace isNewItem and getIdentifier with the actual logic that applies to your feed items. If you continue to experience issues, please provide the code of your custom source, and I can help you debug it further. If
her it is the code:
import rss from "../../app/rss.app";
import { defineSource } from "@pipedream/types";
import rssCommon from "../common/common";
export default defineSource({
...rssCommon,
key: "rss-new-item-in-feed-reversed",
name: "New Item in Feed Reversed",
description: "Emit new items from an RSS feed",
version: "2.1.1",
type: "source",
dedupe: "unique",
props: {
...rssCommon.props,
url: {
propDefinition: [
rss,
"url",
],
},
publishedAfterThan: {
type: "string",
label: "Published After Than",
description: "Emit items published after the specified date in ISO 8601 format .e.g `2022-12-07T12:57:10+07:00`",
optional: true,
},
},
hooks: {
async activate() {
// Try to parse the feed one time to confirm we can fetch and parse.
// The code will throw any errors to the user.
await this.rss.fetchAndParseFeed(this.url);
},
},
methods: {
...rssCommon.methods,
generateMeta: function (item) {
return {
id: this.rss.itemKey(item),
summary: item.title,
ts: Date.now(),
};
},
},
async run() {
const items = await this.rss.fetchAndParseFeed(this.url);
for (const item of items) {
if (this.publishedAfterThan) {
const publishedAfterThan = +new Date(this.publishedAfterThan);
const ts = this.rss.itemTs(item);
if (Number.isNaN(publishedAfterThan) || publishedAfterThan > ts) {
continue;
}
}
const meta = this.generateMeta(item);
this.$emit(item, meta);
}
},
});
It looks like you’ve shared a Pipedream component that defines a custom RSS source, which emits new items from an RSS feed in reverse order. This source includes a unique property publishedAfterThan that allows users to filter items published after a specific date. If you have any specific questions about this code or need assistance with modifications or understanding how it works, please let me know, and I’d be happy to help!