How to Build an RSS and Mutate Data using AI in a Newly Signed-up Product?

This topic was automatically generated from Slack. You can find the original thread here.

Hey there! Fresh user here. Recently signed-up this awesome product to play a bit.

My use case is to build an RSS and add some items by fetching and the data from an API and mutate it (to categorize) using AI. Is that possible? How can build an RSS? Any example video, docs, etc, would be great!

Welcome to Pipedream! Why don’t you try out asking Pi, our support bot? Just tag Pi and then ask your question, like this:

Hey there! Fresh user here. Recently signed-up this awesome product to play a bit.

My use case is to build an RSS and add some items by fetching and the data from an API and mutate it (to categorize) using AI. Is that possible? How can build an RSS? Any example video, docs, etc, would be great!

To build an RSS feed and add items by fetching data from an API, then categorizing it using AI, you can follow these steps in Pipedream: 1. Fetch Data from an API: You’ll start by fetching the data you want to include in your RSS feed. This can be done using an HTTP request to the API from which you want to fetch data. You can use the axios library in a Pipedream workflow to make these requests. 2. Categorize Data Using AI: Once you have your data, you can categorize it using AI. This step depends on the specific AI service you’re using. If you’re using a service like OpenAI, you can make an HTTP request to their API, passing the data you fetched in step 1. 3. Build an RSS Feed: After categorizing your data, you can format it into an RSS feed. This involves creating an XML file that conforms to the RSS standard, including the categorized data in the appropriate XML tags. 4. Serve the RSS Feed: Finally, you’ll need to serve the RSS feed so it can be accessed by RSS readers. This can be done by hosting the XML file on a web server or using a Pipedream HTTP response step to serve the RSS feed directly from your workflow. Here’s a simplified example of how you might fetch data from an API and categorize it using AI within a Pipedream workflow. This example does not include the complete implementation of generating and serving an RSS feed but focuses on fetching and categorizing data: ``` import { axios } from “@pipedream/platform” export default defineComponent({ async run({ steps, $ }) { // Fetch data from an API const apiData = await axios($, { url: “https://api.example.com/data”, method: “GET”, }) // Assume apiData contains an array of items you want to categorize const categorizedData = apiData.map(async (item) => { // Categorize each item using an AI service (e.g., OpenAI) const response = await axios($, { url: “https://api.openai.com/v1/categorize”, method: “POST”, headers: { Authorization: Bearer YOUR_OPENAI_API_KEY, }, data: { text: item.description, // Assuming each item has a description }, })

Thank you , Pi is great but some open questions here:

I’m not so familiar with Node so even though Pi summarized the steps here, I understand I should follow similar steps but,
involves creating an XML file that conforms to the RSS standard, including the categorized data in the appropriate XML tags. i couldn’t see in the provided code that uses xml here.

I’m looking for something solid example or playground that I can test. Appreciate it.

I was checking OpenAI docs and i don’t think they have `

https://api.openai.com/v1/categorize

endpoint: https://platform.openai.com/docs/api-reference/batch/requestInput

So my specific use-case may a bit above and beyond for this bot :thinking_face:

You can direct Pi if it got something wrong

Are you building an RSS from scratch or are you creating a new one?

And the items you’re fetching to categorize is from your RSS feed?

i’d like to build an RSS using the payload that fetched from an API. So that I can consume that RSS using the public endpoint.

Cool, so let’s take this step by step.
assume I have a variable called data. How do I upload data to my RSS?

Uploading data to an RSS feed isn’t a direct action since RSS feeds are typically generated from the content on a website or platform. Instead, you would update the source content or database that your RSS feed is generated from. However, if you’re looking to programmatically interact with RSS feeds, such as creating a new item in a feed or parsing RSS feed data, you can use Pipedream’s capabilities to work with RSS feeds or other APIs. For example, if you have data that you want to include in an RSS feed, you would typically need to: 1. Update the content on your website or platform that generates the RSS feed. 2. Ensure that this update triggers a new item in your RSS feed. If you’re looking to work with RSS feeds in Pipedream, you might be looking to parse an RSS feed and perform actions based on its content. Here’s an example of how you can use Pipedream to parse an RSS feed: import { axios } from "@pipedream/platform" import Parser from "rss-parser"; export default defineComponent({ props: { rss: { type: "app", app: "rss", }, }, async run({ steps, $ }) { const parser = new Parser(); // Replace 'your_feed_url' with the URL of the RSS feed you want to parse const url = "your_feed_url"; const feed = await parser.parseURL(url); const { title, items } = feed; console.log(`Title: ${title}`); items.forEach(item => { console.log(`Item: ${item.title}`); }); // Here you can add logic to perform actions based on the RSS feed content }, }); This example uses the rss-parser package to parse an RSS feed and logs the title of the feed and each item’s title. You can replace the console.log statements with your logic to work with the data variable or perform other actions based on the feed’s content. If you’re looking to update an RSS feed with new data, you would typically do this through the platform or CMS that hosts your feed, as RSS feeds are generated from dynamic content sources like blogs, news sites, or other content management systems.

I’m not familiar with the RSS part but you can definitely fetch data from an API and use ChatGPT to categorize it

Check out our built-in HTTP and OpenAI components

Thank you - I’m checking right now

My struggle point is to figure out how to create an RSS from an input (payload) and serve it to public

I found this example: Generate an RSS feed from HTTP POST requests, retrieve via GET request - Pipedream

but couldn’t able to reproduce. i also had to open it in incognito mode, otherwise it redirects me to: https://pipedream.com/workspace-not-found endpoint (i logged-in)

Oh, that’s a v1 workflow (deprecated). Let me try converting it to the latest version

Thank you!!! Also found this: TikTok RSS Feed Generator - Pipedream
it didn’t seem to work, maybe deprecated?

Yes, that’s also a v1 workflow