Newbie help please - Process question

Help
,
1

I have a process question:

I want to collect data from a website (Weebly) to send to another website (Wordpress). What is the best way to do this:
Should I use a web scraper to fetch static data into a Google sheet and then send the entries from Google to the Wordpress site with a webhook?
Or can I fetch the data with a web scraper and send directly to Wordpress?

And then how to fetch data from the same source website when it is updated on a weekly basis:
Should I use a web scraper, or is there another way to fetch dynamic data? There is no RSS option on this site.
Do I need to collect into a Google sheet and then send to Wordpress or can I send directly from the source site to the destination site?

I hope this question is not too basic … I still need to work through the University videos.

Thanking you in advance.

2

Hi @hello-02e806

I recommend approaching this as simply as possible.

You’ll need to solve these three problems specifically.

  1. Identify new posts on your source website
  2. Parse these new posts into a format for a WordPress post
  3. Create a new WordPress post from the formatted data

For solving problem #1, an API or RSS feed of posts is ideal. If that’s not an option, you can build a custom source that will poll and look for new posts on a specific website’s index page.

Then I assume you’d need to use a webscraper like BeautifulSoup or Cheerio.js to format these posts for #2.

Finally, you can use the built in WordPress Pipedream actions to publish a new post.

This is just a general guide, but hope this helps at least.

3

Much appreciated! Thank you Pierce :pray:

1 Like