Retrieve Site Performance Data with Google Search Console API on New Item From Multiple RSS Feeds from RSS API

Pipedream makes it easy to connect APIs for Google Search Console, RSS and 2,900+ other apps remarkably fast.

Trigger workflow on

New Item From Multiple RSS Feeds from the RSS API

Next, do this

Retrieve Site Performance Data with the Google Search Console API

No credit card required

▶

Watch us build a workflow

8 min

Watch now ➜

Trusted by 1,000,000+ developers from startups to Fortune 500 companies

Developers ♥ Pipedream

Getting Started#

This integration creates a workflow with a RSS trigger and Google Search Console action. When you configure and deploy the workflow, it will run on Pipedream's servers 24x7 for free.

Select this integration
Configure the New Item From Multiple RSS Feeds trigger
1. Connect your RSS account
2. Configure timer
3. Configure Feed URLs
4. Optional- Configure Max per Feed
Configure the Retrieve Site Performance Data action
1. Connect your Google Search Console account
2. Select a Site
3. Configure Start Date (YYYY-MM-DD)
4. Configure End Date (YYYY-MM-DD)
5. Optional- Select one or more Dimensions
6. Optional- Select a Search Type
7. Optional- Select a Aggregation Type
8. Optional- Configure Max Rows
9. Optional- Configure Start Row
10. Optional- Configure Subdomain Filter
11. Optional- Select a Filter Dimension
12. Optional- Select a Filter Operator
13. Optional- Configure Advanced Dimension Filters
14. Optional- Select a Data State
Deploy the workflow
Send a test event to validate your setup
Turn on the trigger

Details#

This integration uses pre-built, source-available components from Pipedream's GitHub repo. These components are developed by Pipedream and the community, and verified and maintained by Pipedream.

To contribute an update to an existing component or create a new component, create a PR on GitHub. If you're new to Pipedream component development, you can start with quickstarts for trigger span and action development, and then review the component API reference.

Trigger#

New Item From Multiple RSS Feeds on RSS

Description:Emit new items from multiple RSS feeds

Version:1.2.9

Key:rss-new-item-from-multiple-feeds

View on GitHub

RSS Overview#

The RSS app allows users to automatically fetch and parse updates from web feeds. This functionality is pivotal for staying abreast of content changes or updates from websites, blogs, and news outlets that offer RSS feeds. With Pipedream, you can harness the RSS API to trigger workflows that enable a broad range of automations, like content aggregation, monitoring for specific keywords, notifications, and data synchronization across platforms.

Trigger Code#

import { defineSource } from "@pipedream/types";
import rss from "../../app/rss.app.mjs";
import rssCommon from "../common/common.mjs";
export default defineSource({
    ...rssCommon,
    key: "rss-new-item-from-multiple-feeds",
    name: "New Item From Multiple RSS Feeds",
    type: "source",
    description: "Emit new items from multiple RSS feeds",
    version: "1.2.9",
    props: {
        ...rssCommon.props,
        urls: {
            propDefinition: [
                rss,
                "urls",
            ],
            description: "Enter one or multiple URLs from any public RSS feed. To avoid timeouts, 5 or less URLs is recommended.",
        },
        max: {
            type: "integer",
            label: "Max per Feed",
            description: "Maximum number of posts per feed to retrieve at one time. Defaults to 20.",
            optional: true,
            default: 20,
        },
    },
    dedupe: "unique",
    hooks: {
        async activate() {
            // Try to parse the feed one time to confirm we can fetch and parse.
            // The code will throw any errors to the user.
            for (const url of this.urls) {
                await this.rss.fetchAndParseFeed(url);
            }
        },
    },
    async run() {
        const items = [];
        for (const url of this.urls) {
            const feedItems = (await this.rss.fetchAndParseFeed(url))?.slice(0, this.max);
            console.log(`Retrieved items from ${url}`);
            items.push(...feedItems);
        }
        this.rss.sortItems(items).forEach((item) => {
            const meta = this.generateMeta(item);
            this.$emit(item, meta);
        });
    },
});

Trigger Configuration#

This component may be configured based on the props defined in the component code. Pipedream automatically prompts for input values in the UI and CLI.

Label	Prop	Type	Description
RSS	`rss`	`app`	This component uses the RSS app.
	`timer`	`$.interface.timer`	How often you want to poll the feed for new items
Feed URLs	`urls`	`string[]`	Enter one or multiple URLs from any public RSS feed. To avoid timeouts, 5 or less URLs is recommended.
Max per Feed	`max`	`integer`	Maximum number of posts per feed to retrieve at one time. Defaults to 20.

Trigger Authentication#

The RSS API does not require authentication.

About RSS#

Real Simple Syndication

Action#

Retrieve Site Performance Data on Google Search Console

Description:Fetches search analytics from Google Search Console for a verified site.

Version:0.0.4

Key:google_search_console-retrieve-site-performance-data

View on GitHub

Google Search Console Overview#

The Google Search Console API opens a treasure trove of data and insights about your website's presence in Google Search results. You can get detailed reports on your site's search traffic, manage and test your site's sitemaps and robots.txt files, and see which queries bring users to your site. On Pipedream, utilize this API to automate checks on site performance, integrate with other tools for deeper analysis, or keep tabs on your SEO strategy's effectiveness.

Working with Domain Properties and Subdomains

Google Search Console distinguishes between URL properties and Domain properties:

URL properties are specific site URLs (e.g., https://example.com or https://www.example.com)
Domain properties include all subdomains and protocols (e.g., sc-domain:example.com)

When working with subdomains:

Select the domain property from the dropdown (e.g., sc-domain:example.com)
Enter the subdomain URL in the "Subdomain Filter" field (e.g., https://mcp.example.com)
By default, this will filter for pages containing that subdomain URL, including all subpages like https://mcp.example.com/app/slack

This approach ensures you can access subdomain data even if the subdomain isn't individually verified in Search Console.

Important: Getting Data for Individual Pages

To see data broken down by individual pages (rather than just aggregate data):

Add "page" to your dimensions list
This will return separate rows for each page, rather than a single aggregated row

For advanced filtering needs, you can also:

Change the filter dimension (page, query, country, etc.)
Change the filter operator (contains, equals, etc.)
Or use the advanced filters for complete customization

Example Use Cases

SEO Performance Report to Slack: Automate daily or weekly SEO performance reports. Use the Google Search Console API to fetch search analytics data, then send a summary report to a Slack channel, keeping the team informed about trends, keyword rankings, and click-through rates.
Sync Search Results with Google Sheets: Create a workflow that periodically pulls data from the Google Search Console API and adds it to a Google Sheet. This is useful for maintaining an evolving dataset for deeper analysis, historical reference, or sharing insights across teams without giving direct access to the Search Console.
Automatic Sitemap Submission: Set up a Pipedream workflow that triggers whenever a new sitemap is generated in your content management system (CMS). The workflow can then automatically submit the sitemap to Google Search Console via API, ensuring Google has the latest structure of your site for crawling and indexing.

Action Code#

import googleSearchConsole from "../../google_search_console.app.mjs";
import { trimIfString } from "../../common/utils.mjs";

export default {
  name: "Retrieve Site Performance Data",
  description: "Fetches search analytics from Google Search Console for a verified site.",
  key: "google_search_console-retrieve-site-performance-data",
  version: "0.0.4",
  annotations: {
    destructiveHint: false,
    openWorldHint: true,
    readOnlyHint: true,
  },
  type: "action",
  props: {
    googleSearchConsole,
    siteUrl: {
      propDefinition: [
        googleSearchConsole,
        "siteUrl",
      ],
      description: "Select a verified site from your Google Search Console. For subdomains, select the domain property and use dimension filters.",
    },
    startDate: {
      type: "string",
      label: "Start Date (YYYY-MM-DD)",
      description: "Start date of the range for which to retrieve site performance data",
    },
    endDate: {
      type: "string",
      label: "End Date (YYYY-MM-DD)",
      description: "End date of the range for which to retrieve site performance data",
    },
    dimensions: {
      type: "string[]",
      label: "Dimensions",
      optional: true,
      description: "e.g. ['query', 'page', 'country', 'device']",
      options: [
        "country",
        "device",
        "page",
        "query",
        "searchAppearance",
        "date",
      ],
    },
    searchType: {
      type: "string",
      label: "Search Type",
      description: "The type of search to use",
      optional: true,
      options: [
        "web",
        "image",
        "video",
        "news",
        "googleNews",
        "discover",
      ],
      default: "web",
    },
    aggregationType: {
      type: "string",
      label: "Aggregation Type",
      description: "The aggregation type to use",
      optional: true,
      options: [
        "auto",
        "byPage",
      ],
    },
    rowLimit: {
      type: "integer",
      label: "Max Rows",
      description: "Max number of rows to return",
      default: 10,
      optional: true,
    },
    startRow: {
      type: "integer",
      label: "Start Row",
      description: "Start row (for pagination)",
      optional: true,
    },
    subdomainFilter: {
      type: "string",
      label: "Subdomain Filter",
      optional: true,
      description: "Filter results to a specific subdomain when using a domain property (e.g., `https://subdomain.example.com`). This will include all subpages of the subdomain.",
    },
    filterDimension: {
      type: "string",
      label: "Filter Dimension",
      optional: true,
      description: "Dimension to filter by (defaults to page when subdomain filter is used). Using 'page' will match the subdomain and all its subpages.",
      options: [
        "country",
        "device",
        "page",
        "query",
      ],
      default: "page",
    },
    filterOperator: {
      type: "string",
      label: "Filter Operator",
      optional: true,
      description: "Operator to use for filtering (defaults to contains when subdomain filter is used)",
      options: [
        "contains",
        "equals",
        "notContains",
        "notEquals",
        "includingRegex",
        "excludingRegex",
      ],
      default: "contains",
    },
    advancedDimensionFilters: {
      type: "object",
      label: "Advanced Dimension Filters",
      optional: true,
      description: "For advanced use cases: custom dimension filter groups following Search Console API structure.",
    },
    dataState: {
      type: "string",
      label: "Data State",
      description: "The data state to use",
      optional: true,
      options: [
        "all",
        "final",
      ],
      default: "final",
    },
  },
  async run({ $ }) {
    const {
      googleSearchConsole,
      siteUrl,
      subdomainFilter,
      filterDimension,
      filterOperator,
      advancedDimensionFilters,
      ...fields
    } = this;

    const body = Object.entries(fields).reduce((acc, [
      key,
      value,
    ]) => {
      acc[key] = trimIfString(value);
      return acc;
    }, {});

    // Build dimension filters based on user input
    let dimensionFilterGroups;

    if (subdomainFilter) {
      // If user provided a subdomain filter, create the filter structure
      dimensionFilterGroups = {
        filterGroups: [
          {
            filters: [
              {
                dimension: filterDimension || "page",
                operator: filterOperator || "contains",
                expression: subdomainFilter,
              },
            ],
          },
        ],
      };
    } else if (advancedDimensionFilters) {
      // If user provided advanced filters, use those
      dimensionFilterGroups = googleSearchConsole.parseIfJsonString(advancedDimensionFilters);
    }

    let response;
    try {
      response = await googleSearchConsole.getSitePerformanceData({
        $,
        url: siteUrl,
        data: {
          ...body,
          dimensionFilterGroups,
        },
      });
    } catch (error) {
      // Identify if the error was thrown by internal validation or by the API call
      const thrower = googleSearchConsole.checkWhoThrewError(error);

      // Add more helpful error messages for common 403 errors
      if (error.response?.status === 403) {
        const message = "Access denied. If you're trying to access a subdomain, select the domain property (sc-domain:example.com) and use the subdomain filter to filter for your subdomain.";
        throw new Error(`Failed to fetch data: ${message}`);
      }

      throw new Error(`Failed to fetch data (${thrower.whoThrew} error): ${error.message}`);
    }

    const rowCount = response.rows?.length || 0;
    $.export("$summary", `Fetched ${rowCount} ${rowCount === 1
      ? "row"
      : "rows"} of data.`);
    return response;
  },
};

Action Configuration#

This component may be configured based on the props defined in the component code. Pipedream automatically prompts for input values in the UI.

Label	Prop	Type	Description
Google Search Console	`googleSearchConsole`	`app`	This component uses the Google Search Console app.
Site	`siteUrl`	`string`	Select a value from the drop down menu.
Start Date (YYYY-MM-DD)	`startDate`	`string`	Start date of the range for which to retrieve site performance data
End Date (YYYY-MM-DD)	`endDate`	`string`	End date of the range for which to retrieve site performance data
Dimensions	`dimensions`	`string[]`	Select a value from the drop down menu:`countrydevicepagequerysearchAppearancedate`
Search Type	`searchType`	`string`	Select a value from the drop down menu:`webimagevideonewsgoogleNewsdiscover`
Aggregation Type	`aggregationType`	`string`	Select a value from the drop down menu:`autobyPage`
Max Rows	`rowLimit`	`integer`	Max number of rows to return
Start Row	`startRow`	`integer`	Start row (for pagination)
Subdomain Filter	`subdomainFilter`	`string`	Filter results to a specific subdomain when using a domain property (e.g., `https://subdomain.example.com`). This will include all subpages of the subdomain.
Filter Dimension	`filterDimension`	`string`	Select a value from the drop down menu:`countrydevicepagequery`
Filter Operator	`filterOperator`	`string`	Select a value from the drop down menu:`containsequalsnotContainsnotEqualsincludingRegexexcludingRegex`
Advanced Dimension Filters	`advancedDimensionFilters`	`object`	For advanced use cases: custom dimension filter groups following Search Console API structure.
Data State	`dataState`	`string`	Select a value from the drop down menu:`allfinal`

Action Authentication#

Google Search Console uses OAuth authentication. When you connect your Google Search Console account, Pipedream will open a popup window where you can sign into Google Search Console and grant Pipedream permission to connect to your account. Pipedream securely stores and automatically refreshes the OAuth tokens so you can easily authenticate any Google Search Console API.

Pipedream requests the following authorization scopes when you connect your account:

emailprofilehttps://www.googleapis.com/auth/webmastershttps://www.googleapis.com/auth/webmasters.readonlyhttps://www.googleapis.com/auth/siteverificationhttps://www.googleapis.com/auth/indexing