Get Record Counts by Field with ServiceNow API on New Analysis Complete from Insites API

Pipedream makes it easy to connect APIs for ServiceNow, Insites and 3,000+ other apps remarkably fast.

Trigger workflow on

New Analysis Complete from the Insites API

Next, do this

Get Record Counts by Field with the ServiceNow API

No credit card required

▶

Watch us build a workflow

8 min

Watch now ➜

Trusted by 1,000,000+ developers from startups to Fortune 500 companies

Developers ♥ Pipedream

Getting Started#

This integration creates a workflow with a Insites trigger and ServiceNow action. When you configure and deploy the workflow, it will run on Pipedream's servers 24x7 for free.

Select this integration
Configure the New Analysis Complete trigger
1. Connect your Insites account
2. Configure timer
3. Optional- Configure Filter
4. Optional- Configure Include Historic
5. Optional- Configure List ID
Configure the Get Record Counts by Field action
1. Connect your ServiceNow account
2. Select a Table
3. Configure Group By Field
4. Optional- Configure Count
5. Optional- Configure Query
6. Configure aggregateInfo
7. Optional- Configure Average Aggregate Fields
8. Optional- Configure Minimum Aggregate Fields
9. Optional- Configure Maximum Aggregate Fields
10. Optional- Configure Sum Aggregate Fields
11. Optional- Configure Having Query
12. Optional- Select a Response Data Format
13. Optional- Configure Order By
Deploy the workflow
Send a test event to validate your setup
Turn on the trigger

Details#

This integration uses pre-built, source-available components from Pipedream's GitHub repo. These components are developed by Pipedream and the community, and verified and maintained by Pipedream.

To contribute an update to an existing component or create a new component, create a PR on GitHub. If you're new to Pipedream component development, you can start with quickstarts for trigger span and action development, and then review the component API reference.

Trigger#

New Analysis Complete on Insites

Description:Emit new event when a new analysis is completed. [See the documentation](https://help.insites.com/en/articles/7994946-report-api#h_e59622a8e7)

Version:0.0.2

Key:insites-analysis-complete

View on GitHub

Insites Overview#

The Insites API offers a suite of tools for website testing and monitoring, enabling users to automate the process of checking website quality, performance, and compliance with SEO and accessibility standards. With Pipedream, you can harness this API to create workflows that trigger on events across your apps, perform actions based on website analysis results, and automate repetitive tasks that ensure your website maintains high standards for your users.

Trigger Code#

import {
  ConfigurationError,
  DEFAULT_POLLING_SOURCE_TIMER_INTERVAL,
} from "@pipedream/platform";
import app from "../../insites.app.mjs";

export default {
  dedupe: "unique",
  type: "source",
  key: "insites-analysis-complete",
  name: "New Analysis Complete",
  description: "Emit new event when a new analysis is completed. [See the documentation](https://help.insites.com/en/articles/7994946-report-api#h_e59622a8e7)",
  version: "0.0.2",
  props: {
    app,
    db: "$.service.db",
    timer: {
      type: "$.interface.timer",
      default: {
        intervalSeconds: DEFAULT_POLLING_SOURCE_TIMER_INTERVAL,
      },
    },
    filter: {
      type: "string",
      label: "Filter",
      description: "JSON encoded string - attributes by which the reports should be filtered by. [See the documentation for more details](https://help.insites.com/en/articles/7994946-report-api#h_e59622a8e7). Example: `[{\"operator\":\"equal\",\"property\":\"mobile.is_mobile\",\"targetValue\":false}]`",
      optional: true,
    },
    includeHistoric: {
      type: "boolean",
      label: "Include Historic",
      description: "Whether the results should contain old versions of the business reports.",
      optional: true,
    },
    listId: {
      type: "string",
      label: "List ID",
      description: "Will filter the results and return only those in the given list. Note: list ID should be the 32-character hexadecimal ID provided by Insites (not the list name which is assigned by the user).",
      optional: true,
    },
  },
  methods: {
    setLastExecutionTime(time) {
      this.db.set("lastExecutionTime", time);
    },
    getLastExecutionTime() {
      const lastExecutionTime = this.db.get("lastExecutionTime");
      if (!lastExecutionTime) {
        const YESTERDAY = 86400000;
        return new Date(Date.now() - YESTERDAY).toISOString();
      }
      return new Date(lastExecutionTime).toISOString();
    },
    validateJsonString(obj) {
      if (typeof obj !== "string") {
        return obj;
      }
      try {
        return JSON.parse(obj);
      } catch (err) {
        throw new ConfigurationError("Filter must be a valid JSON string.");
      }
    },
    emitEvent(report) {
      const meta = this.generateMeta(report);
      this.$emit(report, meta);
    },
    generateMeta(report) {
      return {
        id: report.report_id,
        summary: report.domain,
        ts: report.meta.report_completed_at,
      };
    },
  },
  async run() {
    let userFilter = [];
    if (this.filter) {
      userFilter = this.validateJsonString(this.filter);
    }
    const lastExecutionTime = this.getLastExecutionTime();
    const filter = [
      {
        "property": "report.meta.report_completed_at",
        "operator": "more_than",
        "targetValue": lastExecutionTime,
        "numeric": "false",
      },
    ];

    const data = [];
    const LIMIT = 100;
    let offset = 0;
    while (true) {
      const res = await this.app.searchAllReports({
        filter: JSON.stringify(filter.concat(userFilter)),
        offset,
        limit: LIMIT,
        include_historic: this.includeHistoric,
        list_id: this.listId,
      });
      if (!res.reports || res.reports.length === 0) {
        break;
      }
      data.push(...res.reports);
      offset += LIMIT;
    }

    for (const report of data.reverse()) {
      this.emitEvent(report);
    }
  },
};

Trigger Configuration#

This component may be configured based on the props defined in the component code. Pipedream automatically prompts for input values in the UI and CLI.

Label	Prop	Type	Description
Insites	`app`	`app`	This component uses the Insites app.
N/A	`db`	`$.service.db`	This component uses `$.service.db` to maintain state between executions.
	`timer`	`$.interface.timer`
Filter	`filter`	`string`	JSON encoded string - attributes by which the reports should be filtered by. See the documentation for more details. Example: `[{"operator":"equal","property":"mobile.is_mobile","targetValue":false}]`
Include Historic	`includeHistoric`	`boolean`	Whether the results should contain old versions of the business reports.
List ID	`listId`	`string`	Will filter the results and return only those in the given list. Note: list ID should be the 32-character hexadecimal ID provided by Insites (not the list name which is assigned by the user).

Trigger Authentication#

Insites uses API keys for authentication. When you connect your Insites account, Pipedream securely stores the keys so you can easily authenticate to Insites APIs in both code and no-code steps.

About Insites#

Insites lets you audit any website's online presence in under 60 seconds.

Action#

Get Record Counts by Field on ServiceNow

Description:Retrieves the count of records grouped by a specified field from a ServiceNow table. [See the documentation](https://www.servicenow.com/docs/bundle/zurich-api-reference/page/integrate/inbound-rest/concept/c_AggregateAPI.html#title_aggregate-GET-stats)

Version:0.0.2

Key:servicenow-get-record-counts-by-field

View on GitHub

ServiceNow Overview#

The ServiceNow API lets developers access and manipulate records, manage workflows, and integrate with other services on its IT service management platform. These capabilities support automating tasks, syncing data across platforms, and boosting operational efficiencies.

Action Code#

import { ConfigurationError } from "@pipedream/platform";
import servicenow from "../../servicenow.app.mjs";

export default {
  key: "servicenow-get-record-counts-by-field",
  name: "Get Record Counts by Field",
  description: "Retrieves the count of records grouped by a specified field from a ServiceNow table. [See the documentation](https://www.servicenow.com/docs/bundle/zurich-api-reference/page/integrate/inbound-rest/concept/c_AggregateAPI.html#title_aggregate-GET-stats)",
  version: "0.0.2",
  annotations: {
    destructiveHint: false,
    openWorldHint: true,
    readOnlyHint: true,
  },
  type: "action",
  props: {
    servicenow,
    table: {
      propDefinition: [
        servicenow,
        "table",
      ],
    },
    groupByField: {
      type: "string",
      label: "Group By Field",
      description: "The field to group records by (e.g., `priority`, `state`, `category`)",
    },
    count: {
      type: "boolean",
      label: "Count",
      description: "If true, returns the number of records returned by the query",
      optional: true,
      default: true,
    },
    query: {
      label: "Query",
      type: "string",
      description: "An [encoded query string](https://www.servicenow.com/docs/bundle/zurich-platform-user-interface/page/use/using-lists/concept/c_EncodedQueryStrings.html) to filter records before aggregation (e.g., `active=true^priority=1`)",
      optional: true,
    },
    aggregateInfo: {
      type: "alert",
      alertType: "info",
      content: "You must provide at least one Aggregate Field. [See the documentation](https://www.servicenow.com/docs/bundle/zurich-api-reference/page/integrate/inbound-rest/concept/c_AggregateAPI.html#title_aggregate-GET-stats) for more information.",
    },
    avgFields: {
      type: "string[]",
      label: "Average Aggregate Fields",
      description: "Numeric fields to calculate averages for (e.g., `reassignment_count`, `reopen_count`)",
      optional: true,
    },
    minFields: {
      type: "string[]",
      label: "Minimum Aggregate Fields",
      description: "Numeric fields to find minimum values for",
      optional: true,
    },
    maxFields: {
      type: "string[]",
      label: "Maximum Aggregate Fields",
      description: "Numeric fields to find maximum values for",
      optional: true,
    },
    sumFields: {
      type: "string[]",
      label: "Sum Aggregate Fields",
      description: "Numeric fields to calculate sums for",
      optional: true,
    },
    havingQuery: {
      type: "string",
      label: "Having Query",
      description: "Filter the aggregated results (e.g., `COUNT>10` to only show groups with more than 10 records)",
      optional: true,
    },
    responseDataFormat: {
      propDefinition: [
        servicenow,
        "responseDataFormat",
      ],
    },
    orderBy: {
      type: "string",
      label: "Order By",
      description: "Field to sort results by. Prefix with `^ORDER_BY` for ascending or `^ORDER_BYDESC` for descending (e.g., `^ORDER_BYDESC` + field name)",
      optional: true,
    },
  },
  async run({ $ }) {
    const params = {
      sysparm_count: this.count,
      sysparm_query: this.query,
      sysparm_group_by: this.groupByField,
      sysparm_display_value: this.responseDataFormat,
      sysparm_having: this.havingQuery,
      sysparm_order_by: this.orderBy,
    };

    let hasAggregateFields = false;

    // Add aggregate functions for each field type
    if (this.avgFields?.length) {
      params.sysparm_avg_fields = this.avgFields.join?.() || this.avgFields;
      hasAggregateFields = true;
    }
    if (this.minFields?.length) {
      params.sysparm_min_fields = this.minFields.join?.() || this.minFields;
      hasAggregateFields = true;
    }
    if (this.maxFields?.length) {
      params.sysparm_max_fields = this.maxFields.join?.() || this.maxFields;
      hasAggregateFields = true;
    }
    if (this.sumFields?.length) {
      params.sysparm_sum_fields = this.sumFields.join?.() || this.sumFields;
      hasAggregateFields = true;
    }

    if (!hasAggregateFields) {
      throw new ConfigurationError("You must provide at least one Aggregate Field. [See the documentation](https://www.servicenow.com/docs/bundle/zurich-api-reference/page/integrate/inbound-rest/concept/c_AggregateAPI.html#title_aggregate-GET-stats) for more information.");
    }

    const response = await this.servicenow.getRecordCountsByField({
      $,
      table: this.table,
      params,
    });

    $.export("$summary", `Successfully retrieved ${response?.length || 0} records grouped by field "${this.groupByField}"`);

    return response;
  },
};

Action Configuration#

This component may be configured based on the props defined in the component code. Pipedream automatically prompts for input values in the UI.

Label	Prop	Type	Description
ServiceNow	`servicenow`	`app`	This component uses the ServiceNow app.
Table	`table`	`string`	Select a value from the drop down menu.
Group By Field	`groupByField`	`string`	The field to group records by (e.g., `priority`, `state`, `category`)
Count	`count`	`boolean`	If true, returns the number of records returned by the query
Query	`query`	`string`	An encoded query string to filter records before aggregation (e.g., `active=true^priority=1`)
Average Aggregate Fields	`avgFields`	`string[]`	Numeric fields to calculate averages for (e.g., `reassignment_count`, `reopen_count`)
Minimum Aggregate Fields	`minFields`	`string[]`	Numeric fields to find minimum values for
Maximum Aggregate Fields	`maxFields`	`string[]`	Numeric fields to find maximum values for
Sum Aggregate Fields	`sumFields`	`string[]`	Numeric fields to calculate sums for
Having Query	`havingQuery`	`string`	Filter the aggregated results (e.g., `COUNT>10` to only show groups with more than 10 records)
Response Data Format	`responseDataFormat`	`string`	Select a value from the drop down menu:`{ "value": "true", "label": "Returns the display values for all fields" }{ "value": "false", "label": "Returns the actual values from the database" }{ "value": "all", "label": "Returns both actual and display values" }`
Order By	`orderBy`	`string`	Field to sort results by. Prefix with `^ORDER_BY` for ascending or `^ORDER_BYDESC` for descending (e.g., `^ORDER_BYDESC` + field name)

Action Authentication#

ServiceNow uses API keys for authentication. When you connect your ServiceNow account, Pipedream securely stores the keys so you can easily authenticate to ServiceNow APIs in both code and no-code steps.

Please follow the steps in this doc to configure a client application on your ServiceNow instance that will allow Pipedream access to your instance's REST API.

Once you're done, enter the client ID and secret you configured in Step 2, along with your instance name. The instance name is the host portion of your instance's URL: that is, the dev123 in https://dev123.service-now.com/.