Get Key-Value Store Record with Apify API

Pipedream makes it easy to connect APIs for Apify and 3,000+ other apps remarkably fast.

Trigger workflow on

HTTP requests, schedules and app events

Next, do this

Get Key-Value Store Record with the Apify API

No credit card required

▶

Watch us build a workflow

8 min

Watch now ➜

Trusted by 1,000,000+ developers from startups to Fortune 500 companies

Developers ♥ Pipedream

Getting Started#

Create a workflow to Get Key-Value Store Record with the Apify API. When you configure and deploy the workflow, it will run on Pipedream's servers 24x7 for free.

Configure the Get Key-Value Store Record action
1. Connect your Apify account
2. Select a Key-Value Store ID
3. Configure Record Key
Select a trigger to run your workflow on HTTP requests, schedules or app events
Deploy the workflow
Send a test event to validate your setup
Turn on the trigger

Details#

This is a pre-built, source-available component from Pipedream's GitHub repo. The component is developed by Pipedream and the community, and verified and maintained by Pipedream.

To contribute an update to an existing component or create a new component, create a PR on GitHub. If you're new to Pipedream component development, you can start with quickstarts for trigger span and action development, and then review the component API reference.

Get Key-Value Store Record on Apify

Description:Gets a record from a key-value store. [See the documentation](https://docs.apify.com/api/v2/key-value-store-record-get)

Version:0.0.1

Key:apify-get-kvs-record

View on GitHub

Code#

import axios from "axios";
import apify from "../../apify.app.mjs";

export default {
  key: "apify-get-kvs-record",
  name: "Get Key-Value Store Record",
  description: "Gets a record from a key-value store. [See the documentation](https://docs.apify.com/api/v2/key-value-store-record-get)",
  version: "0.0.1",
  type: "action",
  annotations: {
    destructiveHint: false,
    openWorldHint: true,
    readOnlyHint: true,
  },
  props: {
    apify,
    keyValueStoreId: {
      propDefinition: [
        apify,
        "keyValueStoreId",
        () => ({
          unnamed: false,
        }),
      ],
    },
    key: {
      type: "string",
      label: "Record Key",
      description: "If the record is a valid JSON object, the output will include all parsed attributes as individual fields. If the record is any other file type (such as PDFs, images, or plain text), the output will instead be a file reference.",
    },
  },
  async run({ $ }) {
    const token = this.apify.getAuthToken();

    // HEAD request to get size and content-type
    const recordUrl = `https://api.apify.com/v2/key-value-stores/${this.keyValueStoreId}/records/${encodeURIComponent(this.key)}`;
    const headResp = await axios({
      url: recordUrl,
      method: "HEAD",
      headers: {
        Authorization: `Bearer ${token}`,
      },
      validateStatus: () => true,
    });

    if (headResp.status === 404) {
      $.export("$summary", "Record not found");
      return {};
    }
    if (headResp.status >= 400) {
      throw new Error(`Error retrieving record: ${headResp.statusText}`);
    }

    const contentType = headResp.headers["content-type"];
    const contentLength = parseInt(headResp.headers["content-length"] || "0");

    const PARSEABLE_MIMES = [
      "application/json",
      "text/plain",
    ];
    const MAX_INLINE_SIZE = 10 * 1000 * 1000;

    // If parseable and small enough, get data via Apify client
    if (PARSEABLE_MIMES.some((pattern) => contentType.includes(pattern))
        && contentLength < MAX_INLINE_SIZE) {
      const record = await this.apify.getKVSRecord(this.keyValueStoreId, this.key);
      const data = record?.value;

      if (typeof data === "object" && !Array.isArray(data) && data !== null) {
        $.export("$summary", "Retrieved JSON record");
        return data;
      } else {
        $.export("$summary", "Retrieved text record");
        return {
          value: data ?? null,
        };
      }
    } else {
      // For large/binary files, return signed URL via Apify client
      const signedUrl = await this.apify.getKVSRecordUrl(this.keyValueStoreId, this.key);

      $.export("$summary", "Retrieved file pointer");
      return {
        contentType,
        value: signedUrl,
      };
    }
  },
};

Configuration#

This component may be configured based on the props defined in the component code. Pipedream automatically prompts for input values in the UI and CLI.

Label	Prop	Type	Description
Apify	`apify`	`app`	This component uses the Apify app.
Key-Value Store ID	`keyValueStoreId`	`string`	Select a value from the drop down menu.
Record Key	`key`	`string`	If the record is a valid JSON object, the output will include all parsed attributes as individual fields. If the record is any other file type (such as PDFs, images, or plain text), the output will instead be a file reference.

Authentication#

Apify uses API keys for authentication. When you connect your Apify account, Pipedream securely stores the keys so you can easily authenticate to Apify APIs in both code and no-code steps.

About Apify#

Apify is the place to find, develop, buy and run cloud programs called actors. Use actors to turn any website into an API.

More Ways to Use Apify#

Triggers#

New Finished Actor Run (Instant) from the Apify API

Emit new event when a selected Actor is run and finishes.

Try it

New Finished Task Run (Instant) from the Apify API

Emit new event when a selected task is run and finishes.

Try it

Actions#

Get Dataset Items with the Apify API

Returns data stored in a dataset. See the documentation

Try it

Run Actor with the Apify API

Performs an execution of a selected Actor in Apify. See the documentation

Try it

Run Task with the Apify API

Run a specific task and optionally wait for its termination. Not for use with Pipedream Connect. See the documentation

Try it

Run Task Synchronously with the Apify API

Run a specific task and return its dataset items. See the documentation

Try it

Scrape Single URL with the Apify API

Executes a scraper on a specific website and returns its content as HTML. This action is perfect for extracting content from a single page. See the documentation

Try it