Classify Text with Google Vertex AI API on Job Run Completed (Instant) from dbt Cloud API

Pipedream makes it easy to connect APIs for Google Vertex AI, dbt Cloud and 2,500+ other apps remarkably fast.

Trigger workflow on

Job Run Completed (Instant) from the dbt Cloud API

Next, do this

Classify Text with the Google Vertex AI API

No credit card required

▶

Watch us build a workflow

8 min

Watch now ➜

Trusted by 1,000,000+ developers from startups to Fortune 500 companies

Developers ♥ Pipedream

Getting Started#

This integration creates a workflow with a dbt Cloud trigger and Google Vertex AI action. When you configure and deploy the workflow, it will run on Pipedream's servers 24x7 for free.

Select this integration
Configure the Job Run Completed (Instant) trigger
1. Connect your dbt Cloud account
2. Select a Account
Configure the Classify Text action
1. Connect your Google Vertex AI account
2. Select a Project ID
3. Configure Categories
4. Configure Text
Deploy the workflow
Send a test event to validate your setup
Turn on the trigger

Details#

This integration uses pre-built, source-available components from Pipedream's GitHub repo. These components are developed by Pipedream and the community, and verified and maintained by Pipedream.

To contribute an update to an existing component or create a new component, create a PR on GitHub. If you're new to Pipedream component development, you can start with quickstarts for trigger span and action development, and then review the component API reference.

Trigger#

Job Run Completed (Instant) on dbt Cloud

Description:Emit new event when a job run has completed.

Version:0.0.2

Key:dbt-job-run-completed

View on GitHub

dbt Cloud Overview#

The dbt Cloud API allows users to initiate jobs, check on their status, and interact with dbt Cloud programmatically. On Pipedream, you can harness this functionality to automate workflows, such as triggering dbt runs, monitoring your data transformation jobs, and integrating dbt Cloud with other data services. By leveraging Pipedream's serverless platform, you can create custom workflows that act on dbt Cloud events or use the dbt Cloud API to manage your data transformation processes seamlessly.

Trigger Code#

import common from "../common/common.mjs";
import sampleEmit from "./test-event.mjs";

export default {
  ...common,
  key: "dbt-job-run-completed",
  name: "Job Run Completed (Instant)",
  description: "Emit new event when a job run has completed.",
  version: "0.0.2",
  type: "source",
  dedupe: "unique",
  methods: {
    ...common.methods,
    getEventTypes() {
      return [
        "job.run.completed",
      ];
    },
    generateMeta(body) {
      return {
        id: body.eventId,
        summary: `Job "${body.data.jobName}"" completed`,
        ts: Date.parse(body.data.runFinishedAt),
      };
    },
  },
  sampleEmit,
};

Trigger Configuration#

This component may be configured based on the props defined in the component code. Pipedream automatically prompts for input values in the UI and CLI.

Label	Prop	Type	Description
dbt Cloud	`dbt`	`app`	This component uses the dbt Cloud app.
N/A	`db`	`$.service.db`	This component uses `$.service.db` to maintain state between executions.
N/A	`http`	`$.interface.http`	This component uses `$.interface.http` to generate a unique URL when the component is first instantiated. Each request to the URL will trigger the `run()` method of the component.
Account	`accountId`	`string`	Select a value from the drop down menu.

Trigger Authentication#

dbt Cloud uses API keys for authentication. When you connect your dbt Cloud account, Pipedream securely stores the keys so you can easily authenticate to dbt Cloud APIs in both code and no-code steps.

About dbt Cloud#

dbt is an intuitive, collaborative platform that lets you reliably transform data using SQL and Python code.

Action#

Classify Text on Google Vertex AI

Description:Groups a provided text into predefined categories. [See the documentation](https://cloud.google.com/vertex-ai/docs/reference/rest/v1/projects.locations.publishers.models/generateContent)

Version:0.0.1

Key:google_vertex_ai-classify-text

View on GitHub

Google Vertex AI Overview#

With the Google Vertex AI API, you can tap into a robust suite of AI tools offered by Google Cloud to build, deploy, and scale machine learning models. Whether you're processing data, training custom models, or using pre-trained ones, Vertex AI provides a unified platform for AI development. In Pipedream, you can create serverless workflows that interact with Vertex AI, allowing you to automate tasks like model training, prediction, and resource management without provisioning your own infrastructure.

Action Code#

import vertexAi from "../../google_vertex_ai.app.mjs";

export default {
  key: "google_vertex_ai-classify-text",
  name: "Classify Text",
  description: "Groups a provided text into predefined categories. [See the documentation](https://cloud.google.com/vertex-ai/docs/reference/rest/v1/projects.locations.publishers.models/generateContent)",
  version: "0.0.1",
  type: "action",
  props: {
    vertexAi,
    projectId: {
      propDefinition: [
        vertexAi,
        "projectId",
      ],
    },
    categories: {
      type: "string[]",
      label: "Categories",
      description: "The categories to group the text into",
    },
    text: {
      type: "string",
      label: "Text",
      description: "The text to classify",
    },
  },
  async run({ $ }) {
    const categories = (this.categories.map((category) => `\n- ${category}`)).join("");
    const response = await this.vertexAi.generateContent({
      $,
      projectId: this.projectId,
      model: "gemini-1.0-pro",
      data: {
        contents: [
          {
            role: "user",
            parts: [
              {
                text: `Define the categories for the text below?\nOptions:${categories}\n\nText: ${this.text}`,
              },
            ],
          },
        ],
      },
    });
    $.export("$summary", "Text classified into categories");
    return response;
  },
};

Action Configuration#

This component may be configured based on the props defined in the component code. Pipedream automatically prompts for input values in the UI.

Label	Prop	Type	Description
Google Vertex AI	`vertexAi`	`app`	This component uses the Google Vertex AI app.
Project ID	`projectId`	`string`	Select a value from the drop down menu.
Categories	`categories`	`string[]`	The categories to group the text into
Text	`text`	`string`	The text to classify

Action Authentication#

Google Vertex AI uses OAuth authentication. When you connect your Google Vertex AI account, Pipedream will open a popup window where you can sign into Google Vertex AI and grant Pipedream permission to connect to your account. Pipedream securely stores and automatically refreshes the OAuth tokens so you can easily authenticate any Google Vertex AI API.

Pipedream requests the following authorization scopes when you connect your account:

emailprofilehttps://www.googleapis.com/auth/cloud-platform