Generate Content from Text and Image with Google Gemini API on BigQuery - Query Results from Google Cloud API

Pipedream makes it easy to connect APIs for Google Gemini, Google Cloud and 2,700+ other apps remarkably fast.

Trigger workflow on

BigQuery - Query Results from the Google Cloud API

Next, do this

Generate Content from Text and Image with the Google Gemini API

No credit card required

▶

Watch us build a workflow

8 min

Watch now ➜

Trusted by 1,000,000+ developers from startups to Fortune 500 companies

Developers ♥ Pipedream

Getting Started#

This integration creates a workflow with a Google Cloud trigger and Google Gemini action. When you configure and deploy the workflow, it will run on Pipedream's servers 24x7 for free.

Select this integration
Configure the BigQuery - Query Results trigger
1. Connect your Google Cloud account
2. Configure Polling interval
3. Configure Event Size
4. Select a Dataset ID
5. Configure SQL Query
6. Optional- Configure De-duplication Key
Configure the Generate Content from Text and Image action
1. Connect your Google Gemini account
2. Select a Model
3. Configure Prompt Text
4. Optional- Configure JSON Output
5. Optional- Configure Conversation History
6. Optional- Configure Safety Settings
7. Configure Media File Paths or URLs
8. Optional- Configure syncDir
Deploy the workflow
Send a test event to validate your setup
Turn on the trigger

Details#

This integration uses pre-built, source-available components from Pipedream's GitHub repo. These components are developed by Pipedream and the community, and verified and maintained by Pipedream.

To contribute an update to an existing component or create a new component, create a PR on GitHub. If you're new to Pipedream component development, you can start with quickstarts for trigger span and action development, and then review the component API reference.

Trigger#

BigQuery - Query Results on Google Cloud

Description:Emit new events with the results of an arbitrary query

Version:0.1.5

Key:google_cloud-bigquery-query-results

View on GitHub

Google Cloud Overview#

The Google Cloud API opens a world of possibilities for enhancing cloud operations and automating tasks. It empowers you to manage, scale, and fine-tune various services within the Google Cloud Platform (GCP) programmatically. With Pipedream, you can harness this power to create intricate workflows, trigger cloud functions based on events from other apps, manage resources, and analyze data, all in a serverless environment. The ability to interconnect GCP services with numerous other apps enriches automation, making it easier to synchronize data, streamline development workflows, and deploy applications efficiently.

Trigger Code#

import crypto from "crypto";
import { v4 as uuidv4 } from "uuid";
import common from "../common/bigquery.mjs";

export default {
  ...common,
  key: "google_cloud-bigquery-query-results",
  // eslint-disable-next-line pipedream/source-name
  name: "BigQuery - Query Results",
  description: "Emit new events with the results of an arbitrary query",
  version: "0.1.5",
  dedupe: "unique",
  type: "source",
  props: {
    ...common.props,
    sqlQuery: {
      type: "string",
      label: "SQL Query",
      description: "Your SQL query",
    },
    dedupeKey: {
      type: "string",
      label: "De-duplication Key",
      description: `
        The name of a column in the table to use for deduplication. See [the
        docs](https://github.com/PipedreamHQ/pipedream/tree/master/components/google_cloud/sources/bigquery-query-results#technical-details)
        for more info.
      `,
      optional: true,
    },
  },
  methods: {
    ...common.methods,
    getQueryOpts() {
      return {
        query: this.sqlQuery,
      };
    },
    generateMeta(row, ts) {
      const { [this.dedupeKey]: id = uuidv4() } = row;
      const summary = `New event (ID: ${id})`;
      return {
        id,
        summary,
        ts,
      };
    },
    generateMetaForCollection(rows, ts) {
      const hash = crypto.createHash("sha1");
      rows
        .map((i) => i[this.dedupeKey] || uuidv4())
        .map((i) => i.toString())
        .forEach((i) => hash.update(i));
      const id = hash.digest("base64");
      const summary = `New event (ID: ${id})`;
      return {
        id,
        summary,
        ts,
      };
    },
  },
};

Trigger Configuration#

This component may be configured based on the props defined in the component code. Pipedream automatically prompts for input values in the UI and CLI.

Label	Prop	Type	Description
Google Cloud	`googleCloud`	`app`	This component uses the Google Cloud app.
N/A	`db`	`$.service.db`	This component uses `$.service.db` to maintain state between executions.
Polling interval	`timer`	`$.interface.timer`	How often to run your query
Event Size	`eventSize`	`integer`	The number of rows to include in a single event (by default, emits 1 event per row)
Dataset ID	`datasetId`	`string`	Select a value from the drop down menu.
SQL Query	`sqlQuery`	`string`	Your SQL query
De-duplication Key	`dedupeKey`	`string`	`The name of a column in the table to use for deduplication. See [the docs](https://github.com/PipedreamHQ/pipedream/tree/master/components/google_cloud/sources/bigquery-query-results#technical-details) for more info.`

Trigger Authentication#

Google Cloud uses API keys for authentication. When you connect your Google Cloud account, Pipedream securely stores the keys so you can easily authenticate to Google Cloud APIs in both code and no-code steps.

Create a service account in GCP and set the permissions you need for Pipedream workflows.
Generate a service account key
Download the key details in JSON format
Upload the key below.

About Google Cloud#

The Google Cloud Platform, including BigQuery

Action#

Generate Content from Text and Image on Google Gemini

Description:Generates content from both text and image input using the Gemini API. [See the documentation](https://ai.google.dev/tutorials/rest_quickstart#text-and-image_input)

Version:1.0.1

Key:google_gemini-generate-content-from-text-and-image

View on GitHub

Google Gemini Overview#

The Google Gemini API is a cutting-edge tool from Google that enables developers to leverage AI models like Imagen and MusicLM to create and manipulate images and music based on textual descriptions. With Pipedream, you can harness this API to automate workflows that integrate AI-generated content into a variety of applications, from generating visuals for social media posts to composing background music for videos. Pipedream's serverless platform allows you to connect Google Gemini API with other apps to create complex, event-driven workflows without managing infrastructure.

Action Code#

import {
  ConfigurationError, getFileStreamAndMetadata,
} from "@pipedream/platform";
import mime from "mime";
import common from "../common/generate-content.mjs";
import utils from "../../common/utils.mjs";

export default {
  ...common,
  key: "google_gemini-generate-content-from-text-and-image",
  name: "Generate Content from Text and Image",
  description: "Generates content from both text and image input using the Gemini API. [See the documentation](https://ai.google.dev/tutorials/rest_quickstart#text-and-image_input)",
  version: "1.0.1",
  type: "action",
  props: {
    ...common.props,
    mediaFiles: {
      type: "string[]",
      label: "Media File Paths or URLs",
      description: "A list of file paths from the `/tmp` directory or URLs for the media to process.",
    },
    syncDir: {
      type: "dir",
      accessMode: "read",
      sync: true,
      optional: true,
    },
  },
  methods: {
    ...common.methods,
    streamToBase64(stream) {
      return new Promise((resolve, reject) => {
        const chunks = [];
        stream.on("data", (chunk) => chunks.push(chunk));
        stream.on("end", () => {
          const buffer = Buffer.concat(chunks);
          resolve(buffer.toString("base64"));
        });
        stream.on("error", reject);
      });
    },
    async fileToGenerativePart(file) {
      if (!file) {
        return;
      }

      const {
        stream, metadata,
      } = await getFileStreamAndMetadata(file);

      const data = await this.streamToBase64(stream);

      return {
        inline_data: {
          mime_type: metadata.contentType ?? mime.getType(metadata.name),
          data,
        },
      };
    },
  },
  async run({ $ }) {
    const {
      app,
      model,
      text,
      history,
      mediaFiles,
      safetySettings,
      responseFormat,
      responseSchema,
      maxOutputTokens,
      temperature,
      topP,
      topK,
      stopSequences,
    } = this;

    if (!Array.isArray(mediaFiles)) {
      throw new ConfigurationError("Image/Video files must be an array.");
    }

    if (!mediaFiles.length) {
      throw new ConfigurationError("At least one media file must be provided.");
    }

    const contents = [
      ...this.formatHistoryToContent(history),
      {
        parts: [
          ...(await Promise.all(mediaFiles.map((path) => this.fileToGenerativePart(path)))),
          {
            text,
          },
        ],
        role: "user",
      },
    ];

    const response = await app.generateContent({
      $,
      model,
      data: {
        contents,
        safetySettings: this.formatSafetySettings(safetySettings),
        ...(
          responseFormat || maxOutputTokens || temperature || topP || topK || stopSequences?.length
            ? {
              generationConfig: {
                responseMimeType: "application/json",
                responseSchema: utils.parse(responseSchema),
                maxOutputTokens,
                temperature,
                topP,
                topK,
                stopSequences,
              },
            }
            : {}
        ),
      },
    });

    $.export("$summary", "Successfully generated content from text and image.");

    return response;
  },
};

Action Configuration#

This component may be configured based on the props defined in the component code. Pipedream automatically prompts for input values in the UI.

Label	Prop	Type	Description
Google Gemini	`app`	`app`	This component uses the Google Gemini app.
Model	`model`	`string`	Select a value from the drop down menu.
Prompt Text	`text`	`string`	The text to use as the prompt for content generation
JSON Output	`responseFormat`	`boolean`	Enable to receive responses in structured JSON format instead of plain text. Useful for automated processing, data extraction, or when you need to parse the response programmatically. You can optionally define a specific schema for the response structure.
Conversation History	`history`	`string[]`	Previous messages in the conversation. Each item must be a valid JSON string with `text` and `role` (either `user` or `model`). Example: `{"text": "Hello", "role": "user"}`
Safety Settings	`safetySettings`	`string[]`	Configure content filtering for different harm categories. Each item must be a valid JSON string with `category` (one of: `HARASSMENT`, `HATE_SPEECH`, `SEXUALLY_EXPLICIT`, `DANGEROUS`, `CIVIC`) and `threshold` (one of: `BLOCK_NONE`, `BLOCK_ONLY_HIGH`, `BLOCK_MEDIUM_AND_ABOVE`, `BLOCK_LOW_AND_ABOVE`). Example: `{"category": "HARASSMENT", "threshold": "BLOCK_MEDIUM_AND_ABOVE"}`
Media File Paths or URLs	`mediaFiles`	`string[]`	A list of file paths from the `/tmp` directory or URLs for the media to process.
syncDir	`syncDir`	`dir`

Action Authentication#

Google Gemini uses API keys for authentication. When you connect your Google Gemini account, Pipedream securely stores the keys so you can easily authenticate to Google Gemini APIs in both code and no-code steps.