Create Transcription with OpenAI (ChatGPT) API on New Monitor Event (Instant) from Datadog API

Pipedream makes it easy to connect APIs for OpenAI (ChatGPT), Datadog and 1400+ other apps remarkably fast.

Trigger workflow on

New Monitor Event (Instant) from the Datadog API

Next, do this

Create Transcription with the OpenAI (ChatGPT) API

No credit card required

▶

Watch us build a workflow

7 min

Watch now ➜

Trusted by 500,000+ developers from startups to Fortune 500 companies

Developers ♥ Pipedream

Getting Started#

This integration creates a workflow with a Datadog trigger and OpenAI (ChatGPT) action. When you configure and deploy the workflow, it will run on Pipedream's servers 24x7 for free.

Select this integration
Configure the New Monitor Event (Instant) trigger
1. Connect your Datadog account
2. Select a Region
3. Select one or more Monitors
Configure the Create Transcription action
1. Connect your OpenAI (ChatGPT) account
2. Select a Audio Upload Type
3. Optional- Select a Language
Deploy the workflow
Send a test event to validate your setup
Turn on the trigger

Details#

This integration uses pre-built, source-available components from Pipedream's GitHub repo. These components are developed by Pipedream and the community, and verified and maintained by Pipedream.

To contribute an update to an existing component or create a new component, create a PR on GitHub. If you're new to Pipedream component development, you can start with quickstarts for trigger span and action development, and then review the component API reference.

Trigger#

New Monitor Event (Instant) on Datadog

Description:Emit new events captured by a Datadog monitor

Version:0.1.2

Key:datadog-new-monitor-event

View on GitHub

Datadog Overview#

With the Datadog API, you can build a variety of applications and tools to help
you monitor and optimize your Datadog account. Here are a few examples:

A tool to help you visualize your Datadog account data
A tool to help you manage your Datadog account settings
A tool to help you monitor and troubleshoot your Datadog account
A tool to help you migrate your data to Datadog

Trigger Code#

import datadog from "../../datadog.app.mjs";
import { payloadFormat } from "../common/payload-format.mjs";

export default {
  key: "datadog-new-monitor-event",
  name: "New Monitor Event (Instant)",
  description: "Emit new events captured by a Datadog monitor",
  dedupe: "unique",
  version: "0.1.2",
  type: "source",
  props: {
    datadog,
    db: "$.service.db",
    http: {
      type: "$.interface.http",
      customResponse: true,
    },
    region: {
      propDefinition: [
        datadog,
        "region",
      ],
    },
    monitors: {
      propDefinition: [
        datadog,
        "monitors",
        (c) => ({
          region: c.region,
        }),
      ],
    },
  },
  hooks: {
    async deploy() {
      // emit historical events
      const { events } = await this.datadog.getEvents({
        params: {
          start: Math.floor(this.datadog.daysAgo(7) / 1000), // one week ago
          end: Math.floor(Date.now() / 1000), // now
        },
        region: this.region,
      });
      const relevantEvents = events.filter((event) => this.monitors.includes(event.monitor_id));

      for (const event of relevantEvents.reverse().slice(-25)) {
        const payload = {
          alertTitle: event.eventTitle,
          alertType: event.alert_type,
          date: event.date_happened,
          eventMsg: event.text,
          eventTitle: event.title,
          hostname: event.host,
          id: event.id,
          lastUpdated: event.date_happened,
          link: `https://app.datadoghq.com${event.url}`,
          priority: event.priority,
          tags: event.tags.join(),
        };
        const meta = this.generateMeta(payload);
        this.$emit(payload, meta);
      }
    },
    async activate() {
      const {
        name: webhookName,
        secretKey: webhookSecretKey,
      } = await this.datadog.createWebhook(
        this.http.endpoint,
        payloadFormat,
        this.region,
      );

      console.log(`Created webhook "${webhookName}"`);
      this._setWebhookName(webhookName);
      this._setWebhookSecretKey(webhookSecretKey);

      await Promise.all(
        this.monitors.map((monitorId) =>
          this.datadog.addWebhookNotification(webhookName, monitorId, this.region)),
      );
    },
    async deactivate() {
      const webhookName = this._getWebhookName();
      await this.datadog.removeWebhookNotifications(webhookName, this.region);
      await this.datadog.deleteWebhook(webhookName, this.region);
    },
  },
  methods: {
    _getWebhookName() {
      return this.db.get("webhookName");
    },
    _setWebhookName(webhookName) {
      this.db.set("webhookName", webhookName);
    },
    _getWebhookSecretKey() {
      return this.db.get("webhookSecretKey");
    },
    _setWebhookSecretKey(webhookSecretKey) {
      this.db.set("webhookSecretKey", webhookSecretKey);
    },
    generateMeta(data) {
      const {
        id,
        eventTitle: summary,
        date: ts,
      } = data;
      return {
        id,
        summary,
        ts,
      };
    },
  },
  async run(event) {
    const webhookSecretKey = this._getWebhookSecretKey();
    if (!this.datadog.isValidSource(event, webhookSecretKey)) {
      console.log("Skipping event from unrecognized source");
      this.http.respond({
        status: 404,
      });
      return;
    }

    // Acknowledge the event back to Datadog.
    this.http.respond({
      status: 200,
    });

    const { body } = event;
    const meta = this.generateMeta(body);
    this.$emit(body, meta);
  },
};

Trigger Configuration#

This component may be configured based on the props defined in the component code. Pipedream automatically prompts for input values in the UI and CLI.

Label	Prop	Type	Description
Datadog	`datadog`	`app`	This component uses the Datadog app.
N/A	`db`	`$.service.db`	This component uses `$.service.db` to maintain state between executions.
N/A	`http`	`$.interface.http`	This component uses `$.interface.http` to generate a unique URL when the component is first instantiated. Each request to the URL will trigger the `run()` method of the component.
Region	`region`	`string`	Select a value from the drop down menu:`{ "label": "US1 - East", "value": "datadoghq.com" }{ "label": "US3 - West", "value": "us3.datadoghq.com" }{ "label": "US5 - Central", "value": "us5.datadoghq.com" }{ "label": "EU1 - Europe", "value": "datadoghq.eu" }{ "label": "US1-FED (FedRamp)", "value": "ddog-gov.com" }`
Monitors	`monitors`	`integer[]`	Select a value from the drop down menu.

Trigger Authentication#

Datadog uses API keys for authentication. When you connect your Datadog account, Pipedream securely stores the keys so you can easily authenticate to Datadog APIs in both code and no-code steps.

Manage your account’s API and applications keys in Datadog

About Datadog#

Cloud monitoring as a service

Action#

Create Transcription on OpenAI (ChatGPT)

Description:Transcribes audio into the input language. [See docs here](https://platform.openai.com/docs/api-reference/audio/create).

Version:0.0.9

Key:openai-create-transcription

View on GitHub

OpenAI (ChatGPT) Overview#

The OpenAI API is a powerful tool that provides access to a range of
high-powered machine learning models. With the OpenAI API, developers can
create products, services, and tools that enable humanizing AI experiences, and
build applications that extend the power of human language.

Using the OpenAI API, developers can create language-driven applications such
as:

Natural language understanding and sentiment analysis
Text-based search
Question answering
Dialogue systems and conversation agents
Intelligent text completion
Text summarization
Text classification
Machine translation
Language generation
Multi-factor authentication
Anomaly detection
Text analysis

Action Code#

import axios from "axios";
import Bottleneck from "bottleneck";
import fs from "fs";
import {
  join, extname,
} from "path";
import FormData from "form-data";
import { ConfigurationError } from "@pipedream/platform";
import common from "../common/common.mjs";
import constants from "../common/constants.mjs";
import lang from "../common/lang.mjs";
import openai from "../../app/openai.app.mjs";
import { promisify } from "util";
import stream from "stream";
import { exec } from "child_process";
import ffmpegInstaller from "@ffmpeg-installer/ffmpeg";

const COMMON_AUDIO_FORMATS_TEXT = "Your audio file must be in one of these formats: mp3, mp4, mpeg, mpga, m4a, wav, or webm.";

const execAsync = promisify(exec);
const pipelineAsync = promisify(stream.pipeline);

export default {
  name: "Create Transcription",
  version: "0.0.9",
  key: "openai-create-transcription",
  description: "Transcribes audio into the input language. [See docs here](https://platform.openai.com/docs/api-reference/audio/create).",
  type: "action",
  props: {
    openai,
    uploadType: {
      label: "Audio Upload Type",
      description: "Are you uploading an audio file from [your workflow's `/tmp` directory](https://pipedream.com/docs/code/nodejs/working-with-files/#the-tmp-directory), or providing a URL to the file?",
      type: "string",
      options: [
        "File",
        "URL",
      ],
      reloadProps: true,
    },
    language: {
      label: "Language",
      description: "**Optional**. The language of the input audio. Supplying the input language will improve accuracy and latency.",
      type: "string",
      optional: true,
      default: "en",
      options: lang.LANGUAGES.map((l) => ({
        label: l.label,
        value: l.value,
      })),
    },
  },
  async additionalProps() {
    const props = {};
    switch (this.uploadType) {
    case "File":
      props.path = {
        type: "string",
        label: "File Path",
        description: `A path to your audio file to transcribe, e.g. \`/tmp/audio.mp3\`. ${COMMON_AUDIO_FORMATS_TEXT} Add the appropriate extension (mp3, mp4, etc.) on your filename — OpenAI uses the extension to determine the file type. [See the Pipedream docs on saving files to \`/tmp\`](https://pipedream.com/docs/code/nodejs/working-with-files/#writing-a-file-to-tmp)`,
      };
      break;
    case "URL":
      props.url = {
        type: "string",
        label: "URL",
        description: `A public URL to the audio file to transcribe. This URL must point directly to the audio file, not a webpage that links to the audio file. ${COMMON_AUDIO_FORMATS_TEXT}`,
      };
      break;
    default:
      throw new ConfigurationError("Invalid upload type specified. Please provide 'File' or 'URL'.");
    }
    // Because we need to display the file or URL above, and not below, these optional props
    // TODO: Will be fixed when we render optional props correctly when used with additionalProps
    props.prompt = {
      label: "Prompt",
      description: "**Optional** text to guide the model's style or continue a previous audio segment. The [prompt](https://platform.openai.com/docs/guides/speech-to-text/prompting) should match the audio language.",
      type: "string",
      optional: true,
    };
    props.responseFormat = {
      label: "Response Format",
      description: "**Optional**. The format of the response. The default is `json`.",
      type: "string",
      default: "json",
      optional: true,
      options: constants.TRANSCRIPTION_FORMATS,
    };
    props.temperature = common.props.temperature;

    return props;
  },
  methods: {
    createForm({
      file, outputDir,
    }) {
      const form = new FormData();
      form.append("model", "whisper-1");
      if (this.prompt) form.append("prompt", this.prompt);
      if (this.temperature) form.append("temperature", this.temperature);
      if (this.language) form.append("language", this.language);
      if (this.responseFormat) form.append("response_format", this.responseFormat);
      const readStream = fs.createReadStream(join(outputDir, file));
      form.append("file", readStream);
      return form;
    },
    async chunkFileAndTranscribe({
      file, $,
    }) {
      const outputDir = join("/tmp", "chunks");
      await execAsync(`mkdir -p "${outputDir}"`);
      await execAsync(`rm -f "${outputDir}/*"`);

      await this.chunkFile({
        file,
        outputDir,
      });

      const files = await fs.promises.readdir(outputDir);

      return await this.transcribeFiles({
        files,
        outputDir,
        $,
      });
    },
    async chunkFile({
      file, outputDir,
    }) {
      const ffmpegPath = ffmpegInstaller.path;
      const ext = extname(file);

      const fileSizeInMB = fs.statSync(file).size / (1024 * 1024);
      const numberOfChunks = Math.ceil(fileSizeInMB / 24);

      if (numberOfChunks === 1) {
        await execAsync(`cp "${file}" "${outputDir}/chunk-000${ext}"`);
        return;
      }

      const { stdout } = await execAsync(`${ffmpegPath} -i "${file}" 2>&1 | grep "Duration"`);
      const duration = stdout.match(/\d{2}:\d{2}:\d{2}\.\d{2}/s)[0];
      const [
        hours,
        minutes,
        seconds,
      ] = duration.split(":").map(parseFloat);

      const totalSeconds = (hours * 60 * 60) + (minutes * 60) + seconds;
      const segmentTime = Math.ceil(totalSeconds / numberOfChunks);

      const command = `${ffmpegPath} -i "${file}" -f segment -segment_time ${segmentTime} -c copy "${outputDir}/chunk-%03d${ext}"`;
      await execAsync(command);
    },
    transcribeFiles({
      files, outputDir, $,
    }) {
      const limiter = new Bottleneck({
        maxConcurrent: 1,
        minTime: 1000 / 59,
      });

      return Promise.all(files.map((file) => {
        return limiter.schedule(() => this.transcribe({
          file,
          outputDir,
          $,
        }));
      }));
    },
    transcribe({
      file, outputDir, $,
    }) {
      const form = this.createForm({
        file,
        outputDir,
      });
      return this.openai.createTranscription({
        $,
        form,
      });
    },
    getFullText(transcriptions = []) {
      return transcriptions.map((t) => t.text || t).join(" ");
    },
  },
  async run({ $ }) {
    const {
      url,
      path,
    } = this;

    if (!url && !path) {
      throw new Error("Must specify either File URL or File Path");
    }

    let file;

    if (path) {
      if (!fs.existsSync(path)) {
        throw new Error(`${path} does not exist`);
      }

      file = path;
    } else if (url) {
      const ext = extname(url);

      const response = await axios({
        method: "GET",
        url,
        responseType: "stream",
        timeout: 250000,
      });

      const bufferStream = new stream.PassThrough();
      response.data.pipe(bufferStream);

      const downloadPath = join("/tmp", `audio${ext}`);
      const writeStream = fs.createWriteStream(downloadPath);

      await pipelineAsync(bufferStream, writeStream);

      file = downloadPath;
    }

    const transcriptions = await this.chunkFileAndTranscribe({
      file,
      $,
    });

    if (transcriptions.length) {
      $.export("$summary", "Successfully created transcription");
    }

    return {
      transcription: this.getFullText(transcriptions),
      transcriptions,
    };
  },
};

Action Configuration#

This component may be configured based on the props defined in the component code. Pipedream automatically prompts for input values in the UI.

Label	Prop	Type	Description
OpenAI (ChatGPT)	`openai`	`app`	This component uses the OpenAI (ChatGPT) app.
Audio Upload Type	`uploadType`	`string`	Select a value from the drop down menu:`FileURL`
Language	`language`	`string`	Select a value from the drop down menu:`{ "label": "Afar", "value": "aa" }{ "label": "Abkhaz", "value": "ab" }{ "label": "Avestan", "value": "ae" }{ "label": "Afrikaans", "value": "af" }{ "label": "Akan", "value": "ak" }{ "label": "Amharic", "value": "am" }{ "label": "Aragonese", "value": "an" }{ "label": "Arabic", "value": "ar" }{ "label": "Arabic (Algeria)", "value": "ar-dz" }{ "label": "Arabic (Bahrain)", "value": "ar-bh" }{ "label": "Arabic (Egypt)", "value": "ar-eg" }{ "label": "Arabic (Iraq)", "value": "ar-iq" }{ "label": "Arabic (Jordan)", "value": "ar-jo" }{ "label": "Arabic (Kuwait)", "value": "ar-kw" }{ "label": "Arabic (Lebanon)", "value": "ar-lb" }{ "label": "Arabic (Libya)", "value": "ar-ly" }{ "label": "Arabic (Morocco)", "value": "ar-ma" }{ "label": "Arabic (Oman)", "value": "ar-om" }{ "label": "Arabic (Qatar)", "value": "ar-qa" }{ "label": "Arabic (Saudi Arabia)", "value": "ar-sa" }{ "label": "Arabic (Syria)", "value": "ar-sy" }{ "label": "Arabic (Tunisia)", "value": "ar-tn" }{ "label": "Arabic (U.A.E.)", "value": "ar-ae" }{ "label": "Arabic (Yemen)", "value": "ar-ye" }{ "label": "Assamese", "value": "as" }{ "label": "Avaric", "value": "av" }{ "label": "Aymara", "value": "ay" }{ "label": "Azerbaijani", "value": "az" }{ "label": "Bashkir", "value": "ba" }{ "label": "Belarusian", "value": "be" }{ "label": "Bulgarian", "value": "bg" }{ "label": "Bislama", "value": "bi" }{ "label": "Bambara", "value": "bm" }{ "label": "Bengali", "value": "bn" }{ "label": "Tibetan", "value": "bo" }{ "label": "Breton", "value": "br" }{ "label": "Bosnian", "value": "bs" }{ "label": "Catalan", "value": "ca" }{ "label": "Chechen", "value": "ce" }{ "label": "Chamorro", "value": "ch" }{ "label": "Corsican", "value": "co" }{ "label": "Cree", "value": "cr" }{ "label": "Czech", "value": "cs" }{ "label": "Old Church Slavonic", "value": "cu" }{ "label": "Chuvash", "value": "cv" }{ "label": "Welsh", "value": "cy" }{ "label": "Danish", "value": "da" }{ "label": "German", "value": "de" }{ "label": "Divehi", "value": "dv" }{ "label": "Dzongkha", "value": "dz" }{ "label": "Ewe", "value": "ee" }{ "label": "Greek", "value": "el" }{ "label": "English", "value": "en" }{ "label": "Esperanto", "value": "eo" }{ "label": "Spanish", "value": "es" }{ "label": "Estonian", "value": "et" }{ "label": "Basque", "value": "eu" }{ "label": "Persian", "value": "fa" }{ "label": "Fula", "value": "ff" }{ "label": "Finnish", "value": "fi" }{ "label": "Fijian", "value": "fj" }{ "label": "Faroese", "value": "fo" }{ "label": "French", "value": "fr" }{ "label": "Western Frisian", "value": "fy" }{ "label": "Irish", "value": "ga" }{ "label": "Scottish Gaelic", "value": "gd" }{ "label": "Galician", "value": "gl" }{ "label": "Guaraní", "value": "gn" }{ "label": "Gujarati", "value": "gu" }{ "label": "Manx", "value": "gv" }{ "label": "Hausa", "value": "ha" }{ "label": "Hebrew", "value": "he" }{ "label": "Hindi", "value": "hi" }{ "label": "Hiri Motu", "value": "ho" }{ "label": "Croatian", "value": "hr" }{ "label": "Haitian", "value": "ht" }{ "label": "Hungarian", "value": "hu" }{ "label": "Armenian", "value": "hy" }{ "label": "Herero", "value": "hz" }{ "label": "Interlingua", "value": "ia" }{ "label": "Indonesian", "value": "id" }{ "label": "Interlingue", "value": "ie" }{ "label": "Igbo", "value": "ig" }{ "label": "Nuosu", "value": "ii" }{ "label": "Inupiaq", "value": "ik" }{ "label": "Ido", "value": "io" }{ "label": "Icelandic", "value": "is" }{ "label": "Italian", "value": "it" }{ "label": "Inuktitut", "value": "iu" }{ "label": "Japanese", "value": "ja" }{ "label": "Javanese", "value": "jv" }{ "label": "Georgian", "value": "ka" }{ "label": "Kongo", "value": "kg" }{ "label": "Kikuyu", "value": "ki" }{ "label": "Kwanyama", "value": "kj" }{ "label": "Kazakh", "value": "kk" }{ "label": "Kalaallisut", "value": "kl" }{ "label": "Khmer", "value": "km" }{ "label": "Kannada", "value": "kn" }{ "label": "Korean", "value": "ko" }{ "label": "Kanuri", "value": "kr" }{ "label": "Kashmiri", "value": "ks" }{ "label": "Kurdish", "value": "ku" }{ "label": "Komi", "value": "kv" }{ "label": "Cornish", "value": "kw" }{ "label": "Kyrgyz", "value": "ky" }{ "label": "Latin", "value": "la" }{ "label": "Luxembourgish", "value": "lb" }{ "label": "Ganda", "value": "lg" }{ "label": "Limburgish", "value": "li" }{ "label": "Lingala", "value": "ln" }{ "label": "Lao", "value": "lo" }{ "label": "Lithuanian", "value": "lt" }{ "label": "Luba-Katanga", "value": "lu" }{ "label": "Latvian", "value": "lv" }{ "label": "Malagasy", "value": "mg" }{ "label": "Marshallese", "value": "mh" }{ "label": "Māori", "value": "mi" }{ "label": "Macedonian", "value": "mk" }{ "label": "Malayalam", "value": "ml" }{ "label": "Mongolian", "value": "mn" }{ "label": "Marathi", "value": "mr" }{ "label": "Malay", "value": "ms" }{ "label": "Maltese", "value": "mt" }{ "label": "Burmese", "value": "my" }{ "label": "Nauru", "value": "na" }{ "label": "Norwegian Bokmål", "value": "nb" }{ "label": "Northern Ndebele", "value": "nd" }{ "label": "Nepali", "value": "ne" }{ "label": "Ndonga", "value": "ng" }{ "label": "Dutch", "value": "nl" }{ "label": "Norwegian Nynorsk", "value": "nn" }{ "label": "Norwegian", "value": "no" }{ "label": "Southern Ndebele", "value": "nr" }{ "label": "Navajo", "value": "nv" }{ "label": "Chichewa", "value": "ny" }{ "label": "Occitan", "value": "oc" }{ "label": "Ojibwe", "value": "oj" }{ "label": "Oromo", "value": "om" }{ "label": "Oriya", "value": "or" }{ "label": "Ossetian", "value": "os" }{ "label": "Panjabi", "value": "pa" }{ "label": "Pāli", "value": "pi" }{ "label": "Polish", "value": "pl" }{ "label": "Pashto", "value": "ps" }{ "label": "Portuguese", "value": "pt" }{ "label": "Quechua", "value": "qu" }{ "label": "Romansh", "value": "rm" }{ "label": "Kirundi", "value": "rn" }{ "label": "Romanian", "value": "ro" }{ "label": "Russian", "value": "ru" }{ "label": "Kinyarwanda", "value": "rw" }{ "label": "Sanskrit", "value": "sa" }{ "label": "Sardinian", "value": "sc" }{ "label": "Sindhi", "value": "sd" }{ "label": "Northern Sami", "value": "se" }{ "label": "Sango", "value": "sg" }{ "label": "Sinhala", "value": "si" }{ "label": "Slovak", "value": "sk" }{ "label": "Slovenian", "value": "sl" }{ "label": "Samoan", "value": "sm" }{ "label": "Shona", "value": "sn" }{ "label": "Somali", "value": "so" }{ "label": "Albanian", "value": "sq" }{ "label": "Serbian", "value": "sr" }{ "label": "Swati", "value": "ss" }{ "label": "Southern Sotho", "value": "st" }{ "label": "Sundanese", "value": "su" }{ "label": "Swedish", "value": "sv" }{ "label": "Swahili", "value": "sw" }{ "label": "Tamil", "value": "ta" }{ "label": "Telugu", "value": "te" }{ "label": "Tajik", "value": "tg" }{ "label": "Thai", "value": "th" }{ "label": "Tigrinya", "value": "ti" }{ "label": "Turkmen", "value": "tk" }{ "label": "Tagalog", "value": "tl" }{ "label": "Tswana", "value": "tn" }{ "label": "Tonga", "value": "to" }{ "label": "Turkish", "value": "tr" }{ "label": "Tsonga", "value": "ts" }{ "label": "Tatar", "value": "tt" }{ "label": "Twi", "value": "tw" }{ "label": "Tahitian", "value": "ty" }{ "label": "Uyghur", "value": "ug" }{ "label": "Ukrainian", "value": "uk" }{ "label": "Urdu", "value": "ur" }{ "label": "Uzbek", "value": "uz" }{ "label": "Venda", "value": "ve" }{ "label": "Vietnamese", "value": "vi" }{ "label": "Volapük", "value": "vo" }{ "label": "Walloon", "value": "wa" }{ "label": "Wolof", "value": "wo" }{ "label": "Xhosa", "value": "xh" }{ "label": "Yiddish", "value": "yi" }{ "label": "Yoruba", "value": "yo" }{ "label": "Zhuang", "value": "za" }{ "label": "Chinese", "value": "zh" }{ "label": "Zulu", "value": "zu" }`

Action Authentication#

OpenAI (ChatGPT) uses API keys for authentication. When you connect your OpenAI (ChatGPT) account, Pipedream securely stores the keys so you can easily authenticate to OpenAI (ChatGPT) APIs in both code and no-code steps.

Create a new API key and enter it below.

About OpenAI (ChatGPT)#

OpenAI is an AI research and deployment company with the mission to ensure that artificial general intelligence benefits all of humanity. They are the makers of popular apps like ChatGPT and DALL·E 2.