Apache Kafka is an open-source distributed event streaming platform used by thousands of companies for high-performance data pipelines, streaming analytics, data integration, and mission-critical applications.
Emit new event when a new batch is completed in OpenAI. See the documentation
Emit new event when a new file is created in OpenAI. See the documentation
Emit new event when a new fine-tuning job is created in OpenAI. See the documentation
Emit new event every time a run changes its status. See the documentation
The Chat API, using the gpt-3.5-turbo
or gpt-4
model. See the documentation
Summarizes text using the Chat API. See the documentation
Classify items into specific categories using the Chat API. See the documentation
Translate text from one language to another using the Chat API. See the documentation
Transcribes audio into the input language. See the documentation.
import { Kafka } from "kafkajs"
export default defineComponent({
props: {
kafka: {
type: "app",
app: "kafka",
}
},
async run({steps, $}) {
const kafka = new Kafka({
brokers: [`${this.kafka.$auth.host}:${this.kafka.$auth.port}`],
});
const consumer = kafka.consumer({ groupId: 'TestGroup' });
await consumer.connect()
await consumer.subscribe({ topic: 'SampleTopic', fromBeginning: true });
let consumedMessage = "";
const eachMessage = async function({ topic, partition, message }){
consumedMessage = message.value.toString();
return consumedMessage;
};
await consumer.run({
eachMessage,
});
const producer = kafka.producer();
await producer.connect()
await producer.send({
topic: 'SampleTopic',
messages: [
{ value: 'Welcome KafkaJS + Pipedream users! '+ new Date().toISOString() },
],
});
await producer.disconnect();
const data = await consumer.describeGroup();
return { consumedMessage, groupDescription: data };
},
})
OpenAI provides a suite of powerful AI models through its API, enabling developers to integrate advanced natural language processing and generative capabilities into their applications. Here’s an overview of the services offered by OpenAI's API:
Use Python or Node.js code to make fully authenticated API requests with your OpenAI account:
import { axios } from "@pipedream/platform"
export default defineComponent({
props: {
openai: {
type: "app",
app: "openai",
}
},
async run({steps, $}) {
return await axios($, {
url: `https://api.openai.com/v1/models`,
headers: {
Authorization: `Bearer ${this.openai.$auth.api_key}`,
},
})
},
})