Confluent API Integrations
Build and run workflows using the Confluent API. Use 1000s of source-available triggers and actions across 2,000+ apps. Or write custom code to integrate any app or API in seconds.
Overview#
The Confluent API provides programmatic interaction with Confluent Cloud, a fully managed Kafka service. It lets you manage Kafka clusters, topics, users, and configurations, enabling seamless integration and data streaming capabilities. With Pipedream, you can create workflows that automate interactions with your Kafka infrastructure, such as triggering events on message arrival, managing topics, or integrating Kafka data with other services.
import { axios } from "@pipedream/platform"
export default defineComponent({
props: {
confluent: {
type: "app",
app: "confluent",
}
},
async run({steps, $}) {
const data = {
"topic_name": `pipedream-test`,
}
return await axios($, {
method: "post",
url: `https://${this.confluent.$auth.rest_endpoint_url}/kafka/v3/clusters/${this.confluent.$auth.cluster_id}/topics`,
headers: {
"Content-Type": `application/json`,
},
auth: {
username: `${this.confluent.$auth.api_key}`,
password: `${this.confluent.$auth.api_secret}`,
},
data,
})
},
})
Choose an API to Connect with Confluent API#
1-24of2,000+apps by most popular
Example Use Cases#
Automatically Manage Kafka Topics: Set up a workflow that listens for webhooks from your apps, triggering the creation or deletion of topics in your Confluent Cloud based on specific events or conditions met within your application ecosystem.
Stream Data to Analytics Platforms: Ingest messages from a Kafka topic and send them to an analytics platform like Google BigQuery. This workflow can transform and batch messages before loading, providing real-time analytics capabilities to your data.
Kafka User Management: Automate user creation, deletion, and access control list updates in Confluent Cloud in response to changes in your identity provider service, such as Okta or Active Directory, ensuring your Kafka user permissions stay in sync with your organization's user management policies.
Authentication#
Confluent uses API keys for authentication. When you connect your Confluent account, Pipedream securely stores the keys so you can easily authenticate to Confluent APIs in both code and no-code steps.
To connect your Confluent credentials, first, navigate to your Confluent account and sign in
Then, to retrieve your Cluster ID and REST Endpoint,
- Go to “Cluster Overview” > “Cluster Settings”
Your REST Endpoint URL is 123.confluent.cloud:443 if your REST Endpoint is https://123.confluent.cloud:443
Finally, to retrieve your API keys,
- Go to “Cluster Overview” > “API Keys”