Weaviate is a vector storage / search technology which integrates with ML apps like OpenAI. It would be awesome if pipedream implemented an integration.
@lahill3030 Happy to help support. Could you share the docs to the Weaviate API(s) you want to integrate with? Are you using their cloud service, or running your own Weaviate server?
Primarily this: text2vec-openai | Weaviate - vector search engine
I am using their cloud service right now, but if they don’t accept my request for beta access I will likely have to switch to running my own weaviate server. My ultimate end-goal toward the project I’m working on would be to combine different sources of print-on-demand ecommerce catalogues into one vector database in order to create one consistent taxonomy based on their clustering. I also have other projects that this kind of openai-weaviate integration would be useful toward. I’m not really experienced with pipedream but it seams like this kind of combinatorics of different api’s and services goes along with what pipedream is trying to do.
Thanks! Definitely, those docs help.
I opened an issue on our GitHub repo so our team can add the integration.
In the meantime, you can connect to the service using their GraphQL API (take a look at the GraphQL example here). Try this:
- Create a new Pipedream workflow
- Choose the relevant trigger for your use case (e.g. HTTP, Schedule)
- Add a step to your workflow and search for OpenAI. Choose the option to Send an HTTP Request for any OpenAI API. Since the API you shared uses your OpenAI API key, this will let you connect your OpenAI account and make the GraphQL request there.
- Configure the HTTP request based on those docs you shared. I’d recommend taking a look at the
Curlexample from the docs above — that’s helpful to see the exact headers, body, etc. you need to pass.
When we add the integration, we can simply abstract this HTTP request as an action, so you can configure it without needing to write code.
Let me know if that works!