The Unstructured Data ETL for Your LLM.
Write custom Node.js code and use any of the 400k+ npm packages available. Refer to the Pipedream Node docs to learn more.
import { axios } from "@pipedream/platform"
import FormData from "form-data";
import request from "request";
export default defineComponent({
props: {
unstructured: {
type: "app",
app: "unstructured",
},
},
async run({ steps, $ }) {
const data = new FormData();
data.append("files", request("https://www.learningcontainer.com/wp-content/uploads/2019/09/sample-pdf-file.pdf"));
return await axios($,{
method: "POST",
url: `${this.unstructured.$auth.url}/general/v0/general`,
headers: {
"Content-Type": `multipart/form-data`,
"unstructured-api-key": `${this.unstructured.$auth.api_key}`
},
data
});
},
})
Develop, run and deploy your Node.js code in Pipedream workflows, using it between no-code steps, with connected accounts, or integrate Data Stores and File Stores.
This includes installing NPM packages, within your code without having to manage a package.json
file or running npm install
.
Below is an example of installing the axios
package in a Pipedream Node.js code step. Pipedream imports the axios
package, performs the API request, and shares the response with subsequent workflow steps:
// To use previous step data, pass the `steps` object to the run() function
export default defineComponent({
async run({ steps, $ }) {
// Return data to use it in future steps
return steps.trigger.event
},
})