To set up the function with axios and use it within the run method of a Pipedream component, you’ll need to modify your code slightly. Here’s how you can integrate the essData function using axios from the @pipedream/platform package: import { axios } from "@pipedream/platform"; export default defineComponent({ props: { // Define any props you need here }, async run({ steps, $ }) { // Define the essData function within the run method const essData = async (key) => { const response = await axios($, { url: `${essUrl}${key}`, // Make sure essUrl is defined and accessible // Add any required headers or other axios config here }); return response; // axios automatically parses JSON responses }; // Call essData with the desired key and return its result return await essData(this.url); // Make sure this.url is defined in your props }, }); Please ensure that essUrl is defined and accessible within the scope of the run method, and that this.url is a prop defined in your component with the URL you want to pass to essData. If you need to pass headers or other configuration options to axios, you can add them within the axios config object.
is it mandatory in pipedream to use props? can’t i just define my parameters as simple variables with const variable = value?
No, it’s not mandatory to use props in Pipedream components. You can define parameters as simple variables within the run method of your component using const or let. However, props are useful when you want to expose inputs to the Pipedream UI, allowing users to enter values without modifying the code. Here’s an example of using a simple variable: export default defineComponent({ async run({ steps, $ }) { const variable = "value"; console.log(variable); }, }) In this example, variable is a constant within the run method, and its value is not exposed to the UI for user input. If you need to hardcode values and do not require user input, this approach is perfectly fine.
can you show me how to properly wrap this code so it works in a Pipedream Node.js code step? with all the detail possible of he changes that need to be done, I don’t want to use props, I prefer using simple variables
import fetch from 'node-fetch';
const collectionId = "11111"
const keyApi = "2222"
const essUrl = "https://placeholder.com/unit_types?api_key="
// Parameters to get the item listed from the webflow CMS
const url = `https://api.webflow.com/collections/${collectionId}/items`;
const options = {
method: 'GET',
headers: {
accept: 'application/json',
authorization: `Bearer ${keyApi}`
}
};
// Function to fetch data fom the ESS API
const essData = async (key) => {
const data = await fetch(essUrl + key)
const jsonData = await data.json()
//console.log(jsonData)
return jsonData
};
// Function to list all the items of the webflow CMS
const webflowItems = async (e) => {
let urlOffset = url + `?offset=${e}`
const itemList = await fetch(urlOffset, options);
const itemJson = await itemList.json();
return itemJson.items
}
// Function to update the items on webflow CMS
const update = async (key) => {
//Call to the ESS API, brings the data from the company
const ess = await essData(key);
//Call to the webflow API list the items on the CMS
const webFlow = await webflowItems(0);
await webFlow.map(async (element, index) => {
//Find the item on ess that matches and itme on webflow
let updatedObject = ess.find(e => e.id === parseInt(element["ess-id"]));
// If a match is found upddate the value on webflow with the data on ess
if (updatedObject !== undefined) {
//create a URL tor tigger that specific item on webflow
let itemID = element._id
let urlupdate = url + `/${itemID}` + "?live=true";
// create the values that will be updated on webflow CMS
let slug = element.slug;
let name = element.name;
let price = updatedObject.price;
price = price.split('.')[0]
price = parseInt(price)
let availUnits = updatedObject.available_unit_count;
// Method for the call and authentication on webflow API
const options2 = {
method: 'PUT',
headers: {
accept: 'application/json',
'content-type': 'application/json',
authorization: `Bearer ${keyApi}`
},
body: JSON.stringify({
fields: {
slug: slug,
name: name,
_archived: false,
_draft: false,
price: price,
"available-units": availUnits,
}
})
};
// Delay each iteration by 1 second using setTimeout and await
await new Promise(resolve => setTimeout(resolve, 700 * index));
// Fetch call to make update
fetch(urlupdate, options2)
.then(res => res.json())
.then(json => console.log(json))
.catch(err => console.error('error:' + err));
}
})
}
const keys = [
"sdfsdfds",
"dfdfddaf",
"afadfadf",
"sdfsdfadf"
]
keys.forEach((key) => update(key))
To adapt the provided code to a Pipedream Node.js code step without using props and using simple variables, you’ll need to make a few changes. Here’s how you can wrap the code: ``` import { axios } from “@pipedream/platform”; export default defineComponent({ async run({ steps, $ }) { const collectionId = “11111”; const keyApi = “2222”; const essUrl = “Warehouse Space for Rent on Flexible Terms - Placeholder”; // Function to fetch data from the ESS API const essData = async (key) => { const { data } = await axios($, { url: ${essUrl}${key}, method: “GET”, }); return data; }; // Function to list all the items of the webflow CMS const webflowItems = async (e) => { const { data } = await axios($, { url: https://api.webflow.com/collections/${collectionId}/items?offset=${e}, method: “GET”, headers: { accept: ‘application/json’, authorization: Bearer ${keyApi}, }, }); return data.items; }; // Function to update the items on webflow CMS const update = async (key) => { const ess = await essData(key); const webFlow = await webflowItems(0); for (let index = 0; index < webFlow.length; index++) { const element = webFlow[index]; let updatedObject = ess.find(e => e.id === parseInt(element[“ess-id”])); if (updatedObject !== undefined) { let itemID = element._id; let urlupdate = https://api.webflow.com/collections/${collectionId}/items/${itemID}?live=true; let slug = element.slug; let name = element.name; let price = updatedObject.price.split(‘.’)[0]; price = parseInt(price); let availUnits = updatedObject.available_unit_count; await axios($, { url: urlupdate, method: “PATCH”, headers: { accept: ‘application/json’, ‘content-type’: ‘application/json’, authorization: Bearer ${keyApi}, }, data: { fields: { slug: slug, name: name, _arch
I also highly recommend watching our University series on building with Node.js in code steps here:
Each episode is under 5 minutes long, and it doesn’t take much to get up to speed.
Hey I 've been trying to implement all that you have suggested here, I went over the documentation and pipe university videos, and try a couple of different code structures but could not make my workflow to work, it worked for a couple of tries and then I kept getting a error, and the it worked again I am not sure what is wrong with it, here is how the code looks right now:
const axios = require('axios');
const collectionId = "6579c74107a6c12"
const keyApi = "f15f90619c05473ba571d84db316767f0"
const essUrl = "https://software.com/unit_types?api_key=";
// Parameters to get the item listed from the webflow CMS
const url = `https://api.webflow.com/collections/${collectionId}/items`;
// Global Axios configuration
axios.interceptors.request.use((config) => {
config.headers['Authorization'] = `Bearer ${keyApi}`;
config.headers['Accept'] = 'application/json';
config.headers['Content-Type'] = 'application/json';
return config;
});
const keys = [
"BIsE5W",
"GW0P",
"FWrGvQ",
"lLTu9Laj",
"gqdP4dYkg",
"Is7Nlwk8MzD",
"0A3TdnvxT",
"tTke1gy4",
"xWAmzc7N",
];
const essData = async (key) => {
try {
const { data } = await axios.get(essUrl + key);
return data;
} catch (error) {
console.error('Error fetching data from ESS API:', error.message);
throw error;
}
};
const webflowItems = async (e) => {
try {
const { data } = await axios.get(`${url}?offset=${e}`);
return data.items;
} catch (error) {
console.error('Error fetching data from Webflow API:', error.message);
throw error;
}
};
const update = async (key) => {
try {
const ess = await essData(key);
const webFlow = await webflowItems(0);
// Function to introduce a delay with exponential backoff
const delay = (ms) => new Promise(resolve => setTimeout(resolve, ms));
let delayTime = 5000; // Initial delay time
await Promise.all(webFlow.map(async (element, index) => {
let updatedObject = ess.find(e => e.id === parseInt(element["ess-id"]));
if (updatedObject !== undefined) {
let itemID = element._id;
let urlupdate = `${url}/${itemID}?live=true`;
let slug = element.slug;
let name = element.name;
let price = updatedObject.price.split('.')[0];
price = parseInt(price);
let availUnits = updatedObject.available_unit_count;
const options2 = {
headers: {
accept: 'application/json',
'content-type': 'application/json',
authorization: `Bearer ${keyApi}`
},
};
// Introduce a delay with exponential backoff
await delay(delayTime);
// Axios call to make update
const response = await axios.put(urlupdate, {
fields: {
slug: slug,
name: name,
_archived: false,
_draft: false,
price: price,
"available-units": availUnits,
}
}, options2);
console.log(`Item updated - ESS ID: ${updatedObject.id}, Webflow ID: ${element._id}`);
console.log(response.data);
// Update the delay time exponentially
delayTime *= 2;
}
}));
} catch (error) {
console.error('Error updating items:', error.message);
}
};
const updateAll = async () => {
try {
await Promise.all(keys.map(async (key) => {
await update(key);
}));
} catch (error) {
console.error('Error updating all keys:', error.message);
}
};
updateAll();
It looks like you’re trying to integrate with Webflow and another API, and you’re encountering intermittent errors. The code you’ve provided is using CommonJS imports and global axios configuration, which is not the standard approach in Pipedream components. Let’s refactor this code into a Pipedream component using ESM imports and local axios configuration. Here’s a refactored version of your code as a Pipedream component: import { axios } from "@pipedream/platform"; export default defineComponent({ props: { webflow: { type: "app", app: "webflow", }, }, async run({ steps, $ }) { const collectionId = "6579c74107a6c12"; const essUrl = "https://software.com/unit_types?api_key="; const url = `https://api.webflow.com/collections/${collectionId}/items`; const keys = [ "BIsE5W", "GW0P", "FWrGvQ", "lLTu9Laj", "gqdP4dYkg", "Is7Nlwk8MzD", "0A3TdnvxT", "tTke1gy4", "xWAmzc7N", ]; const essData = async (key) => { try { return await axios($, { url: `${essUrl}${key}`, headers: { Authorization: `Bearer ${this.webflow.$auth.oauth_access_token}`, }, }); } catch (error) { throw new Error(`Error fetching data from ESS API: ${error.message}`); } }; const webflowItems = async (e) => { try { return await axios($, { url: `${url}?offset=${e}`, headers: { Authorization: `Bearer ${this.webflow.$auth.oauth_access_token}`, }, }); } catch (error) { throw new Error(`Error fetching data from Webflow API: ${error.message}`); } }; // ... rest of your code to update items }, }); Please note the following changes: 1. We’re using the axios constructor from the @pipedream/platform package, which is the standard way to make HTTP requests in Pipedream components
hey i am trying to make an update on a webflow CMS using their API and a node js and a pipedream component, I having truble with the axios update request, I am following webflow’s documentation but the code does not seem to work on pipedream, can you help me? here is the structure found on webflows documentations
const options = {
method: 'PATCH',
url: `https://api.webflow.com/v2/collections/${this.collectionId}/items/${itemID}`,
headers: {
accept: 'application/json',
'content-type': 'application/json',
authorization: `${this.keyApi}`
},
data: {
isArchived: false,
isDraft: false,
fieldData: {
slug: slug,
name: name,
_archived: false,
_draft: false,
price: price,
"available-units": availUnits
}
}
};
It looks like you’re trying to make a PATCH request to the Webflow API using Pipedream. Here’s how you can structure your Pipedream component to make an update to a Webflow CMS item: import { axios } from "@pipedream/platform"; export default defineComponent({ props: { webflow: { type: "app", app: "webflow", }, }, async run({ steps, $ }) { const collectionId = "your-collection-id"; // Replace with your actual collection ID const itemID = "your-item-id"; // Replace with your actual item ID const slug = "your-slug"; // Replace with your actual slug const name = "your-name"; // Replace with your actual name const price = "your-price"; // Replace with your actual price const availUnits = "your-available-units"; // Replace with your actual available units const options = { method: 'PATCH', url: `https://api.webflow.com/collections/${collectionId}/items/${itemID}`, headers: { accept: 'application/json', 'content-type': 'application/json', Authorization: `Bearer ${this.webflow.$auth.oauth_access_token}`, 'accept-version': '1.0.0', }, data: { fields: { slug: slug, name: name, _archived: false, _draft: false, price: price, "available-units": availUnits } } }; return await axios($, options); }, }); Please replace 'your-collection-id', 'your-item-id', 'your-slug', 'your-name', 'your-price', and 'your-available-units' with the actual values you want to use for the update. Note that the authorization header uses the OAuth access token from the connected Webflow account, and the accept-version header specifies the API version. If you encounter any issues or need further assistance, please let me know!
I am getting an error but I am not sure if it come from the axios request structure or it is a problem with webflow credentials , here is the error I get: (file:///tmp/pdg/dist/code/48267c4c81a8ceca241f181449ea4c2c2c6a294b914fb435e95c3a5853eeed7d/component.mjs:141:42)
at Function.all (null:null:null)
at null.update (file:///tmp/pdg/dist/code/48267c4c81a8ceca241f181449ea4c2c2c6a294b914fb435e95c3a5853eeed7d/component.mjs:95:13)
at null.null (file:///tmp/pdg/dist/code/48267c4c81a8ceca241f181449ea4c2c2c6a294b914fb435e95c3a5853eeed7d/component.mjs:153:13)
at Function.all (null:null:null)
at Object.run (file:///tmp/pdg/dist/code/48267c4c81a8ceca241f181449ea4c2c2c6a294b914fb435e95c3a5853eeed7d/component.mjs:152:9)
at null.executeComponent (/var/task/launch_worker.js:267:22)
at MessagePort.messageHandler (/var/task/launch_worker.js:764:28)
Based on the error message you’ve provided, it’s not immediately clear whether the issue is with the axios request structure or with your Webflow credentials. To better assist you, I would need to see the specific code that’s causing the error. However, I can provide some general guidance that might help you troubleshoot the problem. First, ensure that your Webflow credentials are correctly set up in Pipedream. You should have connected your Webflow account to the Pipedream component and have access to this.webflow.$auth.oauth_access_token within your run method. Here’s an example of how to structure the axios request with Webflow credentials: import { axios } from "@pipedream/platform"; export default defineComponent({ props: { webflow: { type: "app", app: "webflow", }, }, async run({ steps, $ }) { return await axios($, { url: `https://api.webflow.com/YOUR_ENDPOINT`, method: "GET", // or "POST", "PUT", "DELETE", etc., depending on your needs headers: { Authorization: `Bearer ${this.webflow.$auth.oauth_access_token}`, "accept-version": `1.0.0`, // Include any other headers required by the Webflow API }, // Include data or params if needed for your request }); }, }); Make sure to replace YOUR_ENDPOINT with the actual endpoint you’re trying to reach and include any additional headers or parameters required by the Webflow API. If your credentials are correct and you’re still encountering errors, the issue might be with the axios request structure. Double-check that you’re using the correct HTTP method, headers, and parameters as required by the Webflow API. If you’re unable to resolve the issue, please provide the specific code that’s causing the error, and I’ll be happy to take a closer look. If you prefer to speak with someone directly, you can visit https://pipedream.com/support for more support options.
is there a way I can review my code and my workflow with an expert I cannot make it work in pipedream but it runs without any problem when I do it locally in my laptop.
I’m sorry to hear you’re experiencing issues with your workflow on Pipedream. While I can’t review your code in real-time, I can certainly help troubleshoot the problem. Could you please provide more details about the issue you’re facing? For example: - What is the expected behavior of your workflow? - What errors or unexpected behavior are you encountering on Pipedream? - Can you share the code that works locally but not on Pipedream? - Are there any specific services, APIs, or triggers you’re using in your workflow? The more information you can provide, the better I can assist you in resolving the problem. If you prefer to speak with someone directly, you can also visit https://pipedream.com/support for more support options.
Hi Felipe,
I notice a few things that might be the cause of the issue.
- Your code appears to attempt to save the file in the current directory. Only
/tmpin workflows is writeable. - Your async function doesn’t return a promise that resolves when the end of the stream is finished. The code step might pass execution to the next step if there’s no promise to hold the execution to finish the stream
- I’m not 100% sure if pdfkit is supported, if it needs to compile binaries outside of
/tmp, then you’ll run into the same issue as #1. I suggest attempting to install that package in it’s own code step just as a test to make sure it’s compatible.
My workflow is triggered with date every day at midnight, then it runs a node js component which aims to update some values of a webflow CMS, all the connections and methods to update the CMS values were taken directly from their documentation.
I have a code contribution that works without problem locally, basically it loads the updated information in json format in the ess variable, then in the webFlow variable I save all the CMS elements, then I update it one by one when I find the ID values that match both in the ess and in webFlow here is the code using fetch which is the original one I also tried to use axios but it still doesn’t work.
const fetch = require('node-fetch');
const collectionId = "collection id"
const keyApi = "key api"
const essUrl = "https://software.com/api/unit_types?api_key="
// Parameters to get the item listed from the webflow CMS
const url = `https://api.webflow.com/collections/${collectionId}/items`;
const options = {
method: 'GET',
headers: {
accept: 'application/json',
authorization: `Bearer ${keyApi}`
}
};
// Function to fetch data fom the ESS API
const essData = async (key) => {
const data = await fetch(essUrl + key)
const jsonData = await data.json()
//console.log(jsonData)
return jsonData
};
// Function to list all the items of the webflow CMS
const webflowItems = async (e) => {
let urlOffset = url + `?offset=${e}`
const itemList = await fetch(urlOffset, options);
const itemJson = await itemList.json();
return itemJson.items
}
// Function to update the items on webflow CMS
const update = async (key) => {
//Call to the ESS API, brings the data from the company
const ess = await essData(key);
//Call to the webflow API list the items on the CMS
const webFlow = await webflowItems(0);
await webFlow.map(async (element, index) => {
//Find the item on ess that matches and itme on webflow
let updatedObject = ess.find(e => e.id === parseInt(element["ess-id"]));
// If a match is found upddate the value on webflow with the data on ess
if (updatedObject !== undefined) {
//create a URL tor tigger that specific item on webflow
let itemID = element._id
let urlupdate = url + `/${itemID}` + "?live=true";
// create the values that will be updated on webflow CMS
let slug = element.slug;
let name = element.name;
let price = updatedObject.price;
price = price.split('.')[0]
price = parseInt(price)
let availUnits = updatedObject.available_unit_count;
// Method for the call and authentication on webflow API
const options2 = {
method: 'PUT',
headers: {
accept: 'application/json',
'content-type': 'application/json',
authorization: `Bearer ${keyApi}`
},
body: JSON.stringify({
fields: {
slug: slug,
name: name,
_archived: false,
_draft: false,
price: price,
"available-units": availUnits,
}
})
};
// Delay each iteration by 1 second using setTimeout and await
await new Promise(resolve => setTimeout(resolve, 700 * index));
// Fetch call to make update
fetch(urlupdate, options2)
.then(res => res.json())
.then(json => console.log(json))
.catch(err => console.error('error:' + err));
}
})
}
const keys = [
"BJcQE5W",
"GW0QyP",
"FWrQ",
"lLgxk2j",
]
keys.forEach((key) => update(key))
thanks for following up, I am not trying to save in the local directory
Based on your description, it seems like you’ve developed a local Node.js script that interacts with the Webflow CMS API to update items based on data fetched from another API (ESS). To adapt this code to run within a Pipedream workflow, you’ll need to modify it to fit the Pipedream component model. Here’s an example of how you might convert your existing code to a Pipedream component. This example assumes that you have set up your Webflow app within Pipedream and have the necessary permissions to update CMS items: ``` import { axios } from “@pipedream/platform”; export default defineComponent({ props: { webflow: { type: “app”, app: “webflow”, }, }, async run({ steps, $ }) { const collectionId = “collection id”; // Replace with your actual collection ID const essUrl = “https://software.com/api/unit_types?api_key=”; // Replace with your actual ESS API URL // Function to fetch data from the ESS API const essData = async (key) => { const response = await axios($, { url: ${essUrl}${key}, method: “GET”, }); return response.data; }; // Function to list all the items of the Webflow CMS const webflowItems = async (offset) => { const response = await axios($, { url: https://api.webflow.com/collections/${collectionId}/items?offset=${offset}, method: “GET”, headers: { Authorization: Bearer ${this.webflow.$auth.oauth_access_token}, accept: “application/json”, }, }); return response.items; }; // Function to update the items on Webflow CMS const update = async (key) => { const ess = await essData(key); const webFlow = await webflowItems(0); for (const [index, element] of webFlow.entries()) { let updatedObject = ess.find(e => e.id === parseInt(element[“ess-id”])); if (updatedObject !== undefined) { let itemID = element._id; let price = updatedObject.price.split(‘.’)[0]; price = parseInt(price); let availUnits = updatedObject.available_unit_count; await axios($, { url: `https://api
how would the last part of the code be?
To provide an accurate answer, I need more context about the code you’re referring to. Could you please provide more details or share the existing part of the code you’re working on?