This topic was automatically generated from Slack. You can find the original thread here.
how many temp files can you store at once in a single code step?
You can store files in the /tmp directory within your Pipedream workflow steps. However, there are some limitations to keep in mind: 1. Storage Limit: The /tmp directory can store up to a certain amount of data, defined by the environment variable process.env.TMP_SIZE_LIMIT. This limit can vary, so it’s essential to check the specific value for your environment. 2. Persistence: The storage in /tmp may be wiped or may not exist between workflow executions. Therefore, you should assume that the /tmp directory is empty between runs to avoid errors. Given these constraints, the number of temporary files you can store at once in a single code step depends on the total size of the files and the available storage limit defined by process.env.TMP_SIZE_LIMIT. For more details, you can refer to the Pipedream documentation on file storage. If you have any further questions or need more specific information, feel free to ask!
The issue I am having is when I run this code:
import { axios } from “@pipedream/platform”;
import fs from ‘fs/promises’;
export default defineComponent({
props: {
roadmapPage: {
type: “string”,
label: “Roadmap Page”,
},
humanCapitalPage: {
type: “string”,
label: “Human Capital Page”,
},
kpiPage: {
type: “string”,
label: “KPI Page”,
},
roiPage: {
type: “string”,
label: “ROI Page”,
},
pricingPage: {
type: “string”,
label: “Pricing Page”,
},
businessName: {
type: “string”,
label: “Business Name”,
},
},
async run({ steps, $ }) {
const files = [
{ name: ${this.businessName} Roadmap Page.html, content: this.roadmapPage },
{ name: ${this.businessName} Human Capital Page.html, content: this.humanCapitalPage },
{ name: ${this.businessName} KPI Page.html, content: this.kpiPage },
{ name: ${this.businessName} ROI Page.html, content: this.roiPage },
{ name: ${this.businessName} Pricing Page.html, content: this.pricingPage },
];
const results = await Promise.all(files.map(async (file) => {
const filePath = `/tmp/${file.name}`;
console.log(`Writing to file: ${filePath}`);
console.log(`File content: ${file.content}`);
await fs.writeFile(filePath, file.content);
return {
fileName: file.name,
filePath: filePath,
message: `File ${file.name} has been created successfully.`,
};
}));
// Read back the files to verify content
const verificationResults = await Promise.all(files.map(async (file) => {
const filePath = `/tmp/${file.name}`;
const fileContent = await fs.readFile(filePath, 'utf8');
return {
fileName: file.name,
filePath: filePath,
fileContent: fileContent,
};
}));
console.log('Verification Results:', verificationResults);
return { results, verificationResults };
},
});
It properly exports the temp file paths but when I check the file contents, they are all the pricing page data
It looks like the issue you’re encountering is that all the files are being written with the content of the pricingPage prop. This could be due to how the props are being accessed within the files array. Let’s ensure that each file gets the correct content from the corresponding prop. Here’s the corrected code: ``` import { axios } from “@pipedream/platform”; import fs from ‘fs/promises’; export default defineComponent({ props: { roadmapPage: { type: “string”, label: “Roadmap Page”, }, humanCapitalPage: { type: “string”, label: “Human Capital Page”, }, kpiPage: { type: “string”, label: “KPI Page”, }, roiPage: { type: “string”, label: “ROI Page”, }, pricingPage: { type: “string”, label: “Pricing Page”, }, businessName: { type: “string”, label: “Business Name”, }, }, async run({ steps, $ }) { const files = [ { name: ${this.businessName} Roadmap Page.html, content: this.roadmapPage }, { name: ${this.businessName} Human Capital Page.html, content: this.humanCapitalPage }, { name: ${this.businessName} KPI Page.html, content: this.kpiPage }, { name: ${this.businessName} ROI Page.html, content: this.roiPage }, { name: ${this.businessName} Pricing Page.html, content: this.pricingPage }, ]; const results = await Promise.all(files.map(async (file) => { const filePath = /tmp/${file.name}; console.log(Writing to file: ${filePath}); console.log(File content: ${file.content}); await fs.writeFile(filePath, file.content); return { fileName: file.name, filePath: filePath, message: File ${file.name} has been created successfully., }; })); // Read back the files to verify content const verificationResults = await Promise.all(files.map(async (file) => { const filePath = /tmp/${file.name}; const fileContent = await fs.readFile(filePath, ‘utf8’); return { fileName: file.name,
Unexpected keyword ‘import’ (16:24)
It looks like you’re encountering an error related to the use of the import statement in your code. In Node.js, you need to ensure that you’re using the correct module system. Pipedream supports ECMAScript modules (ESM), so you should use import statements instead of require. Here’s a quick guide to resolving the issue: 1. Ensure you’re using ESM imports: Convert any require statements to import statements. For example: // Incorrect const axios = require("axios"); // Correct import axios from "axios"; 2. Check for mixed imports: Make sure you’re not mixing CommonJS (require) and ESM (import) in the same step. If you need to use both, separate them into different steps. Here’s an example of a Pipedream component using ESM imports correctly: ```
import { axios } from “@pipedream/platform”; export default defineComponent({ props: { openai: { type: “app”, app: “openai”, } }, async run({ steps, $ }) { return await axios($, { url: “https://api.openai.com/v1/models”, headers: { Authorization: Bearer ${this.openai.$auth.api_key}, }, }); }, });