How to Call the DreamStudio API in Pipedream?

Help
41

I apologize for the confusion earlier. You’re right, we need to address the issue in the step where the error occurs. In your current code, you’re saving the upscaled image to the /tmp directory, but you’re also returning the whole image buffer in the response object, which is likely causing the payload limit to be exceeded. To fix this, you should only return the filename instead of the entire response object. Modify the last part of your code like this: ```
// Save the upscaled image to the /tmp/ directory fs.writeFileSync(/tmp/${filename}, Buffer.from(response.data)); // Return the filename for use in other steps return { filename: /tmp/${filename}, };

42

still getting this error:

Workflow Payload Limit Exceeded
You’ve exceeded the limit on the logs and data stored for this execution of your workflow. See Troubleshooting Common Issues

Here is my code:

import { axios } from “@pipedream/platform”;
import fs from ‘fs’;

export default defineComponent({
props: {
apiKey: {
type: “string”,
label: “API Key”,
secret: true,
},
inputFilename: {
type: “string”,
label: “Input File Name”,
default: “”,
},
},
async run({ steps, $ }) {
// Read the previously generated image file
const imageData = fs.readFileSync(this.inputFilename);

**const** path = "https://api.stability.ai/v1/generation/esrgan-v1-x2plus/image-to-image/upscale";

**const** headers = {
  Accept: "image/png",
  Authorization: `Bearer ${**this**.apiKey}`,
  "Content-Type": "application/octet-stream",
};

_// Perform the upscale API call_
**const** response = **await** axios($, {
  url: path,
  headers,
  method: "POST",
  data: imageData,
  responseType: 'arraybuffer'
});

_// Generate a unique filename for the upscaled image_
**const** timestamp = Date.now();
**const** filename = `/tmp/upscaled_image_${timestamp}.png`;

_// Save the upscaled image to the /tmp/ directory_
fs.writeFileSync(`/tmp/${filename}`, Buffer.**from**(response.data));


_// Return the filename for use in other steps_
**return** {
  filename
};

},
});

What else can I do to avoid this error? Any other methods of saving this image that would prevent me from exceeding the limit?

43

I apologize for the confusion earlier. It seems that the issue might be related to the logs generated by the workflow. To minimize the logs, you can try disabling the console.log() statements in your code, if any, and avoid logging large objects. Additionally, you can try using the $.send.http() method to send the upscaled image directly to another service or storage without returning it in the step. This way, you avoid the payload limit by not including the image data in the step exports. Here’s an example of how to send the upscaled image to another service using $.send.http(): import { axios } from "@pipedream/platform"; import fs from 'fs'; export default defineComponent({ props: { apiKey: { type: "string", label: "API Key", secret: true, }, inputFilename: { type: "string", label: "Input File Name", default: "", }, }, async run({ steps, $ }) { // Read the previously generated image file const imageData = fs.readFileSync(this.inputFilename); const path = "https://api.stability.ai/v1/generation/esrgan-v1-x2plus/image-to-image/upscale>"; const headers = { Accept: "image/png", Authorization: `Bearer ${this.apiKey}`, "Content-Type": "application/octet-stream", }; // Perform the upscale API call const response = await axios($, { url: path, headers, method: "POST", data: imageData, responseType: 'arraybuffer' }); // Generate a unique filename for the upscaled image const timestamp = Date.now(); const filename = `/tmp/upscaled_image_${timestamp}.png`; // Save the upscaled image to the /tmp/ directory fs.writeFileSync(`/tmp/${filename}`, Buffer.from(response.data)); // Send the upscaled image to another service using $.send.http() await $.send.http({ method: "POST", url: "https://your-target-service-url.com", headers: { "Content-Type": "image/png", }, data: fs.createReadStream(filename), }); // Return the filename for use in other steps return { filename }; }, }); Make sure to replace `https://your-target-service-url

44

there are no console logs in any of my code.

I am now sending the image to CloudFlare – but I am still exceeding the Workflow Payload Limit.

Here is my code. Please help me:

import { axios } from “@pipedream/platform”;
import fs from ‘fs’;

export default defineComponent({
props: {
apiKey: {
type: “string”,
label: “API Key”,
secret: true,
},
cloudflareToken: {
type: “string”,
label: “Cloudflare Token”,
secret: true,
},
inputFilename: {
type: “string”,
label: “Input File Name”,
default: “”,
},
},
async run({ steps, $ }) {
// Read the previously generated image file
const imageData = fs.readFileSync(this.inputFilename);

**const** path = "https://api.stability.ai/v1/generation/esrgan-v1-x2plus/image-to-image/upscale";

**const** headers = {
  Accept: "image/png",
  Authorization: `Bearer ${**this**.apiKey}`,
  "Content-Type": "application/octet-stream",
};

_// Perform the upscale API call_
**const** response = **await** axios($, {
  url: path,
  headers,
  method: "POST",
  data: imageData,
  responseType: 'arraybuffer'
});

_// Generate a unique filename for the upscaled image_
**const** timestamp = Date.now();
**const** filename = `/tmp/upscaled_image_${timestamp}.png`;

_// Save the upscaled image to the /tmp/ directory_
fs.writeFileSync(filename, Buffer.**from**(response.data));

_// Send the upscaled image to Cloudflare using $.send.http()_
**await** $.send.http({
  method: "POST",
  url: "https://api.cloudflare.com/client/v4/accounts/anon/images/v1",
  headers: {
    "Authorization": `Bearer ${**this**.cloudflareToken}`,
  },
  data: fs.createReadStream(filename),
});

_// Return the filename for use in other steps_
**return** {
  filename
};

},
});

45

I apologize for the confusion earlier. It seems that the issue might be related to the logs generated by the workflow. To minimize the logs, you can try disabling the console.log() statements in your code, if any, and avoid logging large objects. Additionally, you can try using the $.send.http() method to send the upscaled image directly to another service or storage without returning it in the step. This way, you avoid the payload limit by not including the image data in the step exports. Here’s an example of how to send the upscaled image to another service using $.send.http(): import { axios } from "@pipedream/platform"; import fs from 'fs'; export default defineComponent({ props: { apiKey: { type: "string", label: "API Key", secret: true, }, inputFilename: { type: "string", label: "Input File Name", default: "", }, }, async run({ steps, $ }) { // Read the previously generated image file const imageData = fs.readFileSync(this.inputFilename); const path = "https://api.stability.ai/v1/generation/esrgan-v1-x2plus/image-to-image/upscale>"; const headers = { Accept: "image/png", Authorization: `Bearer ${this.apiKey}`, "Content-Type": "application/octet-stream", }; // Perform the upscale API call const response = await axios($, { url: path, headers, method: "POST", data: imageData, responseType: 'arraybuffer' }); // Generate a unique filename for the upscaled image const timestamp = Date.now(); const filename = `/tmp/upscaled_image_${timestamp}.png`; // Save the upscaled image to the /tmp/ directory fs.writeFileSync(`/tmp/${filename}`, Buffer.from(response.data)); // Send the upscaled image to another service using $.send.http() await $.send.http({ method: "POST", url: "https://your-target-service-url.com", headers: { "Content-Type": "image/png", }, data: fs.createReadStream(filename), }); // Return the filename for use in other steps return { filename }; }, }); Make sure to replace https://your-target-service-url with

46

your code / formatting seems to be broken.

I have no logs, as I mentioned

47

I apologize for the confusion earlier. Since you don’t have any logs, it’s difficult for me to pinpoint the exact issue. Can you please provide more information about the problem you’re facing? This will help me better understand the issue and provide a more accurate solution.

48

I am trying to use DreamStudio’s Upscaler API to take an image from the /tmp folder, run it through the upscaler, and send it to a cloudflare URL.

It seems like PipeDream is loading the entire image into memory before sending it to CloudFlare, which is causing me to hit the limit. Is there a way to prevent the image from being loaded into memory and simply point the upscaler API to post the image directly to the cloudflare folder?

Here is my code:

import { axios } from “@pipedream/platform”;
import fs from ‘fs’;

export default defineComponent({
props: {
apiKey: {
type: “string”,
label: “API Key”,
secret: true,
},
cloudflareToken: {
type: “string”,
label: “Cloudflare Token”,
secret: true,
},
inputFilename: {
type: “string”,
label: “Input File Name”,
default: “”,
},
},
async run({ steps, $ }) {
// Read the previously generated image file
const imageData = fs.readFileSync(this.inputFilename);

**const** path = "https://api.stability.ai/v1/generation/esrgan-v1-x2plus/image-to-image/upscale";

**const** headers = {
  Accept: "image/png",
  Authorization: `Bearer ${**this**.apiKey}`,
  "Content-Type": "application/octet-stream",
};

_// Perform the upscale API call_
**const** response = **await** axios($, {
  url: path,
  headers,
  method: "POST",
  data: imageData,
  responseType: 'arraybuffer'
});

_// Generate a unique filename for the upscaled image_
**const** timestamp = Date.now();
**const** filename = `/tmp/upscaled_image_${timestamp}.png`;

_// Save the upscaled image to the /tmp/ directory_
fs.writeFileSync(filename, Buffer.**from**(response.data));

_// Send the upscaled image to Cloudflare using $.send.http()_
**await** $.send.http({
  method: "POST",
  url: "https://api.cloudflare.com/client/v4/accounts/myid/images/v1",
  headers: {
    "Authorization": `Bearer ${**this**.cloudflareToken}`,
  },
  data: fs.createReadStream(filename),
});

_// Return the filename for use in other steps_
**return** {
  filename
};

},
});

49

I’ll remove this restriction — we have a check for blank input, but clearly you want to be able to call Pi again if you forgot (or if it failed the first time). @U05A7KLNALC can you help?

50

I understand that you want to remove a restriction related to blank input and allow calling the function again if needed. However, I need more context and information about your code and the specific issue you’re facing. Can you please provide more details or share the relevant code snippet? This will help me better understand the problem and provide a more accurate solution.

51

I understand that you’re trying to upscale an image using DreamStudio’s Upscaler API and then upload it to Cloudflare without loading the entire image into memory. To achieve this, you can use streams to read the image file and send it directly to the Upscaler API and Cloudflare without loading the entire file into memory. Here’s the modified code: ```
import { axios } from “@pipedream/platform”; import fs from ‘fs’; import stream from ‘stream’; import util from ‘util’; const pipeline = util.promisify(stream.pipeline); export default defineComponent({ props: { apiKey: { type: “string”, label: “API Key”, secret: true, }, cloudflareToken: { type: “string”, label: “Cloudflare Token”, secret: true, }, inputFilename: { type: “string”, label: “Input File Name”, default: “”, }, }, async run({ steps, $ }) { const path = “https://api.stability.ai/v1/generation/esrgan-v1-x2plus/image-to-image/upscale”; const headers = { Accept: “image/png”, Authorization: Bearer ${this.apiKey}, “Content-Type”: “application/octet-stream”, }; const timestamp = Date.now(); const filename = /tmp/upscaled_image_${timestamp}.png; // Use streams to read the image file and send it directly to the Upscaler API await pipeline( fs.createReadStream(this.inputFilename), axios($, { url: path, headers, method: “POST”, responseType: “stream”, }).then((response) => response.data), fs.createWriteStream(filename) ); // Send the upscaled image to Cloudflare using $.send.http() await $.send.http({ method: “POST”, url: “https://api.cloudflare.com/client/v4/accounts/myid/images/v1”, headers: { “Authorization”: Bearer ${this.cloudflareToken}, }, data: fs.createReadStream(filename), }); // Return the filename for use in other steps return { filename }; }, });

52

can you modify this code to send to ImageKit API instead? Here’s how I have it setup currently:

import { axios } from “@pipedream/platform”;
import ImageKit from “imagekit”;
import fs from “fs/promises”; // Using fs promises API

export default {
async run({ steps, $ }) {
const imagekit = new ImageKit({
publicKey: “”,
privateKey: ",
urlEndpoint: “”
});

**const** filename = steps.DreamStudio_Create_Cover_Image.$return_value.filename;

**try** {
  _// Read the file data_
  **const** fileData = **await** fs.readFile(`${filename}`); _// Replace with the correct path to the file_

  **const** result = **await** **new** Promise((resolve, reject) => {
    imagekit.upload({
      file: fileData,
      fileName: filename
    }, (error, result) => {
      **if** (error) reject(error);
      **else** resolve(result);
    });
  });
  console.log(result);
} **catch** (error) {
  console.log(error);
}

}
};

53

I understand you want to modify the code to send the upscaled image to ImageKit API. Here’s the updated code that uses streams to read the upscaled image and send it directly to ImageKit API without loading the entire file into memory: ``` import { axios } from “@pipedream/platform”; import ImageKit from “imagekit”; import fs from “fs”; import stream from “stream”; import util from “util”; const pipeline = util.promisify(stream.pipeline); export default defineComponent({ props: { apiKey: { type: “string”, label: “API Key”, secret: true, }, imageKitPublicKey: { type: “string”, label: “ImageKit Public Key”, secret: true, }, imageKitPrivateKey: { type: “string”, label: “ImageKit Private Key”, secret: true, }, imageKitUrlEndpoint: { type: “string”, label: “ImageKit URL Endpoint”, }, inputFilename: { type: “string”, label: “Input File Name”, default: “”, }, }, async run({ steps, $ }) { const path = “https://api.stability.ai/v1/generation/esrgan-v1-x2plus/image-to-image/upscale”; const headers = { Accept: “image/png”, Authorization: Bearer ${this.apiKey}, “Content-Type”: “application/octet-stream”, }; const timestamp = Date.now(); const filename = /tmp/upscaled_image_${timestamp}.png; // Use streams to read the image file and send it directly to the Upscaler API await pipeline( fs.createReadStream(this.inputFilename), axios($, { url: path, headers, method: “POST”, responseType: “stream”, }).then((response) => response.data), fs.createWriteStream(filename) ); // Initialize ImageKit instance const imagekit = new ImageKit({ publicKey: this.imageKitPublicKey, privateKey: this.imageKitPrivateKey, urlEndpoint: this.imageKitUrlEndpoint, }); // Upload the upscaled image to ImageKit using streams const result = await new Promise((resolve, reject) => { imagekit.upload( { file: fs.createReadStream(filename), fileName: `upscaled_image

54

my filename is this path: steps.DreamStudio_Create_Cover_Image.$return_value.filename

How can I make sure the code reads it properly?

55

To use the filename from steps.DreamStudio_Create_Cover_Image.$return_value.filename, you can update the inputFilename prop in the code I provided earlier. Here’s the modified code: import { axios } from "@pipedream/platform"; import ImageKit from "imagekit"; import fs from "fs"; import stream from "stream"; import util from "util"; const pipeline = util.promisify(stream.pipeline); export default defineComponent({ props: { // Other props... inputFilename: { type: "string", label: "Input File Name", default: steps.DreamStudio_Create_Cover_Image.$return_value.filename, }, }, async run({ steps, $ }) { // Rest of the code... }, }); Now, the inputFilename prop will use the filename from steps.DreamStudio_Create_Cover_Image.$return_value.filename.

56

Code Error
Unexpected token (38:27)
DETAILS

SyntaxError: Unexpected token (38:27)
    at pp$4.raise (/var/task/node_modules/acorn/dist/acorn.js:2927:15)
    at pp.unexpected (/var/task/node_modules/acorn/dist/acorn.js:698:10)
    at pp.expect (/var/task/node_modules/acorn/dist/acorn.js:692:28)
    at pp$3.parseObj (/var/task/node_modules/acorn/dist/acorn.js:2563:14)
    at pp$3.parseExprAtom (/var/task/node_modules/acorn/dist/acorn.js:2302:19)
    at anonymous.parseExprAtom (/var/task/node_modules/acorn-node/lib/dynamic-import/index.js:77:117)
    at anonymous.parseExprAtom (/var/task/node_modules/acorn-node/lib/import-meta/index.js:27:102)
    at pp$3.parseExprSubscripts (/var/task/node_modules/acorn/dist/acorn.js:2129:21)
    at pp$3.parseMaybeUnary (/var/task/node_modules/acorn/dist/acorn.js:2106:19)
    at anonymous.parseMaybeUnary (/var/task/node_modules/acorn-node/lib/private-class-elements/index.js:122:54)
57

I apologize for the error in the code I provided earlier. It seems there’s a syntax issue. Let’s correct it by removing the default value from the inputFilename prop and using it directly in the run method. Here’s the updated code: import { axios } from "@pipedream/platform"; import ImageKit from "imagekit"; import fs from "fs"; import stream from "stream"; import util from "util"; const pipeline = util.promisify(stream.pipeline); export default defineComponent({ props: { // Other props... inputFilename: { type: "string", label: "Input File Name", }, }, async run({ steps, $ }) { const inputFilename = steps.DreamStudio_Create_Cover_Image.$return_value.filename; // Rest of the code... }, }); Now, the inputFilename will be assigned the value from steps.DreamStudio_Create_Cover_Image.$return_value.filename within the run method.

58

can you please fix this code for me with what you just said? Plus it looks like you gave me incomplete code in the previous step:

import { axios } from “@pipedream/platform”;
import ImageKit from “imagekit”;
import fs from “fs”;
import stream from “stream”;
import util from “util”;
const pipeline = util.promisify(stream.pipeline);
export default defineComponent({
props: {
apiKey: {
type: “string”,
label: “API Key”,
secret: true,
},
imageKitPublicKey: {
type: “string”,
label: “ImageKit Public Key”,
secret: true,
},
imageKitPrivateKey: {
type: “string”,
label: “ImageKit Private Key”,
secret: true,
},
imageKitUrlEndpoint: {
type: “string”,
label: “ImageKit URL Endpoint”,
},
inputFilename: {
type: “string”,
label: “Input File Name”,
default: steps.DreamStudio_Create_Cover_Image.$return_value.filename,
},
},
async run({ steps, $ }) {
const path = “https://api.stability.ai/v1/generation/esrgan-v1-x2plus/image-to-image/upscale”;
const headers = {
Accept: “image/png”,
Authorization: Bearer ${this.apiKey},
“Content-Type”: “application/octet-stream”,
};
const timestamp = Date.now();
const filename = /tmp/upscaled_image_${timestamp}.png;
// Use streams to read the image file and send it directly to the Upscaler API
await pipeline(
fs.createReadStream(this.inputFilename),
axios($, {
url: path,
headers,
method: “POST”,
responseType: “stream”,
}).then((response) => response.data),
fs.createWriteStream(filename)
);
// Initialize ImageKit instance
const imagekit = new ImageKit({
publicKey: this.imageKitPublicKey,
privateKey: this.imageKitPrivateKey,
urlEndpoint: this.imageKitUrlEndpoint,
});
// Upload the upscaled image to ImageKit using streams
const result = await new Promise((resolve, reject) => {
imagekit.upload(
{
file: fs.createReadStream(filename),
fileName: `upscaled_image

59

I apologize for the confusion earlier. Here’s the corrected and complete code for the Pipedream component: ``` import { axios } from “@pipedream/platform”; import ImageKit from “imagekit”; import fs from “fs”; import stream from “stream”; import util from “util”; const pipeline = util.promisify(stream.pipeline); export default defineComponent({ props: { apiKey: { type: “string”, label: “API Key”, secret: true, }, imageKitPublicKey: { type: “string”, label: “ImageKit Public Key”, secret: true, }, imageKitPrivateKey: { type: “string”, label: “ImageKit Private Key”, secret: true, }, imageKitUrlEndpoint: { type: “string”, label: “ImageKit URL Endpoint”, }, inputFilename: { type: “string”, label: “Input File Name”, }, }, async run({ steps, $ }) { const inputFilename = steps.DreamStudio_Create_Cover_Image.$return_value.filename; const path = “https://api.stability.ai/v1/generation/esrgan-v1-x2plus/image-to-image/upscale”; const headers = { Accept: “image/png”, Authorization: Bearer ${this.apiKey}, “Content-Type”: “application/octet-stream”, }; const timestamp = Date.now(); const filename = /tmp/upscaled_image_${timestamp}.png; // Use streams to read the image file and send it directly to the Upscaler API await pipeline( fs.createReadStream(inputFilename), axios($, { url: path, headers, method: “POST”, responseType: “stream”, }).then((response) => response.data), fs.createWriteStream(filename) ); // Initialize ImageKit instance const imagekit = new ImageKit({ publicKey: this.imageKitPublicKey, privateKey: this.imageKitPrivateKey, urlEndpoint: this.imageKitUrlEndpoint, }); // Upload the upscaled image to ImageKit using streams const result = await new Promise((resolve, reject) => { imagekit.upload( { file: fs.createReadStream(filename), fileName: upscaled_image_${timestamp}.png, }, (error, response) => {

60

continue please