Why am I still experiencing memory issues in Pipedream despite only accessing a Google Sheet and allocating 1G of memory?

Help
1

This topic was automatically generated from Slack. You can find the original thread here.

Still having memory issues, we tried to fix it the other day. I actually found a way to not have to download my video at all into Pipedream so it’s only accessing a Google Sheet, I have it allocated 1G of memory, even though it should never be using more then 1 or 2 mb of storage, at most if it downloaded the video it would use 40-50 mb yet it’s still giving me memory errors on what looks like just the trigger. Sorry I’m on my phone right now

IMG_2354.png
IMG_2354.png2796×1290 269 KB

IMG_2352.png
IMG_2352.png1290×2796 398 KB

IMG_2353.png
IMG_2353.png2796×1290 218 KB

2

Support viewing is enabled.
https://pipedream.com/@weirdorfunnyfacts/projects/proj_zNsq4pq/youtube-random-facts-p_zAC57xB/inspect

3

The error on the trigger is just a visual glitch.

4

OOM errors are unable to display the specific step where the workflow crashed. :disappointed:

5

I have to say I’m surprised that 1GB of memory is not enough… we have workflows with just as many steps running on 512MB. :man-shrugging:

6

Hopefully Pipedream fixes this soon, and gives us the ability to monitor memory usage per workflow and per step.

7

It will be fun to start optimizing steps to use minimal memory once they do. :grin:

8

Hmm, ya that would be awesome :cry: hopefully soon so we can see where our mistakes are, or where its messing up

9

Hi , actually, the Youtube Upload Video action does stream the file to Youtube, so it essentially loads the whole video in memory

10

Check out the code here:
• https://github.com/PipedreamHQ/pipedream/blob/master/components/youtube_data_api/actions/upload-video/upload-video.mjs
• https://github.com/PipedreamHQ/pipedream/blob/master/components/youtube_data_api/actions/upload-video/common.mjs