Why is the Google Drive API not allowing to rename and properly place a files sent by Python script?

Help
1

This topic was automatically generated from Slack. You can find the original thread here.

Further to the above, I have it sending the file to Google drive, but it won't let me name the file or put it in the right parent. It always appears as Untitled.  The data object doesn't seem to send properly.

import { axios } from "@pipedream/platform";

export default defineComponent({
  props: {
    twitter: {
      type: "app",
      app: "twitter",
    },
    tweet_id: {
      type: "string",
      label: "Tweet ID",
      description: "The ID of the tweet to retrieve images from",
    },
    google_drive: {
      type: "app",
      app: "google_drive",
    },
  },
  async run({ steps, $ }) {
    const response = await axios($, {
      method: "GET",
      url: `https://api.twitter.com/2/tweets/${this.tweet_id}`,
      params: {
        "tweet.fields": "attachments",
        "media.fields": "url",
        "expansions": "attachments.media_keys",
      },
      headers: {
        Authorization: `Bearer ${this.twitter.$auth.oauth_access_token}`,
      },
    });

    const mediaKeys = response.includes?.media?.map(media => media.media_key);
    const images = response.includes?.media
      ?.filter(media => mediaKeys.includes(media.media_key) && media.type === "photo")
      .map(photo => ({ url: photo.url }));
    for (const image of images) {
      await axios($, {
        method: "POST",
        url: `https://www.googleapis.com/upload/drive/v3/files?uploadType=media`,
        headers: {
          Authorization: `Bearer ${this.google_drive.$auth.oauth_access_token}`,
          "Content-Type": "image/jpeg",
        },
        data: {
          name: "TweetImage.jpg",
          parents: "1Uv_REDACTED7WzXE8h4",
        },
        data: await axios($, { url: image.url, responseType: "arraybuffer" }),
      });
    }
    return images;
  },
});

image.png
image.png1340×76 4.03 KB

2

Hi , I’m curious if creating a custom source interests you

3

You can develop your custom source that gets the images for a tweet and emits one event for each image

4

Then you could use the Google Drive - Upload File action and easily select the Folder and Name

5

In the (near) future, we will ship Control Flow and it will allow you do configure loops in one workflow

6

Thanks, . The short answer is I am open to any solution. The longer answer is that I don’t know if I have the skillset to build one of these from scratch. I have been relying on your generative Code AI and some basic examples from the web. I have a coding background, but it’s been years since it’s been in use. I will read the references you provided and see if I can figure this out.

7

I actually have your idea working, but I hit another roadblock. I can do this with Tweet Mentions because those are public, however to capture images from DMs there is a stricter authentication. I can see this in the “New DM” trigger code:

      const oauth = new OAuth({
        consumer,
        signature_method: "HMAC-SHA1",
        hash_function(base_string, key) {
          return crypto
            .createHmac("sha1", key)
            .update(base_string)
            .digest("base64");
        },
      });

      const token = {
        key,
        secret,
      };

      const requestData = transformConfigForOauth(config);

      return oauth.toHeader(oauth.authorize(requestData, token));
    },

But I cannot figure out how to replicate that and the existing trigger doesn’t get the image URLs for the me to pass to google drive…

8

can you please explain this oauth validation?

      const oauth = new OAuth({
        consumer,
        signature_method: "HMAC-SHA1",
        hash_function(base_string, key) {
          return crypto
            .createHmac("sha1", key)
            .update(base_string)
            .digest("base64");
        },
      });

      const token = {
        key,
        secret,
      };

      const requestData = transformConfigForOauth(config);

      return oauth.toHeader(oauth.authorize(requestData, token));
9

The code snippet you’ve provided is an example of OAuth 1.0a signature generation for authenticating HTTP requests. OAuth 1.0a is a protocol for securing API communications, ensuring that the request is being made by an authorized consumer and has not been tampered with during transit. Here’s a breakdown of the key components in your snippet: 1. OAuth Object Creation: An OAuth object is instantiated with configuration details necessary for the signature process. This includes: - consumer: An object containing the key and secret provided by the service you’re trying to access. These are unique to your application and are used to identify and authenticate it. - signature_method: Specifies the method used to generate the signature, in this case, “HMAC-SHA1”. - hash_function: A function that takes a base string and a key to generate a hash signature. Here, it uses the HMAC-SHA1 algorithm to create a hash of the base string using the key, then encodes the hash in base64. 2. Token: This represents the OAuth token provided by the service, which includes a key and a secret. These are used in conjunction with the consumer key and secret to generate the signature. 3. Request Data Transformation: transformConfigForOauth(config) is presumably a function that prepares the HTTP request configuration (config) for OAuth authorization. This might involve adding necessary OAuth parameters or formatting the request in a specific way. 4. Signature Generation and Authorization Header: The oauth.authorize method is called with the transformed request data and the token. This method generates the OAuth signature based on the request details, consumer key/secret, and token key/secret. The oauth.toHeader method then takes this signature and constructs an Authorization header that can be added to the HTTP request. This header contains the OAuth parameters and the generated signature, which the server uses to validate the request. In summary, this code snippet is used to authenticate an HTTP request using OAuth 1.0a by generating a signature and including it in the request’s Authorization header. The server receiving the request will use this information to verify the request’s authenticity.

10

ensuring that the request is being made by an authorized consumer and has not been tampered with during transit
In summary, this code snippet is used to authenticate an HTTP request using OAuth 1.0a by generating a signature and including it in the request’s Authorization header. The server receiving the request will use this information to verify the request’s authenticity.
While it’s not absolutely required, it is good practice to do secret validation

11

Is there an API reference docs that you’re looking at for listing the images in a tweet?

12

Some background, the references I am using are: For Tweets, for DMs

For public tweets I can do this:

const response = await axios($, {
      method: "GET",
      url: `https://api.twitter.com/2/tweets/${this.tweet_id}`,
      params: {
        "tweet.fields": "attachments",
        "media.fields": "url",
        "expansions": "attachments.media_keys",
      },
      headers: {
        Authorization: `Bearer ${this.twitter.$auth.oauth_access_token}`,
      },
    });

However for DMs, because those are private, the same authentication, just changing the endpoint to the DM endpoint:

url: `https://api.twitter.com/2/dm_conversations/${this.tweet_id}/dm_events`,

results in this error:


{
  "title": "Unsupported Authentication",
  "detail": "Authenticating with OAuth 2.0 Application-Only is forbidden for this endpoint. Supported authentication types are [OAuth 1.0a User Context, OAuth 2.0 User Context].",
  "type": "https://api.twitter.com/2/problems/unsupported-authentication",
  "status": 403
}

Upon investigating, it lead me to the need for the authentication method, which is found in the “New DM Trigger” I posted above.

One possible solution would be to add the media params to the existing get DM trigger in PipeDream, I was trying to write my own and got stuck on the authentication.

13

Hi I wanted to update here. I used your generative AI code tool and then some copy/psate from another example and got the authentication to work! That’s excellent. the problem is, it appears the session doesn’t hold for the entire workflow.

So my code object returns the authenticated response object that has a list of any image URLS, in the next step, I take the URL and use an “upload to drive” action with that url, but I get a filestream error “null.getFileStream” because it needs an authenticated session to access that url (presumably).

image.png
image.png1413×871 37.5 KB

14

I presume if I can combine the upload in the code step itself, it will maintain the authenticated session… Is that correct?

15

Hi , thanks for the updates. I think you’re right, access to the download link requires the authenticated credentials and the Google Drive step isn’t passing them into the request.

16

What you can do is add an intermediate step to download the files to the /tmp folder

17

And then use the File Path prop in the GD - Upload File action instead of File URL

18

Does that make sense? Feel free to use the AI codegen or ask Pi here if you need code suggestions

19

The idea makes sense. and I am copying the return object to /tmp, but I think my writeFile() function is not correct as the file size isn’t correct and shows as corrupt. I am investigating, this is my code, not Pipedream. Thanks for the suggestions!

20

How long to files stay in /tmp?