This topic was automatically generated from Slack. You can find the original thread here.
I have a built a web scraper action that i use in a few different workflows, all the existing workflows (deployed) work fine, but the action in any new workflow or an old workflow in edit mode throws error code 401. Seems weird how the same code would simultaneously work and not work with no changes made to it.
It sounds like you’re experiencing an issue with a custom action in your Pipedream workflows, where the action works in deployed workflows but throws a 401 Unauthorized error in new workflows or when editing existing ones. This error typically indicates an issue with authentication or authorization. Given the nature of the problem, here are a few steps you can take to troubleshoot and potentially resolve the issue: 1. Check the Connected Account: If your action relies on a connected account for authentication (for example, an API key or OAuth token), ensure that the account is correctly configured and has the necessary permissions. It’s possible that the credentials work for existing workflows due to previously established sessions or tokens but fail for new instances due to expired or revoked credentials. 2. Review the Action’s Authentication Method: Examine how your action authenticates with the target service. Ensure that the authentication method and credentials are correctly implemented in your action code. If you’re using an API key, token, or OAuth, verify that these are correctly passed in the HTTP request headers or body. 3. Inspect the Scope of Connected Accounts: If your action uses a connected account shared within a workspace, ensure that the account’s access is correctly configured. Sometimes, access control settings might restrict new workflows or edited workflows from using the connected account, leading to authorization errors. 4. Test the Action Independently: Try to isolate the issue by creating a minimal workflow that only contains the problematic action. This can help determine if the issue is specific to the action or influenced by other factors in your workflows. 5. Consult the Pipedream Documentation and Community: If you haven’t already, review the Pipedream troubleshooting guide for additional tips. You might also find it helpful to reach out to the Pipedream community for support or to see if others have encountered similar issues. If after these steps you’re still facing the issue, it might be helpful to review any recent changes to the web scraping service you’re interacting with, as API changes or updates to authentication mechanisms can sometimes cause such behavior. For further assistance, you can visit Pipedream support for more support options.
Hi . The Pipedream runtime was recently updated to support Node.js 20, and re-deploying or testing a workflow uses the new runtime.
Does the error provide any additional information?
Does your web scraper action use a pinned puppeteer-core or playwright-core package version? If so, you may need to pin it to the latest version. Alternatively, you could use the Pipedream-managed @pipedream/browsers npm package, which is compatible with Pipedream’s Node.js execution environment.
Hi Jacob, thank you replying! Currently, I am using pipedream/browsers to import puppeteer. Is that what you were referring to in your answer?
Yes, I was referring to using pipedream/browsers. The first example from the docs works for me when testing a code step (in edit mode).
the following is part of what i get -
message: ‘Request failed with status code 401’,
name: ‘AxiosError’,
code: ‘ERR_BAD_REQUEST’,
headers: Object [AxiosHeaders] {
Accept: ‘application/json, text/plain, /’,
‘Content-Type’: ‘application/json’,
‘User-Agent’: ‘axios/1.6.7’,
‘Accept-Language’: ‘en-US,en;q=0.5’,
‘Accept-Encoding’: ‘gzip, deflate, br’,
Authorization: undefined,
‘Requested-From’: undefined
},
Thanks. A 401 error status code suggests that the request lacks valid authentication credentials. Are you providing credentials to the action using an environment variable or a connected account or neither?
I get an auth token after logging into the website i want to scrape
Would you mind sharing a simple code snippet that I could use to reproduce your issue?