@pipedream/browsers
package into your Node.js code step and launch a browser. Pipedream will start Chromium and launch a Puppeteer or Playwright Browser instance for you.
Usage
The@pipedream/browsers
package exports two modules: puppeteer
& playwright
. Both modules share the same interface:
browser(opts?)
- method to instantiate a new browser (returns a browser instance)launch(opts?)
- an alias to browser()newPage()
- creates a new page instance and returns both the page & browser
Puppeteer
First import thepuppeteer
module from @pipedream/browsers
and use browser()
or launch()
method to instantiate a browser.
Then using this browser you can open new Pages, which have individual controls to open URLs:
Screenshot a webpage
Puppeteer can take a full screenshot of a webpage rendered with Chromium. For full options see the Puppeteer Screenshot method documentation.Save a screenshot within the local
/tmp
directory:Generate a PDF of a webpage
Puppeteer can render a PDF of a webpage. For full options see the Puppeteer Screenshot method documentation.Save the PDF locally to
/tmp
:Scrape content from a webpage
Puppeteer can scrape individual elements or return all content of a webpage.Extract individual elements with a CSS selector:
Submit a form
Puppeteer can also programmatically click and type on a webpage.Page.type
accepts a CSS selector and a string to type into the field.Playwright
First import theplaywright
module from @pipedream/browsers
and use browser()
or launch()
method to instantiate a browser.
Then using this browser you can open new Pages, which have individual controls to open URLs, click elements, generate screenshots and type and more:
Don’t forget to close the browser contextPlaywright differs from Puppeteer slightly in that you have to close the page’s browser context before closing the browser itself.
Screenshot a webpage
Playwright can take a full screenshot of a webpage rendered with Chromium. For full options see the Playwright Screenshot method documentation.Save a screenshot within the local
/tmp
directory:Generate a PDF of a webpage
Playwright can render a PDF of a webpage. For full options see the Playwright Screenshot method documentation.Save a PDF locally to
/tmp
:Scrape content from a webpage
Playwright can scrape individual elements or return all content of a webpage.Extract individual HTML elements using a CSS Selector:
Submit a form
Playwright can also programmatically click and type on a webpage.Page.type
accepts a CSS selector and a string to type into the selected element.FAQ
Can I use this package in sources or actions?
Yes, the same@pipedream/browsers
package can be used in actions as well as sources.
The steps are the same as usage in Node.js code. Open a browser, create a page, and close the browser at the end of the code step.
Memory limitsAt this time it’s not possible to configure the allocated memory to a Source. You may experience a higher rate of Out of Memory errors on Sources that use Puppeteer or Playwright due to the high usage of memory required by Chromium.
Workflow exited before step finished execution
Remember to close the browser instance before the step finishes. Otherwise, the browser will keep the step “open” and not transfer control to the next step.Out of memory errors or slow starts
For best results, we recommend increasing the amount of memory available to your workflow to 2 gigabytes. You can adjust the available memory in the workflow settings.Which browser are these packages using?
The@pipedream/browsers
package includes a specific version of Chromium that is compatible with Pipedream Node.js execution environments that run your code.
For details on the specific versions of Chromium, puppeeter and playwright bundled in this package, visit the package’s README.
How to customize puppeteer.launch()
?
To pass arguments to puppeteer.launch()
to customize the browser instance, you can pass them directly to puppeteer.browser()
.
For example, you can alter the protocolTimeout
length just by passing it as an argument:
@pipedream/browsers
source code for the default arguments that Pipedream provides.
How do I use puppeteer.connect()
?
To use puppeteer.connect()
to connect to a remote browser instance, you can use the puppeteer-core
package:
puppeteer-core
does not download Chrome when installed, which decreases the size of your deployment and can improve cold start times.
To connect to a remote browser instance using Playwright, you can use the playwright-core
package, which is the no-browser Playwright package: