Generate Screenshot From HTML with Urlbox.io API on New Field in Document from MongoDB API

Pipedream makes it easy to connect APIs for Urlbox.io, MongoDB and 2,400+ other apps remarkably fast.

Trigger workflow on

New Field in Document from the MongoDB API

Next, do this

Generate Screenshot From HTML with the Urlbox.io API

No credit card required

▶

Watch us build a workflow

8 min

Watch now ➜

Trusted by 1,000,000+ developers from startups to Fortune 500 companies

Developers ♥ Pipedream

Getting Started#

This integration creates a workflow with a MongoDB trigger and Urlbox.io action. When you configure and deploy the workflow, it will run on Pipedream's servers 24x7 for free.

Select this integration
Configure the New Field in Document trigger
1. Connect your MongoDB account
2. Configure Polling Interval
3. Select a Database
4. Select a Collection
5. Select a Document
Configure the Generate Screenshot From HTML action
1. Connect your Urlbox.io account
2. Configure HTML
3. Select a Format
4. Optional- Configure Width
5. Optional- Configure Height
6. Optional- Configure Full Page
7. Optional- Configure Selector
8. Optional- Configure GPU
9. Optional- Configure Block Ads
10. Optional- Configure Block Urls
11. Optional- Configure Hide Cookie Banners
12. Optional- Configure Click Accept
13. Optional- Configure Hide Selector
14. Optional- Configure JS
15. Optional- Configure CSS
16. Optional- Configure Dark Mode
17. Optional- Configure Retina
18. Optional- Configure Thumb Width
19. Optional- Configure Thumb Height
20. Optional- Configure Quality
21. Optional- Configure Transparent
22. Optional- Configure Max Height
23. Optional- Configure Download
24. Optional- Select a PDF Page Size
25. Optional- Configure PDF Page Width
26. Optional- Configure PDF Page Height
27. Optional- Select a PDF Margin
28. Optional- Configure PDF Margin Top
29. Optional- Configure PDF Margin Right
30. Optional- Configure PDF Margin Bottom
31. Optional- Configure PDF Margin Left
32. Optional- Configure PDF Scale
33. Optional- Select a PDF Orientation
34. Optional- Configure PDF Background
35. Optional- Configure Disable Ligatures
36. Optional- Select a Media
37. Optional- Configure Force
38. Optional- Configure Unique
39. Optional- Configure ttl
40. Optional- Configure Proxy
41. Optional- Configure Header
42. Optional- Configure Cookie
43. Optional- Select a User Agent
44. Optional- Configure Accept Lang
45. Optional- Configure Authorization
46. Optional- Configure TZ
47. Optional- Configure Delay
48. Optional- Configure Timeout
49. Optional- Select a Wait Until
50. Optional- Configure Wait For
51. Optional- Configure Wait To Leave
52. Optional- Configure Wait Timeout
53. Optional- Configure Scroll To
54. Optional- Configure Click
55. Optional- Configure Click All
56. Optional- Configure Hover
57. Optional- Configure Bg Color
58. Optional- Configure Disable JS
59. Optional- Configure Full Width
60. Optional- Configure Allow Infinite
61. Optional- Configure Skip Scroll
62. Optional- Configure Detect Full Height
63. Optional- Configure Max Section Height
64. Optional- Configure Scroll Increment
65. Optional- Configure Scroll Delay
66. Optional- Configure Turbo
67. Optional- Configure Highlight
68. Optional- Configure Highlight Fg
69. Optional- Configure Highlight Bg
70. Optional- Configure Latitude
71. Optional- Configure Longitude
72. Optional- Configure Accuracy
73. Optional- Configure Use S3
74. Optional- Configure S3 Path
75. Optional- Configure S3 Bucket
76. Optional- Select a S3 Storage Class
77. Optional- Configure Fail If Selector Missing
78. Optional- Configure Fail If Selector Present
79. Optional- Configure Fail On 4XX
80. Optional- Configure Fail On 5XX
Deploy the workflow
Send a test event to validate your setup
Turn on the trigger

Details#

This integration uses pre-built, source-available components from Pipedream's GitHub repo. These components are developed by Pipedream and the community, and verified and maintained by Pipedream.

To contribute an update to an existing component or create a new component, create a PR on GitHub. If you're new to Pipedream component development, you can start with quickstarts for trigger span and action development, and then review the component API reference.

Trigger#

New Field in Document on MongoDB

Description:Emit new an event when a new field is added to a document

Version:0.0.7

Key:mongodb-new-field-in-document

View on GitHub

MongoDB Overview#

The MongoDB API provides powerful capabilities to interact with a MongoDB database, allowing you to perform CRUD (Create, Read, Update, Delete) operations, manage databases, and execute sophisticated queries. With Pipedream, you can harness these abilities to automate tasks, sync data across various apps, and react to events in real-time. It’s a combo that’s particularly potent for managing data workflows, syncing application states, or triggering actions based on changes to your data.

Trigger Code#

import common from "../common/base.mjs";

export default {
  ...common,
  key: "mongodb-new-field-in-document",
  name: "New Field in Document",
  description: "Emit new an event when a new field is added to a document",
  version: "0.0.7",
  type: "source",
  dedupe: "unique",
  props: {
    ...common.props,
    database: {
      propDefinition: [
        common.props.mongodb,
        "database",
      ],
    },
    collection: {
      propDefinition: [
        common.props.mongodb,
        "collection",
        (c) => ({
          database: c.database,
        }),
      ],
    },
    document: {
      propDefinition: [
        common.props.mongodb,
        "document",
        (c) => ({
          database: c.database,
          collection: c.collection,
        }),
      ],
    },
  },
  methods: {
    ...common.methods,
    _getFields() {
      return this.db.get("fields");
    },
    _setFields(fields) {
      this.db.set("fields", fields);
    },
    isRelevant(key, fields) {
      return !fields.includes(key);
    },
    async processEvent(client, ts) {
      const fields = this._getFields() || [];
      const collection = this.mongodb.getCollection(client, this.database, this.collection);
      const documents = await this.mongodb.listDocuments(collection);
      const doc = documents.find((obj) => {
        return obj._id == this.document;
      });
      const keys = Object.keys(doc);
      for (const key of keys) {
        if (!this.isRelevant(key, fields)) {
          continue;
        }
        fields.push(key);
        this.emitEvent(key, ts);
      }
      this._setFields(fields);
    },
    generateMeta(key, ts) {
      return {
        id: key,
        summary: `New Field ${key}`,
        ts,
      };
    },
  },
};

Trigger Configuration#

This component may be configured based on the props defined in the component code. Pipedream automatically prompts for input values in the UI and CLI.

Label	Prop	Type	Description
MongoDB	`mongodb`	`app`	This component uses the MongoDB app.
N/A	`db`	`$.service.db`	This component uses `$.service.db` to maintain state between executions.
Polling Interval	`timer`	`$.interface.timer`	Pipedream will poll the API on this schedule
Database	`database`	`string`	Select a value from the drop down menu.
Collection	`collection`	`string`	Select a value from the drop down menu.
Document	`document`	`string`	Select a value from the drop down menu.

Trigger Authentication#

MongoDB uses API keys for authentication. When you connect your MongoDB account, Pipedream securely stores the keys so you can easily authenticate to MongoDB APIs in both code and no-code steps.

Before you connect to your Mongo database from Pipedream, please make sure your database is accessible from the public internet. You may need to add a firewall rule allowing connections from 0.0.0.0/0.

Then, enter the details of your Mongo host / database below.

About MongoDB#

MongoDB is an open source NoSQL database management program.

Action#

Generate Screenshot From HTML on Urlbox.io

Description:Generate a screenshot of a website provided by a html. [See the docs here](https://www.urlbox.io/docs/examplecode/node)

Version:0.0.2

Key:urlbox_io-generate-screenshot-from-html

View on GitHub

Urlbox.io Overview#

The Urlbox.io API lets you capture live, high-quality screenshots of web pages programmatically, offering a range of customization options such as setting the viewport size, format, and full-page capture. On Pipedream, this powerful tool can be integrated into workflows to automate screenshot capture for archiving, monitoring, and reporting tasks, or as part of a larger data collection and analysis pipeline.

Action Code#

import common from "../common/base.mjs";

export default {
  ...common,
  key: "urlbox_io-generate-screenshot-from-html",
  name: "Generate Screenshot From HTML",
  description: "Generate a screenshot of a website provided by a html. [See the docs here](https://www.urlbox.io/docs/examplecode/node)",
  version: "0.0.2",
  type: "action",
  props: {
    urlboxIo: common.props.urlboxIo,
    html: {
      propDefinition: [
        common.props.urlboxIo,
        "html",
      ],
    },
    ...common.props,
  },
};

Action Configuration#

This component may be configured based on the props defined in the component code. Pipedream automatically prompts for input values in the UI.

Label	Prop	Type	Description
Urlbox.io	`urlboxIo`	`app`	This component uses the Urlbox.io app.
HTML	`html`	`string`	The html that you want to take a screenshot.
Format	`format`	`string`	Select a value from the drop down menu:`pngjpgavifwebppdfsvghtml`
Width	`width`	`integer`	Viewport width of the browser in pixels. Default is 1280.
Height	`height`	`integer`	Viewport height of the browser in pixels. Default is 1024.
Full Page	`fullPage`	`boolean`	Specify whether to capture the full-length of the website. Default is FALSE.
Selector	`selector`	`string`	Take a screenshot of the element that matches this selector.
GPU	`gpu`	`boolean`	Enable GPU acceleration to render 3D scenes and heavy WebGL content. This is a beta feature and requires pre-approval. Please contact support to enable this feature on your account.
Block Ads	`blockAds`	`boolean`	Blocks requests from popular advertising networks from loading.
Block Urls	`blockUrls`	`string`	Block requests from specific urls from loading.
Hide Cookie Banners	`hideCookieBanners`	`boolean`	Automatically hides cookie banners from most websites.
Click Accept	`clickAccept`	`boolean`	Automatically clicks 'Accept' buttons, in order to dismiss popups.
Hide Selector	`hideSelector`	`string`	Hides all elements that match the element selector by setting their style to `display:none !important;`. Useful for hiding popups.
JS	`js`	`string`	Inject (and execute) custom JavaScript on the page.
CSS	`css`	`string`	Inject custom CSS into the page.
Dark Mode	`darkMode`	`boolean`	Emulate dark mode on websites by setting prefers-color-scheme: dark.
Retina	`retina`	`boolean`	Take a 'retina' or high definition screenshot equivalent to setting a device pixel ratio of 2.0 or @2x. Please note that retina screenshots will be double the normal dimensions and will normally take slightly longer to process due to the much bigger image size. Default is FALSE
Thumb Width	`thumbWidth`	`string`	The width of the generated thumbnail, in pixels. Omit for a full-size screenshot.
Thumb Height	`thumbHeight`	`string`	The height of the generated thumbnail, in pixels. Omit for a full-size screenshot.
Quality	`quality`	`integer`	The image quality of the resulting screenshot (JPEG/WebP only). Default is 80.
Transparent	`transparent`	`boolean`	If a website has no background color set, the image will have a transparent background (PNG/WebP only).
Max Height	`maxHeight`	`string`	For extremely lengthy websites, it may be preferable to limit the screenshot to a maximum height to prevent Urlbox from spending time scrolling and generating an enormous screenshot.
Download	`download`	`string`	Pass in a filename which sets the content-disposition header on the response. E.g. `download=myfilename.png` This will make the Urlbox link downloadable, and will prompt the user to save the file as `myfilename.png`
PDF Page Size	`pdfPageSize`	`string`	Select a value from the drop down menu:`A0A1A2A3A4A5A6LegalLetterLedgerTabloid`
PDF Page Width	`pdfPageWidth`	`integer`	Sets the PDF page width, in pixels.
PDF Page Height	`pdfPageHeight`	`integer`	Sets the PDF page height, in pixels.
PDF Margin	`pdfMargin`	`string`	Select a value from the drop down menu:`nonedefaultminimum`
PDF Margin Top	`pdfMarginTop`	`integer`	Sets a custom top margin on the PDF.
PDF Margin Right	`pdfMarginRight`	`integer`	Sets a custom right margin on the PDF.
PDF Margin Bottom	`pdfMarginBottom`	`integer`	Sets a custom bottom margin on the PDF.
PDF Margin Left	`pdfMarginLeft`	`integer`	Sets a custom left margin on the PDF.
PDF Scale	`pdfScale`	`string`	Sets the scale factor of the website content in the PDF. Valid values are numbers between `0.1` and `2`. Default is `1`
PDF Orientation	`pdfOrientation`	`string`	Select a value from the drop down menu:`portraitlandscape`
PDF Background	`pdfBackground`	`boolean`	Sets whether to print background images in the PDF.
Disable Ligatures	`disableLigatures`	`boolean`	Prevents ligatures from being used. Useful when rendering a PDF, and you want to extract text which contains ligatures.
Media	`media`	`string`	Select a value from the drop down menu:`printscreen`
Force	`force`	`boolean`	Generate a fresh screenshot or PDF, instead of getting a cached version.
Unique	`unique`	`string`	Pass a unique string such as a UUID, hash or timestamp, to have more control over when to generate a fresh screenshot or PDF. For example, you could do something like unique=$1681161683702 to get a new value for unique each time it is called.
ttl	`ttl`	`integer`	Short for 'time to live'. Number of seconds to keep a screenshot in the cache. Note the default is also the maximum value for this option. Default is `2592000 (30 days - this is also the maximum value)`.
Proxy	`proxy`	`string`	Pass in a proxy server address to make screenshot requests via that server in the format [address]:[port]. If proxy authentication is required, you can use the following format: [user]:[password]@[address]:[port].
Header	`header`	`string`	Set a header on the request when loading the URL. Example: To set the header with key `X-My-Header` to the value `SomeValue`, you would pass `header=X-My-Header%3DSomeValue`. This can be set multiple times, to set more than one header - e.g. `header=X-My-Header%3DSomeValue&header=X-My-Other-Header%3DSomeOtherValue`. As with all options passed via the query string, the header value must be URL encoded - so `X-My-Header=SomeValue` becomes `X-My-Header%3DSomeValue` in order to be interpreted correctly by Urlbox.
Cookie	`cookie`	`string`	Sets a cookie on the request when loading the URL. Example: To set the cookie with key `Opt-In` to the value `yes`, you would pass `cookie=Opt-In%3Dyes`. This can be set multiple times, to set more than one cookie - e.g.`cookie=Opt-In%3Dyes&cookie=Session-Id%3DMTIzNDU`. As with all options passed in the query string, the cookie value must be URL encoded - so `OptIn=true` becomes `OptIn%3Dtrue` in order to be interpreted correctly by Urlbox.
User Agent	`userAgent`	`string`	Select a value from the drop down menu:`randommobiledesktopGooglebot/2.1 (+http://www.google.com/bot.html)facebookexternalhit/1.1`
Accept Lang	`acceptLang`	`string`	Sets an Accept-Language header on requests to the target URL. Default is `en-US`.
Authorization	`authorization`	`string`	Sets an `Authorization` header on requests to the target URL. Can be used to pass an auth token through to the site in order to 'login' before rendering.
TZ	`tz`	`string`	Sets the local timezone to use for the screenshot. Default is `UTC`
Delay	`delay`	`string`	Amount of time to wait in milliseconds before urlbox takes the screenshot.
Timeout	`timeout`	`integer`	Amount of time to wait in milliseconds for the website at url to respond. Default is 30000.
Wait Until	`waitUntil`	`string`	Select a value from the drop down menu:`downloadedmostrequestfinishedrequestsfinishedloaded`
Wait For	`waitFor`	`string`	Waits for the element specified by this selector to be present in the DOM before taking a screenshot or pdf. By default, Urlbox will take a screenshot or PDF if the `wait_for` element is not found after waiting for the time specified by the `wait_timeout` option. If you prefer Urlbox to fail the request when the `wait_for` element is not found, pass `fail_if_selector_missing=true`
Wait To Leave	`waitToLeave`	`string`	Waits for the element specified by this selector to be absent from the DOM before taking a screenshot or PDF. A typical use-case would be waiting for loading spinners to be absent before taking a screenshot. By default, Urlbox will take a screenshot or PDF if the `wait_to_leave` element is still present after the time specified by the `wait_timeout` option. If you prefer Urlbox to fail the request when the `wait_to_leave` element is still present, pass `fail_if_selector_present=true`
Wait Timeout	`waitTimeout`	`integer`	The amount of time to wait for the `wait_for` element to be available before continuing, in milliseconds. Default is `30000 (30 seconds)`
Scroll To	`scrollto`	`string`	Scroll, to either an element or to a pixel offset from the top, before taking a screenshot of PDF.
Click	`click`	`string`	Specifies an element selector to click before generating a screenshot or PDF. Example: `#clickme` would click an element with `id="clickme"`. Can be used multiple times to simulate multiple sequential click events. If the selector matches multiple elements, only the first element will be clicked.
Click All	`clickAll`	`string`	Specifies an element selector to click before generating a screenshot or PDF. Example: `.clickme` would click all elements with `class="clickme"`. Can be used multiple times to simulate multiple sequential click events. If the selector matches multiple elements, all elements will be clicked.
Hover	`hover`	`string`	Specifies an element selector to hover over before generating a screenshot or PDF. Example: `#hoverme` would hover over the element with `id="clickme".`
Bg Color	`bgColor`	`string`	Specify a hex code or CSS color string to use as the background color. Some websites don't set a body background colour, and will show up as transparent backgrounds with PNG, or black when using JPG. Use this setting to set a background colour. If the website explicitly sets a transparent background on the html or body elements, this setting will be overridden.
Disable JS	`disable_js`	`boolean`	Turn off javascript on target url to prevent popups. Enabling this option will prevent full_page=true and many other options, because having javascript disabled prevents Urlbox from evaluating code inside the page's context.
Full Width	`fullWidth`	`boolean`	When full_page=true, specify whether to capture the full width of the website, for example if the site is horizontally scrolling.
Allow Infinite	`allowInfinite`	`boolean`	By default, when Urlbox detects an infinite scrolling page, it does not attempt to continue scrolling to the bottom, as this could result in infinite scrolling! If you want to override this behaviour, pass `true` for this option.
Skip Scroll	`skipScroll`	`boolean`	Enabling `skip_scroll` will speed up renders by skipping an initial scroll through the page, which is used to trigger any lazy loading elements.
Detect Full Height	`detectFullHeight`	`string`	Some pages have full-height backgrounds whose heights are set to 100% of the viewport. This can cause the backgrounds to get stretched when making a full page screenshot. If you are seeing this behaviour in your full page screenshots, pass `true` for this option.
Max Section Height	`maxSectionHeight`	`integer`	When Urlbox takes a `full_page` screenshot, the maximum height of each image section is set to 4096 pixels. If a sites height is greater than this value, Urlbox will start splitting the screenshot into sections. Sometimes it is worthwhile experimenting with this number. Default is `4096`.
Scroll Increment	`scrollIncrement`	`string`	Sets how many pixels to scroll when scrolling the page to trigger lazy loading elements. By default, the scroll increment is set to the browser viewport height. Some pages' lazy loading elements only trigger when the scroll increment is smaller than this, however, e.g. 400px.
Scroll Delay	`scrollDelay`	`string`	When Urlbox decides to split a screenshot into multiple sections, the scroll delay is the time to wait between taking the screenshots of each individual section, in milliseconds. While Urlbox does detect animations, and attempts to wait for them before taking a screenshot, this option could be used to force Urlbox to wait for a certain amount of time after scrolling to the next section, to wait for things like animations to finish.
Turbo	`turbo`	`string`	Warning This is an experimental option. Used to speed up full page screenshots, but at the expense of accuracy.
Highlight	`highlight`	`string`	Word to highlight on the page before capturing a screenshot.
Highlight Fg	`highlightfg`	`string`	Text color of the highlighted word. Default is `white`.
Highlight Bg	`highlightbg`	`string`	Background color of the highlighted word. Default is `red`.
Latitude	`latitude`	`string`	Sets the latitude used to emulate the Geolocation API.
Longitude	`longitude`	`string`	Sets the longitude used to emulate the Geolocation API.
Accuracy	`accuracy`	`string`	Sets the accurate of the Geolocation API, in metres.
Use S3	`useS3`	`boolean`	Save the screenshot directly to the S3 bucket configured on your account.
S3 Path	`s3Path`	`string`	Sets the S3 path, including subdirectories and the filename, to use when saving the screenshot in your S3 bucket. The extension (.png, .jpg or .pdf) will be provided automatically, and should not be included in s3_path.
S3 Bucket	`s3Bucket`	`string`	Overrides the configured S3 bucket to use when saving the screenshot.
S3 Storage Class	`s3StorageClass`	`string`	Select a value from the drop down menu:`standardstandard_iareduced_redundancyonezone_iaintelligent_tieringglacierdeep_archiveoutposts`
Fail If Selector Missing	`failIfSelectorMissing`	`boolean`	Fails the request if the elements specified by `selector` or `wait_for` options are not found on the page after waiting for `wait_timeout`.
Fail If Selector Present	`failIfSelectorPresent`	`boolean`	Fails the request if the element specified by `wait_to_leave` option is found on the page after waiting for `wait_timeout`.
Fail On 4XX	`failOn4xx`	`boolean`	If `fail_on_4xx=true` and the requested URL returns a status code between 400 and 499, Urlbox will fail the request with error code 400 and the message: `Failed to render. Requested URL returned a 4xx error code and fail_on_4xx was true`.
Fail On 5XX	`failOn5xx`	`boolean`	If `fail_on_5xx=true` and the requested URL returns a status code between 500 and 599, Urlbox will fail the request with error code 400 and message: `Failed to render. Requested URL returned a 5xx error code and fail_on_5xx was true`.