Scrape Search Engine with ScrapingBot API

Pipedream makes it easy to connect APIs for ScrapingBot and 2,400+ other apps remarkably fast.

Trigger workflow on

HTTP requests, schedules and app events

Next, do this

Scrape Search Engine with the ScrapingBot API

No credit card required

▶

Watch us build a workflow

8 min

Watch now ➜

Trusted by 1,000,000+ developers from startups to Fortune 500 companies

Developers ♥ Pipedream

Getting Started#

Create a workflow to Scrape Search Engine with the ScrapingBot API. When you configure and deploy the workflow, it will run on Pipedream's servers 24x7 for free.

Configure the Scrape Search Engine action
1. Connect your ScrapingBot account
2. Select a Search Engine
3. Configure Search
4. Optional- Select a Format
5. Optional- Select a Domain Country
6. Optional- Select a Result Language
Select a trigger to run your workflow on HTTP requests, schedules or app events
Deploy the workflow
Send a test event to validate your setup
Turn on the trigger

Integrations#

Scrape Search Engine with ScrapingBot API on New Requests from HTTP / Webhook API

HTTP / Webhook + ScrapingBot

Try it

Scrape Search Engine with ScrapingBot API on New Submission from Typeform API

Typeform + ScrapingBot

Try it

Scrape Search Engine with ScrapingBot API on New Submission (Instant) from Jotform API

Jotform + ScrapingBot

Try it

Scrape Search Engine with ScrapingBot API on New Custom Webhook Events from Stripe API

Stripe + ScrapingBot

Try it

Scrape Search Engine with ScrapingBot API on New Scheduled Tasks from Pipedream API

Pipedream + ScrapingBot

Try it

Details#

This is a pre-built, source-available component from Pipedream's GitHub repo. The component is developed by Pipedream and the community, and verified and maintained by Pipedream.

To contribute an update to an existing component or create a new component, create a PR on GitHub. If you're new to Pipedream component development, you can start with quickstarts for trigger span and action development, and then review the component API reference.

Scrape Search Engine on ScrapingBot

Description:Use ScrapingBot API to extract specific data from Google or Bing search results. [See the documentation](https://www.scraping-bot.io/web-scraping-documentation/)

Version:0.0.1

Key:scrapingbot-scrape-search-engine

View on GitHub

Code#

import scrapingbot from "../../scrapingbot.app.mjs";

export default {
  key: "scrapingbot-scrape-search-engine",
  name: "Scrape Search Engine",
  description: "Use ScrapingBot API to extract specific data from Google or Bing search results. [See the documentation](https://www.scraping-bot.io/web-scraping-documentation/)",
  version: "0.0.1",
  type: "action",
  props: {
    scrapingbot,
    engine: {
      propDefinition: [
        scrapingbot,
        "engine",
      ],
    },
    search: {
      propDefinition: [
        scrapingbot,
        "search",
      ],
    },
    format: {
      propDefinition: [
        scrapingbot,
        "format",
      ],
    },
    domainCountry: {
      propDefinition: [
        scrapingbot,
        "domainCountry",
      ],
    },
    resultLang: {
      propDefinition: [
        scrapingbot,
        "resultLang",
      ],
    },
  },
  async run({ $ }) {
    const response = await this.scrapingbot.scrapeSearchEngine({
      data: {
        engine: this.engine,
        searchType: "search",
        search: this.search,
        format: this.format,
        domainCountry: this.domainCountry,
        resultLang: this.resultLang,
      },
      $,
    });
    if (response) {
      $.export("$summary", "Successfully scraped search engine results.");
    }
    return response;
  },
};

Configuration#

This component may be configured based on the props defined in the component code. Pipedream automatically prompts for input values in the UI and CLI.

Label	Prop	Type	Description
ScrapingBot	`scrapingbot`	`app`	This component uses the ScrapingBot app.
Search Engine	`engine`	`string`	Select a value from the drop down menu:`googlebing`
Search	`search`	`string`	The term to search for
Format	`format`	`string`	Select a value from the drop down menu:`jsonjson-htmlhtml`
Domain Country	`domainCountry`	`string`	Select a value from the drop down menu:`{ "label": "Andorra", "value": "AD" }{ "label": "United Arab Emirates", "value": "AE" }{ "label": "Afghanistan", "value": "AF" }{ "label": "Antigua and Barbuda", "value": "AG" }{ "label": "Anguilla", "value": "AI" }{ "label": "Albania", "value": "AL" }{ "label": "Armenia", "value": "AM" }{ "label": "Angola", "value": "AO" }{ "label": "Antarctica", "value": "AQ" }{ "label": "Argentina", "value": "AR" }{ "label": "American Samoa", "value": "AS" }{ "label": "Austria", "value": "AT" }{ "label": "Australia", "value": "AU" }{ "label": "Aruba", "value": "AW" }{ "label": "Åland Islands", "value": "AX" }{ "label": "Azerbaijan", "value": "AZ" }{ "label": "Bosnia and Herzegovina", "value": "BA" }{ "label": "Barbados", "value": "BB" }{ "label": "Bangladesh", "value": "BD" }{ "label": "Belgium", "value": "BE" }{ "label": "Burkina Faso", "value": "BF" }{ "label": "Bulgaria", "value": "BG" }{ "label": "Bahrain", "value": "BH" }{ "label": "Burundi", "value": "BI" }{ "label": "Benin", "value": "BJ" }{ "label": "Saint Barthélemy", "value": "BL" }{ "label": "Bermuda", "value": "BM" }{ "label": "Brunei Darussalam", "value": "BN" }{ "label": "Bolivia (Plurinational State of)", "value": "BO" }{ "label": "Bonaire, Sint Eustatius and Saba", "value": "BQ" }{ "label": "Brazil", "value": "BR" }{ "label": "Bahamas", "value": "BS" }{ "label": "Bhutan", "value": "BT" }{ "label": "Bouvet Island", "value": "BV" }{ "label": "Botswana", "value": "BW" }{ "label": "Belarus", "value": "BY" }{ "label": "Belize", "value": "BZ" }{ "label": "Canada", "value": "CA" }{ "label": "Cocos (Keeling) Islands", "value": "CC" }{ "label": "Congo, Democratic Republic of the", "value": "CD" }{ "label": "Central African Republic", "value": "CF" }{ "label": "Congo", "value": "CG" }{ "label": "Switzerland", "value": "CH" }{ "label": "Côte d'Ivoire", "value": "CI" }{ "label": "Cook Islands", "value": "CK" }{ "label": "Chile", "value": "CL" }{ "label": "Cameroon", "value": "CM" }{ "label": "China", "value": "CN" }{ "label": "Colombia", "value": "CO" }{ "label": "Costa Rica", "value": "CR" }{ "label": "Cuba", "value": "CU" }{ "label": "Cabo Verde", "value": "CV" }{ "label": "Curaçao", "value": "CW" }{ "label": "Christmas Island", "value": "CX" }{ "label": "Cyprus", "value": "CY" }{ "label": "Czechia", "value": "CZ" }{ "label": "Germany", "value": "DE" }{ "label": "Djibouti", "value": "DJ" }{ "label": "Denmark", "value": "DK" }{ "label": "Dominica", "value": "DM" }{ "label": "Dominican Republic", "value": "DO" }{ "label": "Algeria", "value": "DZ" }{ "label": "Ecuador", "value": "EC" }{ "label": "Estonia", "value": "EE" }{ "label": "Egypt", "value": "EG" }{ "label": "Western Sahara", "value": "EH" }{ "label": "Eritrea", "value": "ER" }{ "label": "Spain", "value": "ES" }{ "label": "Ethiopia", "value": "ET" }{ "label": "Finland", "value": "FI" }{ "label": "Fiji", "value": "FJ" }{ "label": "Falkland Islands (Malvinas)", "value": "FK" }{ "label": "Micronesia (Federated States of)", "value": "FM" }{ "label": "Faroe Islands", "value": "FO" }{ "label": "France", "value": "FR" }{ "label": "Gabon", "value": "GA" }{ "label": "United Kingdom of Great Britain and Northern Ireland", "value": "GB" }{ "label": "Grenada", "value": "GD" }{ "label": "Georgia", "value": "GE" }{ "label": "French Guiana", "value": "GF" }{ "label": "Guernsey", "value": "GG" }{ "label": "Ghana", "value": "GH" }{ "label": "Gibraltar", "value": "GI" }{ "label": "Greenland", "value": "GL" }{ "label": "Gambia", "value": "GM" }{ "label": "Guinea", "value": "GN" }{ "label": "Guadeloupe", "value": "GP" }{ "label": "Equatorial Guinea", "value": "GQ" }{ "label": "Greece", "value": "GR" }{ "label": "South Georgia and the South Sandwich Islands", "value": "GS" }{ "label": "Guatemala", "value": "GT" }{ "label": "Guam", "value": "GU" }{ "label": "Guinea-Bissau", "value": "GW" }{ "label": "Guyana", "value": "GY" }{ "label": "Hong Kong", "value": "HK" }{ "label": "Heard Island and McDonald Islands", "value": "HM" }{ "label": "Honduras", "value": "HN" }{ "label": "Croatia", "value": "HR" }{ "label": "Haiti", "value": "HT" }{ "label": "Hungary", "value": "HU" }{ "label": "Indonesia", "value": "ID" }{ "label": "Ireland", "value": "IE" }{ "label": "Israel", "value": "IL" }{ "label": "Isle of Man", "value": "IM" }{ "label": "India", "value": "IN" }{ "label": "British Indian Ocean Territory", "value": "IO" }{ "label": "Iraq", "value": "IQ" }{ "label": "Iran (Islamic Republic of)", "value": "IR" }{ "label": "Iceland", "value": "IS" }{ "label": "Italy", "value": "IT" }{ "label": "Jersey", "value": "JE" }{ "label": "Jamaica", "value": "JM" }{ "label": "Jordan", "value": "JO" }{ "label": "Japan", "value": "JP" }{ "label": "Kenya", "value": "KE" }{ "label": "Kyrgyzstan", "value": "KG" }{ "label": "Cambodia", "value": "KH" }{ "label": "Kiribati", "value": "KI" }{ "label": "Comoros", "value": "KM" }{ "label": "Saint Kitts and Nevis", "value": "KN" }{ "label": "Korea (Democratic People's Republic of)", "value": "KP" }{ "label": "Korea, Republic of", "value": "KR" }{ "label": "Kuwait", "value": "KW" }{ "label": "Cayman Islands", "value": "KY" }{ "label": "Kazakhstan", "value": "KZ" }{ "label": "Lao People's Democratic Republic", "value": "LA" }{ "label": "Lebanon", "value": "LB" }{ "label": "Saint Lucia", "value": "LC" }{ "label": "Liechtenstein", "value": "LI" }{ "label": "Sri Lanka", "value": "LK" }{ "label": "Liberia", "value": "LR" }{ "label": "Lesotho", "value": "LS" }{ "label": "Lithuania", "value": "LT" }{ "label": "Luxembourg", "value": "LU" }{ "label": "Latvia", "value": "LV" }{ "label": "Libya", "value": "LY" }{ "label": "Morocco", "value": "MA" }{ "label": "Monaco", "value": "MC" }{ "label": "Moldova, Republic of", "value": "MD" }{ "label": "Montenegro", "value": "ME" }{ "label": "Saint Martin (French part)", "value": "MF" }{ "label": "Madagascar", "value": "MG" }{ "label": "Marshall Islands", "value": "MH" }{ "label": "North Macedonia", "value": "MK" }{ "label": "Mali", "value": "ML" }{ "label": "Myanmar", "value": "MM" }{ "label": "Mongolia", "value": "MN" }{ "label": "Macao", "value": "MO" }{ "label": "Northern Mariana Islands", "value": "MP" }{ "label": "Martinique", "value": "MQ" }{ "label": "Mauritania", "value": "MR" }{ "label": "Montserrat", "value": "MS" }{ "label": "Malta", "value": "MT" }{ "label": "Mauritius", "value": "MU" }{ "label": "Maldives", "value": "MV" }{ "label": "Malawi", "value": "MW" }{ "label": "Mexico", "value": "MX" }{ "label": "Malaysia", "value": "MY" }{ "label": "Mozambique", "value": "MZ" }{ "label": "Namibia", "value": "NA" }{ "label": "New Caledonia", "value": "NC" }{ "label": "Niger", "value": "NE" }{ "label": "Norfolk Island", "value": "NF" }{ "label": "Nigeria", "value": "NG" }{ "label": "Nicaragua", "value": "NI" }{ "label": "Netherlands", "value": "NL" }{ "label": "Norway", "value": "NO" }{ "label": "Nepal", "value": "NP" }{ "label": "Nauru", "value": "NR" }{ "label": "Niue", "value": "NU" }{ "label": "New Zealand", "value": "NZ" }{ "label": "Oman", "value": "OM" }{ "label": "Panama", "value": "PA" }{ "label": "Peru", "value": "PE" }{ "label": "French Polynesia", "value": "PF" }{ "label": "Papua New Guinea", "value": "PG" }{ "label": "Philippines", "value": "PH" }{ "label": "Pakistan", "value": "PK" }{ "label": "Poland", "value": "PL" }{ "label": "Saint Pierre and Miquelon", "value": "PM" }{ "label": "Pitcairn", "value": "PN" }{ "label": "Puerto Rico", "value": "PR" }{ "label": "Palestine, State of", "value": "PS" }{ "label": "Portugal", "value": "PT" }{ "label": "Palau", "value": "PW" }{ "label": "Paraguay", "value": "PY" }{ "label": "Qatar", "value": "QA" }{ "label": "Réunion", "value": "RE" }{ "label": "Romania", "value": "RO" }{ "label": "Serbia", "value": "RS" }{ "label": "Russian Federation", "value": "RU" }{ "label": "Rwanda", "value": "RW" }{ "label": "Saudi Arabia", "value": "SA" }{ "label": "Solomon Islands", "value": "SB" }{ "label": "Seychelles", "value": "SC" }{ "label": "Sudan", "value": "SD" }{ "label": "Sweden", "value": "SE" }{ "label": "Singapore", "value": "SG" }{ "label": "Saint Helena, Ascension and Tristan da Cunha", "value": "SH" }{ "label": "Slovenia", "value": "SI" }{ "label": "Svalbard and Jan Mayen", "value": "SJ" }{ "label": "Slovakia", "value": "SK" }{ "label": "Sierra Leone", "value": "SL" }{ "label": "San Marino", "value": "SM" }{ "label": "Senegal", "value": "SN" }{ "label": "Somalia", "value": "SO" }{ "label": "Suriname", "value": "SR" }{ "label": "South Sudan", "value": "SS" }{ "label": "Sao Tome and Principe", "value": "ST" }{ "label": "El Salvador", "value": "SV" }{ "label": "Sint Maarten (Dutch part)", "value": "SX" }{ "label": "Syrian Arab Republic", "value": "SY" }{ "label": "Eswatini", "value": "SZ" }{ "label": "Turks and Caicos Islands", "value": "TC" }{ "label": "Chad", "value": "TD" }{ "label": "French Southern Territories", "value": "TF" }{ "label": "Togo", "value": "TG" }{ "label": "Thailand", "value": "TH" }{ "label": "Tajikistan", "value": "TJ" }{ "label": "Tokelau", "value": "TK" }{ "label": "Timor-Leste", "value": "TL" }{ "label": "Turkmenistan", "value": "TM" }{ "label": "Tunisia", "value": "TN" }{ "label": "Tonga", "value": "TO" }{ "label": "Turkey", "value": "TR" }{ "label": "Trinidad and Tobago", "value": "TT" }{ "label": "Tuvalu", "value": "TV" }{ "label": "Taiwan, Province of China", "value": "TW" }{ "label": "Tanzania, United Republic of", "value": "TZ" }{ "label": "Ukraine", "value": "UA" }{ "label": "Uganda", "value": "UG" }{ "label": "United States Minor Outlying Islands", "value": "UM" }{ "label": "United States of America", "value": "US" }{ "label": "Uruguay", "value": "UY" }{ "label": "Uzbekistan", "value": "UZ" }{ "label": "Holy See", "value": "VA" }{ "label": "Saint Vincent and the Grenadines", "value": "VC" }{ "label": "Venezuela (Bolivarian Republic of)", "value": "VE" }{ "label": "Virgin Islands (British)", "value": "VG" }{ "label": "Virgin Islands (U.S.)", "value": "VI" }{ "label": "Viet Nam", "value": "VN" }{ "label": "Vanuatu", "value": "VU" }{ "label": "Wallis and Futuna", "value": "WF" }{ "label": "Samoa", "value": "WS" }{ "label": "Yemen", "value": "YE" }{ "label": "Mayotte", "value": "YT" }{ "label": "South Africa", "value": "ZA" }{ "label": "Zambia", "value": "ZM" }{ "label": "Zimbabwe", "value": "ZW" }`
Result Language	`resultLang`	`string`	Select a value from the drop down menu:`{ "label": "Afar", "value": "aa" }{ "label": "Amharic", "value": "am" }{ "label": "Arabic", "value": "ar" }{ "label": "Bengali", "value": "bn" }{ "label": "Bosnian", "value": "bs" }{ "label": "Bulgarian", "value": "bg" }{ "label": "Burmese", "value": "my" }{ "label": "Chinese", "value": "zh" }{ "label": "Croatian", "value": "hr" }{ "label": "Czech", "value": "cs" }{ "label": "Danish", "value": "da" }{ "label": "Dutch", "value": "nl" }{ "label": "English", "value": "en" }{ "label": "Estonian", "value": "et" }{ "label": "Finnish", "value": "fi" }{ "label": "French", "value": "fr" }{ "label": "German", "value": "de" }{ "label": "Greek", "value": "el" }{ "label": "Hebrew", "value": "he" }{ "label": "Hindi", "value": "hi" }{ "label": "Hungarian", "value": "hu" }{ "label": "Icelandic", "value": "is" }{ "label": "Indonesian", "value": "id" }{ "label": "Italian", "value": "it" }{ "label": "Japanese", "value": "ja" }{ "label": "Korean", "value": "ko" }{ "label": "Lao", "value": "lo" }{ "label": "Latvian", "value": "lv" }{ "label": "Lithuanian", "value": "lt" }{ "label": "Macedonian", "value": "mk" }{ "label": "Malay", "value": "ms" }{ "label": "Maltese", "value": "mt" }{ "label": "Marathi", "value": "mr" }{ "label": "Māori", "value": "mi" }{ "label": "Nepali", "value": "ne" }{ "label": "Norwegian Bokmål", "value": "nb" }{ "label": "Norwegian", "value": "no" }{ "label": "Persian", "value": "fa" }{ "label": "Polish", "value": "pl" }{ "label": "Portuguese", "value": "pt" }{ "label": "Romanian", "value": "ro" }{ "label": "Russian", "value": "ru" }{ "label": "Serbian", "value": "sr" }{ "label": "Sinhalese", "value": "si" }{ "label": "Slovak", "value": "sk" }{ "label": "Slovene", "value": "sl" }{ "label": "Somali", "value": "so" }{ "label": "Spanish", "value": "es" }{ "label": "Swedish", "value": "sv" }{ "label": "Tajik", "value": "tg" }{ "label": "Tamil", "value": "ta" }{ "label": "Telugu", "value": "te" }{ "label": "Thai", "value": "th" }{ "label": "Ukrainian", "value": "uk" }{ "label": "Urdu", "value": "ur" }{ "label": "Vietnamese", "value": "vi" }`