/webscraping-ai | Type: Application | PCID required: Yes
Tools
| Tool | Description |
|---|---|
webscraping_ai_account | Information about your account calls quota |
webscraping_ai_get_fields | Extract structured data fields from a web page |
webscraping_ai_get_html | Page HTML by URL |
webscraping_ai_get_question | Get an answer to a question about a given web page |
webscraping_ai_get_selected | HTML of a selected page area by URL and CSS selector |
webscraping_ai_get_selected_multiple | HTML of multiple page areas by URL and CSS selectors |
webscraping_ai_get_text | Page text by URL |
webscraping_ai_account
Information about your account calls quotaShow inputSchema
Show inputSchema
{
"type": "object",
"properties": {
"PCID": {
"type": "string",
"description": "Pink Connect ID for the authenticated connection"
}
},
"required": [
"PCID"
]
}
webscraping_ai_get_fields
Extract structured data fields from a web page Parameters:| Parameter | Type | Required | Default | Description |
|---|---|---|---|---|
url | string | Yes | — | URL of the target page. |
fields | object | Yes | — | Object describing fields to extract from the page and their descriptions |
headers | object | No | — | HTTP headers to pass to the target page. Can be specified either via a nested query parameter (…&headers[One]=value1&headers=[Another]=value2) or as a JSON encoded object (…&headers={“One”: “value1”, “Another”: “value2”}). |
timeout | integer | No | — | Maximum web page retrieval time in ms. Increase it in case of timeout errors (10000 by default, maximum is 30000). |
js | boolean | No | — | Execute on-page JavaScript using a headless browser (true by default). |
js_timeout | integer | No | — | Maximum JavaScript rendering time in ms. Increase it in case if you see a loading indicator instead of data on the target page. |
wait_for | string | No | — | CSS selector to wait for before returning the page content. Useful for pages with dynamic content loading. Overrides js_timeout. |
proxy | string | No | — | Type of proxy. Use residential if your site restricts traffic from datacenters, or stealth for the most heavily protected sites with advanced anti-bot detection (datacenter by default). Residential and stealth proxy requests are more expensive than datacenter, see the pricing page for details. |
country | string | No | — | Country of the proxy to use (US by default). |
custom_proxy | string | No | — | Your own proxy URL to use instead of our built-in proxy pool in “http://user:password@host:port” format (<a target=“_blank” href=“https://webscraping.ai/proxies/smartproxy”>Smartproxy</a> for example). |
device | string | No | — | Type of device emulation. |
error_on_404 | boolean | No | — | Return error on 404 HTTP status on the target page (false by default). |
error_on_redirect | boolean | No | — | Return error on redirect on the target page (false by default). |
js_script | string | No | — | Custom JavaScript code to execute on the target page. |
Show inputSchema
Show inputSchema
{
"type": "object",
"properties": {
"PCID": {
"type": "string",
"description": "Pink Connect ID for the authenticated connection"
},
"url": {
"type": "string",
"description": "URL of the target page."
},
"fields": {
"type": "object",
"description": "Object describing fields to extract from the page and their descriptions"
},
"headers": {
"type": "object",
"description": "HTTP headers to pass to the target page. Can be specified either via a nested query parameter (...&headers[One]=value1&headers=[Another]=value2) or as a JSON encoded object (...&headers={\"One\": \"value1\", \"Another\": \"value2\"})."
},
"timeout": {
"type": "integer",
"description": "Maximum web page retrieval time in ms. Increase it in case of timeout errors (10000 by default, maximum is 30000)."
},
"js": {
"type": "boolean",
"description": "Execute on-page JavaScript using a headless browser (true by default)."
},
"js_timeout": {
"type": "integer",
"description": "Maximum JavaScript rendering time in ms. Increase it in case if you see a loading indicator instead of data on the target page."
},
"wait_for": {
"type": "string",
"description": "CSS selector to wait for before returning the page content. Useful for pages with dynamic content loading. Overrides js_timeout."
},
"proxy": {
"type": "string",
"description": "Type of proxy. Use `residential` if your site restricts traffic from datacenters, or `stealth` for the most heavily protected sites with advanced anti-bot detection (`datacenter` by default). Residential and stealth proxy requests are more expensive than datacenter, see the pricing page for details.",
"enum": [
"datacenter",
"residential",
"stealth"
]
},
"country": {
"type": "string",
"description": "Country of the proxy to use (US by default).",
"enum": [
"us",
"gb",
"de",
"it",
"fr",
"ca",
"es",
"ru",
"jp",
"kr",
"in",
"hk",
"tr"
]
},
"custom_proxy": {
"type": "string",
"description": "Your own proxy URL to use instead of our built-in proxy pool in \"http://user:password@host:port\" format (<a target=\"_blank\" href=\"https://webscraping.ai/proxies/smartproxy\">Smartproxy</a> for example)."
},
"device": {
"type": "string",
"description": "Type of device emulation.",
"enum": [
"desktop",
"mobile",
"tablet"
]
},
"error_on_404": {
"type": "boolean",
"description": "Return error on 404 HTTP status on the target page (false by default)."
},
"error_on_redirect": {
"type": "boolean",
"description": "Return error on redirect on the target page (false by default)."
},
"js_script": {
"type": "string",
"description": "Custom JavaScript code to execute on the target page."
}
},
"required": [
"PCID",
"url",
"fields"
]
}
webscraping_ai_get_html
Page HTML by URL Parameters:| Parameter | Type | Required | Default | Description |
|---|---|---|---|---|
url | string | Yes | — | URL of the target page. |
headers | object | No | — | HTTP headers to pass to the target page. Can be specified either via a nested query parameter (…&headers[One]=value1&headers=[Another]=value2) or as a JSON encoded object (…&headers={“One”: “value1”, “Another”: “value2”}). |
timeout | integer | No | — | Maximum web page retrieval time in ms. Increase it in case of timeout errors (10000 by default, maximum is 30000). |
js | boolean | No | — | Execute on-page JavaScript using a headless browser (true by default). |
js_timeout | integer | No | — | Maximum JavaScript rendering time in ms. Increase it in case if you see a loading indicator instead of data on the target page. |
wait_for | string | No | — | CSS selector to wait for before returning the page content. Useful for pages with dynamic content loading. Overrides js_timeout. |
proxy | string | No | — | Type of proxy. Use residential if your site restricts traffic from datacenters, or stealth for the most heavily protected sites with advanced anti-bot detection (datacenter by default). Residential and stealth proxy requests are more expensive than datacenter, see the pricing page for details. |
country | string | No | — | Country of the proxy to use (US by default). |
custom_proxy | string | No | — | Your own proxy URL to use instead of our built-in proxy pool in “http://user:password@host:port” format (<a target=“_blank” href=“https://webscraping.ai/proxies/smartproxy”>Smartproxy</a> for example). |
device | string | No | — | Type of device emulation. |
error_on_404 | boolean | No | — | Return error on 404 HTTP status on the target page (false by default). |
error_on_redirect | boolean | No | — | Return error on redirect on the target page (false by default). |
js_script | string | No | — | Custom JavaScript code to execute on the target page. |
return_script_result | boolean | No | — | Return result of the custom JavaScript code (js_script parameter) execution on the target page (false by default, page HTML will be returned). |
format | string | No | — | Format of the response (text by default). “json” will return a JSON object with the response, “text” will return a plain text/HTML response. |
Show inputSchema
Show inputSchema
{
"type": "object",
"properties": {
"PCID": {
"type": "string",
"description": "Pink Connect ID for the authenticated connection"
},
"url": {
"type": "string",
"description": "URL of the target page."
},
"headers": {
"type": "object",
"description": "HTTP headers to pass to the target page. Can be specified either via a nested query parameter (...&headers[One]=value1&headers=[Another]=value2) or as a JSON encoded object (...&headers={\"One\": \"value1\", \"Another\": \"value2\"})."
},
"timeout": {
"type": "integer",
"description": "Maximum web page retrieval time in ms. Increase it in case of timeout errors (10000 by default, maximum is 30000)."
},
"js": {
"type": "boolean",
"description": "Execute on-page JavaScript using a headless browser (true by default)."
},
"js_timeout": {
"type": "integer",
"description": "Maximum JavaScript rendering time in ms. Increase it in case if you see a loading indicator instead of data on the target page."
},
"wait_for": {
"type": "string",
"description": "CSS selector to wait for before returning the page content. Useful for pages with dynamic content loading. Overrides js_timeout."
},
"proxy": {
"type": "string",
"description": "Type of proxy. Use `residential` if your site restricts traffic from datacenters, or `stealth` for the most heavily protected sites with advanced anti-bot detection (`datacenter` by default). Residential and stealth proxy requests are more expensive than datacenter, see the pricing page for details.",
"enum": [
"datacenter",
"residential",
"stealth"
]
},
"country": {
"type": "string",
"description": "Country of the proxy to use (US by default).",
"enum": [
"us",
"gb",
"de",
"it",
"fr",
"ca",
"es",
"ru",
"jp",
"kr",
"in",
"hk",
"tr"
]
},
"custom_proxy": {
"type": "string",
"description": "Your own proxy URL to use instead of our built-in proxy pool in \"http://user:password@host:port\" format (<a target=\"_blank\" href=\"https://webscraping.ai/proxies/smartproxy\">Smartproxy</a> for example)."
},
"device": {
"type": "string",
"description": "Type of device emulation.",
"enum": [
"desktop",
"mobile",
"tablet"
]
},
"error_on_404": {
"type": "boolean",
"description": "Return error on 404 HTTP status on the target page (false by default)."
},
"error_on_redirect": {
"type": "boolean",
"description": "Return error on redirect on the target page (false by default)."
},
"js_script": {
"type": "string",
"description": "Custom JavaScript code to execute on the target page."
},
"return_script_result": {
"type": "boolean",
"description": "Return result of the custom JavaScript code (js_script parameter) execution on the target page (false by default, page HTML will be returned)."
},
"format": {
"type": "string",
"description": "Format of the response (text by default). \"json\" will return a JSON object with the response, \"text\" will return a plain text/HTML response.",
"enum": [
"json",
"text"
]
}
},
"required": [
"PCID",
"url"
]
}
webscraping_ai_get_question
Get an answer to a question about a given web page Parameters:| Parameter | Type | Required | Default | Description |
|---|---|---|---|---|
url | string | Yes | — | URL of the target page. |
question | string | No | — | Question or instructions to ask the LLM model about the target page. |
headers | object | No | — | HTTP headers to pass to the target page. Can be specified either via a nested query parameter (…&headers[One]=value1&headers=[Another]=value2) or as a JSON encoded object (…&headers={“One”: “value1”, “Another”: “value2”}). |
timeout | integer | No | — | Maximum web page retrieval time in ms. Increase it in case of timeout errors (10000 by default, maximum is 30000). |
js | boolean | No | — | Execute on-page JavaScript using a headless browser (true by default). |
js_timeout | integer | No | — | Maximum JavaScript rendering time in ms. Increase it in case if you see a loading indicator instead of data on the target page. |
wait_for | string | No | — | CSS selector to wait for before returning the page content. Useful for pages with dynamic content loading. Overrides js_timeout. |
proxy | string | No | — | Type of proxy. Use residential if your site restricts traffic from datacenters, or stealth for the most heavily protected sites with advanced anti-bot detection (datacenter by default). Residential and stealth proxy requests are more expensive than datacenter, see the pricing page for details. |
country | string | No | — | Country of the proxy to use (US by default). |
custom_proxy | string | No | — | Your own proxy URL to use instead of our built-in proxy pool in “http://user:password@host:port” format (<a target=“_blank” href=“https://webscraping.ai/proxies/smartproxy”>Smartproxy</a> for example). |
device | string | No | — | Type of device emulation. |
error_on_404 | boolean | No | — | Return error on 404 HTTP status on the target page (false by default). |
error_on_redirect | boolean | No | — | Return error on redirect on the target page (false by default). |
js_script | string | No | — | Custom JavaScript code to execute on the target page. |
format | string | No | — | Format of the response (text by default). “json” will return a JSON object with the response, “text” will return a plain text/HTML response. |
Show inputSchema
Show inputSchema
{
"type": "object",
"properties": {
"PCID": {
"type": "string",
"description": "Pink Connect ID for the authenticated connection"
},
"url": {
"type": "string",
"description": "URL of the target page."
},
"question": {
"type": "string",
"description": "Question or instructions to ask the LLM model about the target page."
},
"headers": {
"type": "object",
"description": "HTTP headers to pass to the target page. Can be specified either via a nested query parameter (...&headers[One]=value1&headers=[Another]=value2) or as a JSON encoded object (...&headers={\"One\": \"value1\", \"Another\": \"value2\"})."
},
"timeout": {
"type": "integer",
"description": "Maximum web page retrieval time in ms. Increase it in case of timeout errors (10000 by default, maximum is 30000)."
},
"js": {
"type": "boolean",
"description": "Execute on-page JavaScript using a headless browser (true by default)."
},
"js_timeout": {
"type": "integer",
"description": "Maximum JavaScript rendering time in ms. Increase it in case if you see a loading indicator instead of data on the target page."
},
"wait_for": {
"type": "string",
"description": "CSS selector to wait for before returning the page content. Useful for pages with dynamic content loading. Overrides js_timeout."
},
"proxy": {
"type": "string",
"description": "Type of proxy. Use `residential` if your site restricts traffic from datacenters, or `stealth` for the most heavily protected sites with advanced anti-bot detection (`datacenter` by default). Residential and stealth proxy requests are more expensive than datacenter, see the pricing page for details.",
"enum": [
"datacenter",
"residential",
"stealth"
]
},
"country": {
"type": "string",
"description": "Country of the proxy to use (US by default).",
"enum": [
"us",
"gb",
"de",
"it",
"fr",
"ca",
"es",
"ru",
"jp",
"kr",
"in",
"hk",
"tr"
]
},
"custom_proxy": {
"type": "string",
"description": "Your own proxy URL to use instead of our built-in proxy pool in \"http://user:password@host:port\" format (<a target=\"_blank\" href=\"https://webscraping.ai/proxies/smartproxy\">Smartproxy</a> for example)."
},
"device": {
"type": "string",
"description": "Type of device emulation.",
"enum": [
"desktop",
"mobile",
"tablet"
]
},
"error_on_404": {
"type": "boolean",
"description": "Return error on 404 HTTP status on the target page (false by default)."
},
"error_on_redirect": {
"type": "boolean",
"description": "Return error on redirect on the target page (false by default)."
},
"js_script": {
"type": "string",
"description": "Custom JavaScript code to execute on the target page."
},
"format": {
"type": "string",
"description": "Format of the response (text by default). \"json\" will return a JSON object with the response, \"text\" will return a plain text/HTML response.",
"enum": [
"json",
"text"
]
}
},
"required": [
"PCID",
"url"
]
}
webscraping_ai_get_selected
HTML of a selected page area by URL and CSS selector Parameters:| Parameter | Type | Required | Default | Description |
|---|---|---|---|---|
selector | string | No | — | CSS selector (null by default, returns whole page HTML) |
url | string | Yes | — | URL of the target page. |
headers | object | No | — | HTTP headers to pass to the target page. Can be specified either via a nested query parameter (…&headers[One]=value1&headers=[Another]=value2) or as a JSON encoded object (…&headers={“One”: “value1”, “Another”: “value2”}). |
timeout | integer | No | — | Maximum web page retrieval time in ms. Increase it in case of timeout errors (10000 by default, maximum is 30000). |
js | boolean | No | — | Execute on-page JavaScript using a headless browser (true by default). |
js_timeout | integer | No | — | Maximum JavaScript rendering time in ms. Increase it in case if you see a loading indicator instead of data on the target page. |
wait_for | string | No | — | CSS selector to wait for before returning the page content. Useful for pages with dynamic content loading. Overrides js_timeout. |
proxy | string | No | — | Type of proxy. Use residential if your site restricts traffic from datacenters, or stealth for the most heavily protected sites with advanced anti-bot detection (datacenter by default). Residential and stealth proxy requests are more expensive than datacenter, see the pricing page for details. |
country | string | No | — | Country of the proxy to use (US by default). |
custom_proxy | string | No | — | Your own proxy URL to use instead of our built-in proxy pool in “http://user:password@host:port” format (<a target=“_blank” href=“https://webscraping.ai/proxies/smartproxy”>Smartproxy</a> for example). |
device | string | No | — | Type of device emulation. |
error_on_404 | boolean | No | — | Return error on 404 HTTP status on the target page (false by default). |
error_on_redirect | boolean | No | — | Return error on redirect on the target page (false by default). |
js_script | string | No | — | Custom JavaScript code to execute on the target page. |
format | string | No | — | Format of the response (text by default). “json” will return a JSON object with the response, “text” will return a plain text/HTML response. |
Show inputSchema
Show inputSchema
{
"type": "object",
"properties": {
"PCID": {
"type": "string",
"description": "Pink Connect ID for the authenticated connection"
},
"selector": {
"type": "string",
"description": "CSS selector (null by default, returns whole page HTML)"
},
"url": {
"type": "string",
"description": "URL of the target page."
},
"headers": {
"type": "object",
"description": "HTTP headers to pass to the target page. Can be specified either via a nested query parameter (...&headers[One]=value1&headers=[Another]=value2) or as a JSON encoded object (...&headers={\"One\": \"value1\", \"Another\": \"value2\"})."
},
"timeout": {
"type": "integer",
"description": "Maximum web page retrieval time in ms. Increase it in case of timeout errors (10000 by default, maximum is 30000)."
},
"js": {
"type": "boolean",
"description": "Execute on-page JavaScript using a headless browser (true by default)."
},
"js_timeout": {
"type": "integer",
"description": "Maximum JavaScript rendering time in ms. Increase it in case if you see a loading indicator instead of data on the target page."
},
"wait_for": {
"type": "string",
"description": "CSS selector to wait for before returning the page content. Useful for pages with dynamic content loading. Overrides js_timeout."
},
"proxy": {
"type": "string",
"description": "Type of proxy. Use `residential` if your site restricts traffic from datacenters, or `stealth` for the most heavily protected sites with advanced anti-bot detection (`datacenter` by default). Residential and stealth proxy requests are more expensive than datacenter, see the pricing page for details.",
"enum": [
"datacenter",
"residential",
"stealth"
]
},
"country": {
"type": "string",
"description": "Country of the proxy to use (US by default).",
"enum": [
"us",
"gb",
"de",
"it",
"fr",
"ca",
"es",
"ru",
"jp",
"kr",
"in",
"hk",
"tr"
]
},
"custom_proxy": {
"type": "string",
"description": "Your own proxy URL to use instead of our built-in proxy pool in \"http://user:password@host:port\" format (<a target=\"_blank\" href=\"https://webscraping.ai/proxies/smartproxy\">Smartproxy</a> for example)."
},
"device": {
"type": "string",
"description": "Type of device emulation.",
"enum": [
"desktop",
"mobile",
"tablet"
]
},
"error_on_404": {
"type": "boolean",
"description": "Return error on 404 HTTP status on the target page (false by default)."
},
"error_on_redirect": {
"type": "boolean",
"description": "Return error on redirect on the target page (false by default)."
},
"js_script": {
"type": "string",
"description": "Custom JavaScript code to execute on the target page."
},
"format": {
"type": "string",
"description": "Format of the response (text by default). \"json\" will return a JSON object with the response, \"text\" will return a plain text/HTML response.",
"enum": [
"json",
"text"
]
}
},
"required": [
"PCID",
"url"
]
}
webscraping_ai_get_selected_multiple
HTML of multiple page areas by URL and CSS selectors Parameters:| Parameter | Type | Required | Default | Description |
|---|---|---|---|---|
selectors | string[] | No | — | Multiple CSS selectors (null by default, returns whole page HTML) |
url | string | Yes | — | URL of the target page. |
headers | object | No | — | HTTP headers to pass to the target page. Can be specified either via a nested query parameter (…&headers[One]=value1&headers=[Another]=value2) or as a JSON encoded object (…&headers={“One”: “value1”, “Another”: “value2”}). |
timeout | integer | No | — | Maximum web page retrieval time in ms. Increase it in case of timeout errors (10000 by default, maximum is 30000). |
js | boolean | No | — | Execute on-page JavaScript using a headless browser (true by default). |
js_timeout | integer | No | — | Maximum JavaScript rendering time in ms. Increase it in case if you see a loading indicator instead of data on the target page. |
wait_for | string | No | — | CSS selector to wait for before returning the page content. Useful for pages with dynamic content loading. Overrides js_timeout. |
proxy | string | No | — | Type of proxy. Use residential if your site restricts traffic from datacenters, or stealth for the most heavily protected sites with advanced anti-bot detection (datacenter by default). Residential and stealth proxy requests are more expensive than datacenter, see the pricing page for details. |
country | string | No | — | Country of the proxy to use (US by default). |
custom_proxy | string | No | — | Your own proxy URL to use instead of our built-in proxy pool in “http://user:password@host:port” format (<a target=“_blank” href=“https://webscraping.ai/proxies/smartproxy”>Smartproxy</a> for example). |
device | string | No | — | Type of device emulation. |
error_on_404 | boolean | No | — | Return error on 404 HTTP status on the target page (false by default). |
error_on_redirect | boolean | No | — | Return error on redirect on the target page (false by default). |
js_script | string | No | — | Custom JavaScript code to execute on the target page. |
Show inputSchema
Show inputSchema
{
"type": "object",
"properties": {
"PCID": {
"type": "string",
"description": "Pink Connect ID for the authenticated connection"
},
"selectors": {
"type": "array",
"items": {
"type": "string"
},
"description": "Multiple CSS selectors (null by default, returns whole page HTML)"
},
"url": {
"type": "string",
"description": "URL of the target page."
},
"headers": {
"type": "object",
"description": "HTTP headers to pass to the target page. Can be specified either via a nested query parameter (...&headers[One]=value1&headers=[Another]=value2) or as a JSON encoded object (...&headers={\"One\": \"value1\", \"Another\": \"value2\"})."
},
"timeout": {
"type": "integer",
"description": "Maximum web page retrieval time in ms. Increase it in case of timeout errors (10000 by default, maximum is 30000)."
},
"js": {
"type": "boolean",
"description": "Execute on-page JavaScript using a headless browser (true by default)."
},
"js_timeout": {
"type": "integer",
"description": "Maximum JavaScript rendering time in ms. Increase it in case if you see a loading indicator instead of data on the target page."
},
"wait_for": {
"type": "string",
"description": "CSS selector to wait for before returning the page content. Useful for pages with dynamic content loading. Overrides js_timeout."
},
"proxy": {
"type": "string",
"description": "Type of proxy. Use `residential` if your site restricts traffic from datacenters, or `stealth` for the most heavily protected sites with advanced anti-bot detection (`datacenter` by default). Residential and stealth proxy requests are more expensive than datacenter, see the pricing page for details.",
"enum": [
"datacenter",
"residential",
"stealth"
]
},
"country": {
"type": "string",
"description": "Country of the proxy to use (US by default).",
"enum": [
"us",
"gb",
"de",
"it",
"fr",
"ca",
"es",
"ru",
"jp",
"kr",
"in",
"hk",
"tr"
]
},
"custom_proxy": {
"type": "string",
"description": "Your own proxy URL to use instead of our built-in proxy pool in \"http://user:password@host:port\" format (<a target=\"_blank\" href=\"https://webscraping.ai/proxies/smartproxy\">Smartproxy</a> for example)."
},
"device": {
"type": "string",
"description": "Type of device emulation.",
"enum": [
"desktop",
"mobile",
"tablet"
]
},
"error_on_404": {
"type": "boolean",
"description": "Return error on 404 HTTP status on the target page (false by default)."
},
"error_on_redirect": {
"type": "boolean",
"description": "Return error on redirect on the target page (false by default)."
},
"js_script": {
"type": "string",
"description": "Custom JavaScript code to execute on the target page."
}
},
"required": [
"PCID",
"url"
]
}
webscraping_ai_get_text
Page text by URL Parameters:| Parameter | Type | Required | Default | Description |
|---|---|---|---|---|
text_format | string | No | — | Format of the text response (plain by default). “plain” will return only the page body text. “json” and “xml” will return a json/xml with “title”, “description” and “content” keys. |
return_links | boolean | No | — | [Works only with text_format=json] Return links from the page body text (false by default). Useful for building web crawlers. |
url | string | Yes | — | URL of the target page. |
headers | object | No | — | HTTP headers to pass to the target page. Can be specified either via a nested query parameter (…&headers[One]=value1&headers=[Another]=value2) or as a JSON encoded object (…&headers={“One”: “value1”, “Another”: “value2”}). |
timeout | integer | No | — | Maximum web page retrieval time in ms. Increase it in case of timeout errors (10000 by default, maximum is 30000). |
js | boolean | No | — | Execute on-page JavaScript using a headless browser (true by default). |
js_timeout | integer | No | — | Maximum JavaScript rendering time in ms. Increase it in case if you see a loading indicator instead of data on the target page. |
wait_for | string | No | — | CSS selector to wait for before returning the page content. Useful for pages with dynamic content loading. Overrides js_timeout. |
proxy | string | No | — | Type of proxy. Use residential if your site restricts traffic from datacenters, or stealth for the most heavily protected sites with advanced anti-bot detection (datacenter by default). Residential and stealth proxy requests are more expensive than datacenter, see the pricing page for details. |
country | string | No | — | Country of the proxy to use (US by default). |
custom_proxy | string | No | — | Your own proxy URL to use instead of our built-in proxy pool in “http://user:password@host:port” format (<a target=“_blank” href=“https://webscraping.ai/proxies/smartproxy”>Smartproxy</a> for example). |
device | string | No | — | Type of device emulation. |
error_on_404 | boolean | No | — | Return error on 404 HTTP status on the target page (false by default). |
error_on_redirect | boolean | No | — | Return error on redirect on the target page (false by default). |
js_script | string | No | — | Custom JavaScript code to execute on the target page. |
Show inputSchema
Show inputSchema
{
"type": "object",
"properties": {
"PCID": {
"type": "string",
"description": "Pink Connect ID for the authenticated connection"
},
"text_format": {
"type": "string",
"description": "Format of the text response (plain by default). \"plain\" will return only the page body text. \"json\" and \"xml\" will return a json/xml with \"title\", \"description\" and \"content\" keys.",
"enum": [
"plain",
"xml",
"json"
]
},
"return_links": {
"type": "boolean",
"description": "[Works only with text_format=json] Return links from the page body text (false by default). Useful for building web crawlers."
},
"url": {
"type": "string",
"description": "URL of the target page."
},
"headers": {
"type": "object",
"description": "HTTP headers to pass to the target page. Can be specified either via a nested query parameter (...&headers[One]=value1&headers=[Another]=value2) or as a JSON encoded object (...&headers={\"One\": \"value1\", \"Another\": \"value2\"})."
},
"timeout": {
"type": "integer",
"description": "Maximum web page retrieval time in ms. Increase it in case of timeout errors (10000 by default, maximum is 30000)."
},
"js": {
"type": "boolean",
"description": "Execute on-page JavaScript using a headless browser (true by default)."
},
"js_timeout": {
"type": "integer",
"description": "Maximum JavaScript rendering time in ms. Increase it in case if you see a loading indicator instead of data on the target page."
},
"wait_for": {
"type": "string",
"description": "CSS selector to wait for before returning the page content. Useful for pages with dynamic content loading. Overrides js_timeout."
},
"proxy": {
"type": "string",
"description": "Type of proxy. Use `residential` if your site restricts traffic from datacenters, or `stealth` for the most heavily protected sites with advanced anti-bot detection (`datacenter` by default). Residential and stealth proxy requests are more expensive than datacenter, see the pricing page for details.",
"enum": [
"datacenter",
"residential",
"stealth"
]
},
"country": {
"type": "string",
"description": "Country of the proxy to use (US by default).",
"enum": [
"us",
"gb",
"de",
"it",
"fr",
"ca",
"es",
"ru",
"jp",
"kr",
"in",
"hk",
"tr"
]
},
"custom_proxy": {
"type": "string",
"description": "Your own proxy URL to use instead of our built-in proxy pool in \"http://user:password@host:port\" format (<a target=\"_blank\" href=\"https://webscraping.ai/proxies/smartproxy\">Smartproxy</a> for example)."
},
"device": {
"type": "string",
"description": "Type of device emulation.",
"enum": [
"desktop",
"mobile",
"tablet"
]
},
"error_on_404": {
"type": "boolean",
"description": "Return error on 404 HTTP status on the target page (false by default)."
},
"error_on_redirect": {
"type": "boolean",
"description": "Return error on redirect on the target page (false by default)."
},
"js_script": {
"type": "string",
"description": "Custom JavaScript code to execute on the target page."
}
},
"required": [
"PCID",
"url"
]
}