> ## Documentation Index
> Fetch the complete documentation index at: https://docs.pinkfish.ai/llms.txt
> Use this file to discover all available pages before exploring further.
# webscraping-ai
> Web Scraping API
**Server path:** `/webscraping-ai` | **Type:** Application | **PCID required:** Yes
## Tools
| Tool | Description |
| ------------------------------------------------------------------------------- | ---------------------------------------------------- |
| [`webscraping_ai_account`](#webscraping_ai_account) | Information about your account calls quota |
| [`webscraping_ai_get_fields`](#webscraping_ai_get_fields) | Extract structured data fields from a web page |
| [`webscraping_ai_get_html`](#webscraping_ai_get_html) | Page HTML by URL |
| [`webscraping_ai_get_question`](#webscraping_ai_get_question) | Get an answer to a question about a given web page |
| [`webscraping_ai_get_selected`](#webscraping_ai_get_selected) | HTML of a selected page area by URL and CSS selector |
| [`webscraping_ai_get_selected_multiple`](#webscraping_ai_get_selected_multiple) | HTML of multiple page areas by URL and CSS selectors |
| [`webscraping_ai_get_text`](#webscraping_ai_get_text) | Page text by URL |
***
## webscraping\_ai\_account
Information about your account calls quota
```json theme={null}
{
"type": "object",
"properties": {
"PCID": {
"type": "string",
"description": "Pink Connect ID for the authenticated connection"
}
},
"required": [
"PCID"
]
}
```
***
## webscraping\_ai\_get\_fields
Extract structured data fields from a web page
**Parameters:**
| Parameter | Type | Required | Default | Description |
| ------------------- | ------- | -------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `url` | string | Yes | — | URL of the target page. |
| `fields` | object | Yes | — | Object describing fields to extract from the page and their descriptions |
| `headers` | object | No | — | HTTP headers to pass to the target page. Can be specified either via a nested query parameter (...\&headers\[One]=value1\&headers=\[Another]=value2) or as a JSON encoded object (...\&headers=\{"One": "value1", "Another": "value2"}). |
| `timeout` | integer | No | — | Maximum web page retrieval time in ms. Increase it in case of timeout errors (10000 by default, maximum is 30000). |
| `js` | boolean | No | — | Execute on-page JavaScript using a headless browser (true by default). |
| `js_timeout` | integer | No | — | Maximum JavaScript rendering time in ms. Increase it in case if you see a loading indicator instead of data on the target page. |
| `wait_for` | string | No | — | CSS selector to wait for before returning the page content. Useful for pages with dynamic content loading. Overrides js\_timeout. |
| `proxy` | string | No | — | Type of proxy. Use `residential` if your site restricts traffic from datacenters, or `stealth` for the most heavily protected sites with advanced anti-bot detection (`datacenter` by default). Residential and stealth proxy requests are more expensive than datacenter, see the pricing page for details. |
| `country` | string | No | — | Country of the proxy to use (US by default). |
| `custom_proxy` | string | No | — | Your own proxy URL to use instead of our built-in proxy pool in "`http://user:password@host:port`" format (\ for example). |
| `device` | string | No | — | Type of device emulation. |
| `error_on_404` | boolean | No | — | Return error on 404 HTTP status on the target page (false by default). |
| `error_on_redirect` | boolean | No | — | Return error on redirect on the target page (false by default). |
| `js_script` | string | No | — | Custom JavaScript code to execute on the target page. |
```json theme={null}
{
"type": "object",
"properties": {
"PCID": {
"type": "string",
"description": "Pink Connect ID for the authenticated connection"
},
"url": {
"type": "string",
"description": "URL of the target page."
},
"fields": {
"type": "object",
"description": "Object describing fields to extract from the page and their descriptions"
},
"headers": {
"type": "object",
"description": "HTTP headers to pass to the target page. Can be specified either via a nested query parameter (...&headers[One]=value1&headers=[Another]=value2) or as a JSON encoded object (...&headers={\"One\": \"value1\", \"Another\": \"value2\"})."
},
"timeout": {
"type": "integer",
"description": "Maximum web page retrieval time in ms. Increase it in case of timeout errors (10000 by default, maximum is 30000)."
},
"js": {
"type": "boolean",
"description": "Execute on-page JavaScript using a headless browser (true by default)."
},
"js_timeout": {
"type": "integer",
"description": "Maximum JavaScript rendering time in ms. Increase it in case if you see a loading indicator instead of data on the target page."
},
"wait_for": {
"type": "string",
"description": "CSS selector to wait for before returning the page content. Useful for pages with dynamic content loading. Overrides js_timeout."
},
"proxy": {
"type": "string",
"description": "Type of proxy. Use `residential` if your site restricts traffic from datacenters, or `stealth` for the most heavily protected sites with advanced anti-bot detection (`datacenter` by default). Residential and stealth proxy requests are more expensive than datacenter, see the pricing page for details.",
"enum": [
"datacenter",
"residential",
"stealth"
]
},
"country": {
"type": "string",
"description": "Country of the proxy to use (US by default).",
"enum": [
"us",
"gb",
"de",
"it",
"fr",
"ca",
"es",
"ru",
"jp",
"kr",
"in",
"hk",
"tr"
]
},
"custom_proxy": {
"type": "string",
"description": "Your own proxy URL to use instead of our built-in proxy pool in \"http://user:password@host:port\" format (Smartproxy for example)."
},
"device": {
"type": "string",
"description": "Type of device emulation.",
"enum": [
"desktop",
"mobile",
"tablet"
]
},
"error_on_404": {
"type": "boolean",
"description": "Return error on 404 HTTP status on the target page (false by default)."
},
"error_on_redirect": {
"type": "boolean",
"description": "Return error on redirect on the target page (false by default)."
},
"js_script": {
"type": "string",
"description": "Custom JavaScript code to execute on the target page."
}
},
"required": [
"PCID",
"url",
"fields"
]
}
```
***
## webscraping\_ai\_get\_html
Page HTML by URL
**Parameters:**
| Parameter | Type | Required | Default | Description |
| ---------------------- | ------- | -------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `url` | string | Yes | — | URL of the target page. |
| `headers` | object | No | — | HTTP headers to pass to the target page. Can be specified either via a nested query parameter (...\&headers\[One]=value1\&headers=\[Another]=value2) or as a JSON encoded object (...\&headers=\{"One": "value1", "Another": "value2"}). |
| `timeout` | integer | No | — | Maximum web page retrieval time in ms. Increase it in case of timeout errors (10000 by default, maximum is 30000). |
| `js` | boolean | No | — | Execute on-page JavaScript using a headless browser (true by default). |
| `js_timeout` | integer | No | — | Maximum JavaScript rendering time in ms. Increase it in case if you see a loading indicator instead of data on the target page. |
| `wait_for` | string | No | — | CSS selector to wait for before returning the page content. Useful for pages with dynamic content loading. Overrides js\_timeout. |
| `proxy` | string | No | — | Type of proxy. Use `residential` if your site restricts traffic from datacenters, or `stealth` for the most heavily protected sites with advanced anti-bot detection (`datacenter` by default). Residential and stealth proxy requests are more expensive than datacenter, see the pricing page for details. |
| `country` | string | No | — | Country of the proxy to use (US by default). |
| `custom_proxy` | string | No | — | Your own proxy URL to use instead of our built-in proxy pool in "`http://user:password@host:port`" format (\ for example). |
| `device` | string | No | — | Type of device emulation. |
| `error_on_404` | boolean | No | — | Return error on 404 HTTP status on the target page (false by default). |
| `error_on_redirect` | boolean | No | — | Return error on redirect on the target page (false by default). |
| `js_script` | string | No | — | Custom JavaScript code to execute on the target page. |
| `return_script_result` | boolean | No | — | Return result of the custom JavaScript code (js\_script parameter) execution on the target page (false by default, page HTML will be returned). |
| `format` | string | No | — | Format of the response (text by default). "json" will return a JSON object with the response, "text" will return a plain text/HTML response. |
```json theme={null}
{
"type": "object",
"properties": {
"PCID": {
"type": "string",
"description": "Pink Connect ID for the authenticated connection"
},
"url": {
"type": "string",
"description": "URL of the target page."
},
"headers": {
"type": "object",
"description": "HTTP headers to pass to the target page. Can be specified either via a nested query parameter (...&headers[One]=value1&headers=[Another]=value2) or as a JSON encoded object (...&headers={\"One\": \"value1\", \"Another\": \"value2\"})."
},
"timeout": {
"type": "integer",
"description": "Maximum web page retrieval time in ms. Increase it in case of timeout errors (10000 by default, maximum is 30000)."
},
"js": {
"type": "boolean",
"description": "Execute on-page JavaScript using a headless browser (true by default)."
},
"js_timeout": {
"type": "integer",
"description": "Maximum JavaScript rendering time in ms. Increase it in case if you see a loading indicator instead of data on the target page."
},
"wait_for": {
"type": "string",
"description": "CSS selector to wait for before returning the page content. Useful for pages with dynamic content loading. Overrides js_timeout."
},
"proxy": {
"type": "string",
"description": "Type of proxy. Use `residential` if your site restricts traffic from datacenters, or `stealth` for the most heavily protected sites with advanced anti-bot detection (`datacenter` by default). Residential and stealth proxy requests are more expensive than datacenter, see the pricing page for details.",
"enum": [
"datacenter",
"residential",
"stealth"
]
},
"country": {
"type": "string",
"description": "Country of the proxy to use (US by default).",
"enum": [
"us",
"gb",
"de",
"it",
"fr",
"ca",
"es",
"ru",
"jp",
"kr",
"in",
"hk",
"tr"
]
},
"custom_proxy": {
"type": "string",
"description": "Your own proxy URL to use instead of our built-in proxy pool in \"http://user:password@host:port\" format (Smartproxy for example)."
},
"device": {
"type": "string",
"description": "Type of device emulation.",
"enum": [
"desktop",
"mobile",
"tablet"
]
},
"error_on_404": {
"type": "boolean",
"description": "Return error on 404 HTTP status on the target page (false by default)."
},
"error_on_redirect": {
"type": "boolean",
"description": "Return error on redirect on the target page (false by default)."
},
"js_script": {
"type": "string",
"description": "Custom JavaScript code to execute on the target page."
},
"return_script_result": {
"type": "boolean",
"description": "Return result of the custom JavaScript code (js_script parameter) execution on the target page (false by default, page HTML will be returned)."
},
"format": {
"type": "string",
"description": "Format of the response (text by default). \"json\" will return a JSON object with the response, \"text\" will return a plain text/HTML response.",
"enum": [
"json",
"text"
]
}
},
"required": [
"PCID",
"url"
]
}
```
***
## webscraping\_ai\_get\_question
Get an answer to a question about a given web page
**Parameters:**
| Parameter | Type | Required | Default | Description |
| ------------------- | ------- | -------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `url` | string | Yes | — | URL of the target page. |
| `question` | string | No | — | Question or instructions to ask the LLM model about the target page. |
| `headers` | object | No | — | HTTP headers to pass to the target page. Can be specified either via a nested query parameter (...\&headers\[One]=value1\&headers=\[Another]=value2) or as a JSON encoded object (...\&headers=\{"One": "value1", "Another": "value2"}). |
| `timeout` | integer | No | — | Maximum web page retrieval time in ms. Increase it in case of timeout errors (10000 by default, maximum is 30000). |
| `js` | boolean | No | — | Execute on-page JavaScript using a headless browser (true by default). |
| `js_timeout` | integer | No | — | Maximum JavaScript rendering time in ms. Increase it in case if you see a loading indicator instead of data on the target page. |
| `wait_for` | string | No | — | CSS selector to wait for before returning the page content. Useful for pages with dynamic content loading. Overrides js\_timeout. |
| `proxy` | string | No | — | Type of proxy. Use `residential` if your site restricts traffic from datacenters, or `stealth` for the most heavily protected sites with advanced anti-bot detection (`datacenter` by default). Residential and stealth proxy requests are more expensive than datacenter, see the pricing page for details. |
| `country` | string | No | — | Country of the proxy to use (US by default). |
| `custom_proxy` | string | No | — | Your own proxy URL to use instead of our built-in proxy pool in "`http://user:password@host:port`" format (\ for example). |
| `device` | string | No | — | Type of device emulation. |
| `error_on_404` | boolean | No | — | Return error on 404 HTTP status on the target page (false by default). |
| `error_on_redirect` | boolean | No | — | Return error on redirect on the target page (false by default). |
| `js_script` | string | No | — | Custom JavaScript code to execute on the target page. |
| `format` | string | No | — | Format of the response (text by default). "json" will return a JSON object with the response, "text" will return a plain text/HTML response. |
```json theme={null}
{
"type": "object",
"properties": {
"PCID": {
"type": "string",
"description": "Pink Connect ID for the authenticated connection"
},
"url": {
"type": "string",
"description": "URL of the target page."
},
"question": {
"type": "string",
"description": "Question or instructions to ask the LLM model about the target page."
},
"headers": {
"type": "object",
"description": "HTTP headers to pass to the target page. Can be specified either via a nested query parameter (...&headers[One]=value1&headers=[Another]=value2) or as a JSON encoded object (...&headers={\"One\": \"value1\", \"Another\": \"value2\"})."
},
"timeout": {
"type": "integer",
"description": "Maximum web page retrieval time in ms. Increase it in case of timeout errors (10000 by default, maximum is 30000)."
},
"js": {
"type": "boolean",
"description": "Execute on-page JavaScript using a headless browser (true by default)."
},
"js_timeout": {
"type": "integer",
"description": "Maximum JavaScript rendering time in ms. Increase it in case if you see a loading indicator instead of data on the target page."
},
"wait_for": {
"type": "string",
"description": "CSS selector to wait for before returning the page content. Useful for pages with dynamic content loading. Overrides js_timeout."
},
"proxy": {
"type": "string",
"description": "Type of proxy. Use `residential` if your site restricts traffic from datacenters, or `stealth` for the most heavily protected sites with advanced anti-bot detection (`datacenter` by default). Residential and stealth proxy requests are more expensive than datacenter, see the pricing page for details.",
"enum": [
"datacenter",
"residential",
"stealth"
]
},
"country": {
"type": "string",
"description": "Country of the proxy to use (US by default).",
"enum": [
"us",
"gb",
"de",
"it",
"fr",
"ca",
"es",
"ru",
"jp",
"kr",
"in",
"hk",
"tr"
]
},
"custom_proxy": {
"type": "string",
"description": "Your own proxy URL to use instead of our built-in proxy pool in \"http://user:password@host:port\" format (Smartproxy for example)."
},
"device": {
"type": "string",
"description": "Type of device emulation.",
"enum": [
"desktop",
"mobile",
"tablet"
]
},
"error_on_404": {
"type": "boolean",
"description": "Return error on 404 HTTP status on the target page (false by default)."
},
"error_on_redirect": {
"type": "boolean",
"description": "Return error on redirect on the target page (false by default)."
},
"js_script": {
"type": "string",
"description": "Custom JavaScript code to execute on the target page."
},
"format": {
"type": "string",
"description": "Format of the response (text by default). \"json\" will return a JSON object with the response, \"text\" will return a plain text/HTML response.",
"enum": [
"json",
"text"
]
}
},
"required": [
"PCID",
"url"
]
}
```
***
## webscraping\_ai\_get\_selected
HTML of a selected page area by URL and CSS selector
**Parameters:**
| Parameter | Type | Required | Default | Description |
| ------------------- | ------- | -------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `selector` | string | No | — | CSS selector (null by default, returns whole page HTML) |
| `url` | string | Yes | — | URL of the target page. |
| `headers` | object | No | — | HTTP headers to pass to the target page. Can be specified either via a nested query parameter (...\&headers\[One]=value1\&headers=\[Another]=value2) or as a JSON encoded object (...\&headers=\{"One": "value1", "Another": "value2"}). |
| `timeout` | integer | No | — | Maximum web page retrieval time in ms. Increase it in case of timeout errors (10000 by default, maximum is 30000). |
| `js` | boolean | No | — | Execute on-page JavaScript using a headless browser (true by default). |
| `js_timeout` | integer | No | — | Maximum JavaScript rendering time in ms. Increase it in case if you see a loading indicator instead of data on the target page. |
| `wait_for` | string | No | — | CSS selector to wait for before returning the page content. Useful for pages with dynamic content loading. Overrides js\_timeout. |
| `proxy` | string | No | — | Type of proxy. Use `residential` if your site restricts traffic from datacenters, or `stealth` for the most heavily protected sites with advanced anti-bot detection (`datacenter` by default). Residential and stealth proxy requests are more expensive than datacenter, see the pricing page for details. |
| `country` | string | No | — | Country of the proxy to use (US by default). |
| `custom_proxy` | string | No | — | Your own proxy URL to use instead of our built-in proxy pool in "`http://user:password@host:port`" format (\ for example). |
| `device` | string | No | — | Type of device emulation. |
| `error_on_404` | boolean | No | — | Return error on 404 HTTP status on the target page (false by default). |
| `error_on_redirect` | boolean | No | — | Return error on redirect on the target page (false by default). |
| `js_script` | string | No | — | Custom JavaScript code to execute on the target page. |
| `format` | string | No | — | Format of the response (text by default). "json" will return a JSON object with the response, "text" will return a plain text/HTML response. |
```json theme={null}
{
"type": "object",
"properties": {
"PCID": {
"type": "string",
"description": "Pink Connect ID for the authenticated connection"
},
"selector": {
"type": "string",
"description": "CSS selector (null by default, returns whole page HTML)"
},
"url": {
"type": "string",
"description": "URL of the target page."
},
"headers": {
"type": "object",
"description": "HTTP headers to pass to the target page. Can be specified either via a nested query parameter (...&headers[One]=value1&headers=[Another]=value2) or as a JSON encoded object (...&headers={\"One\": \"value1\", \"Another\": \"value2\"})."
},
"timeout": {
"type": "integer",
"description": "Maximum web page retrieval time in ms. Increase it in case of timeout errors (10000 by default, maximum is 30000)."
},
"js": {
"type": "boolean",
"description": "Execute on-page JavaScript using a headless browser (true by default)."
},
"js_timeout": {
"type": "integer",
"description": "Maximum JavaScript rendering time in ms. Increase it in case if you see a loading indicator instead of data on the target page."
},
"wait_for": {
"type": "string",
"description": "CSS selector to wait for before returning the page content. Useful for pages with dynamic content loading. Overrides js_timeout."
},
"proxy": {
"type": "string",
"description": "Type of proxy. Use `residential` if your site restricts traffic from datacenters, or `stealth` for the most heavily protected sites with advanced anti-bot detection (`datacenter` by default). Residential and stealth proxy requests are more expensive than datacenter, see the pricing page for details.",
"enum": [
"datacenter",
"residential",
"stealth"
]
},
"country": {
"type": "string",
"description": "Country of the proxy to use (US by default).",
"enum": [
"us",
"gb",
"de",
"it",
"fr",
"ca",
"es",
"ru",
"jp",
"kr",
"in",
"hk",
"tr"
]
},
"custom_proxy": {
"type": "string",
"description": "Your own proxy URL to use instead of our built-in proxy pool in \"http://user:password@host:port\" format (Smartproxy for example)."
},
"device": {
"type": "string",
"description": "Type of device emulation.",
"enum": [
"desktop",
"mobile",
"tablet"
]
},
"error_on_404": {
"type": "boolean",
"description": "Return error on 404 HTTP status on the target page (false by default)."
},
"error_on_redirect": {
"type": "boolean",
"description": "Return error on redirect on the target page (false by default)."
},
"js_script": {
"type": "string",
"description": "Custom JavaScript code to execute on the target page."
},
"format": {
"type": "string",
"description": "Format of the response (text by default). \"json\" will return a JSON object with the response, \"text\" will return a plain text/HTML response.",
"enum": [
"json",
"text"
]
}
},
"required": [
"PCID",
"url"
]
}
```
***
## webscraping\_ai\_get\_selected\_multiple
HTML of multiple page areas by URL and CSS selectors
**Parameters:**
| Parameter | Type | Required | Default | Description |
| ------------------- | --------- | -------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `selectors` | string\[] | No | — | Multiple CSS selectors (null by default, returns whole page HTML) |
| `url` | string | Yes | — | URL of the target page. |
| `headers` | object | No | — | HTTP headers to pass to the target page. Can be specified either via a nested query parameter (...\&headers\[One]=value1\&headers=\[Another]=value2) or as a JSON encoded object (...\&headers=\{"One": "value1", "Another": "value2"}). |
| `timeout` | integer | No | — | Maximum web page retrieval time in ms. Increase it in case of timeout errors (10000 by default, maximum is 30000). |
| `js` | boolean | No | — | Execute on-page JavaScript using a headless browser (true by default). |
| `js_timeout` | integer | No | — | Maximum JavaScript rendering time in ms. Increase it in case if you see a loading indicator instead of data on the target page. |
| `wait_for` | string | No | — | CSS selector to wait for before returning the page content. Useful for pages with dynamic content loading. Overrides js\_timeout. |
| `proxy` | string | No | — | Type of proxy. Use `residential` if your site restricts traffic from datacenters, or `stealth` for the most heavily protected sites with advanced anti-bot detection (`datacenter` by default). Residential and stealth proxy requests are more expensive than datacenter, see the pricing page for details. |
| `country` | string | No | — | Country of the proxy to use (US by default). |
| `custom_proxy` | string | No | — | Your own proxy URL to use instead of our built-in proxy pool in "`http://user:password@host:port`" format (\ for example). |
| `device` | string | No | — | Type of device emulation. |
| `error_on_404` | boolean | No | — | Return error on 404 HTTP status on the target page (false by default). |
| `error_on_redirect` | boolean | No | — | Return error on redirect on the target page (false by default). |
| `js_script` | string | No | — | Custom JavaScript code to execute on the target page. |
```json theme={null}
{
"type": "object",
"properties": {
"PCID": {
"type": "string",
"description": "Pink Connect ID for the authenticated connection"
},
"selectors": {
"type": "array",
"items": {
"type": "string"
},
"description": "Multiple CSS selectors (null by default, returns whole page HTML)"
},
"url": {
"type": "string",
"description": "URL of the target page."
},
"headers": {
"type": "object",
"description": "HTTP headers to pass to the target page. Can be specified either via a nested query parameter (...&headers[One]=value1&headers=[Another]=value2) or as a JSON encoded object (...&headers={\"One\": \"value1\", \"Another\": \"value2\"})."
},
"timeout": {
"type": "integer",
"description": "Maximum web page retrieval time in ms. Increase it in case of timeout errors (10000 by default, maximum is 30000)."
},
"js": {
"type": "boolean",
"description": "Execute on-page JavaScript using a headless browser (true by default)."
},
"js_timeout": {
"type": "integer",
"description": "Maximum JavaScript rendering time in ms. Increase it in case if you see a loading indicator instead of data on the target page."
},
"wait_for": {
"type": "string",
"description": "CSS selector to wait for before returning the page content. Useful for pages with dynamic content loading. Overrides js_timeout."
},
"proxy": {
"type": "string",
"description": "Type of proxy. Use `residential` if your site restricts traffic from datacenters, or `stealth` for the most heavily protected sites with advanced anti-bot detection (`datacenter` by default). Residential and stealth proxy requests are more expensive than datacenter, see the pricing page for details.",
"enum": [
"datacenter",
"residential",
"stealth"
]
},
"country": {
"type": "string",
"description": "Country of the proxy to use (US by default).",
"enum": [
"us",
"gb",
"de",
"it",
"fr",
"ca",
"es",
"ru",
"jp",
"kr",
"in",
"hk",
"tr"
]
},
"custom_proxy": {
"type": "string",
"description": "Your own proxy URL to use instead of our built-in proxy pool in \"http://user:password@host:port\" format (Smartproxy for example)."
},
"device": {
"type": "string",
"description": "Type of device emulation.",
"enum": [
"desktop",
"mobile",
"tablet"
]
},
"error_on_404": {
"type": "boolean",
"description": "Return error on 404 HTTP status on the target page (false by default)."
},
"error_on_redirect": {
"type": "boolean",
"description": "Return error on redirect on the target page (false by default)."
},
"js_script": {
"type": "string",
"description": "Custom JavaScript code to execute on the target page."
}
},
"required": [
"PCID",
"url"
]
}
```
***
## webscraping\_ai\_get\_text
Page text by URL
**Parameters:**
| Parameter | Type | Required | Default | Description |
| ------------------- | ------- | -------- | ------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------ |
| `text_format` | string | No | — | Format of the text response (plain by default). "plain" will return only the page body text. "json" and "xml" will return a json/xml with "title", "description" and "content" keys. |
| `return_links` | boolean | No | — | \[Works only with text\_format=json] Return links from the page body text (false by default). Useful for building web crawlers. |
| `url` | string | Yes | — | URL of the target page. |
| `headers` | object | No | — | HTTP headers to pass to the target page. Can be specified either via a nested query parameter (...\&headers\[One]=value1\&headers=\[Another]=value2) or as a JSON encoded object (...\&headers=\{"One": "value1", "Another": "value2"}). |
| `timeout` | integer | No | — | Maximum web page retrieval time in ms. Increase it in case of timeout errors (10000 by default, maximum is 30000). |
| `js` | boolean | No | — | Execute on-page JavaScript using a headless browser (true by default). |
| `js_timeout` | integer | No | — | Maximum JavaScript rendering time in ms. Increase it in case if you see a loading indicator instead of data on the target page. |
| `wait_for` | string | No | — | CSS selector to wait for before returning the page content. Useful for pages with dynamic content loading. Overrides js\_timeout. |
| `proxy` | string | No | — | Type of proxy. Use `residential` if your site restricts traffic from datacenters, or `stealth` for the most heavily protected sites with advanced anti-bot detection (`datacenter` by default). Residential and stealth proxy requests are more expensive than datacenter, see the pricing page for details. |
| `country` | string | No | — | Country of the proxy to use (US by default). |
| `custom_proxy` | string | No | — | Your own proxy URL to use instead of our built-in proxy pool in "`http://user:password@host:port`" format (\ for example). |
| `device` | string | No | — | Type of device emulation. |
| `error_on_404` | boolean | No | — | Return error on 404 HTTP status on the target page (false by default). |
| `error_on_redirect` | boolean | No | — | Return error on redirect on the target page (false by default). |
| `js_script` | string | No | — | Custom JavaScript code to execute on the target page. |
```json theme={null}
{
"type": "object",
"properties": {
"PCID": {
"type": "string",
"description": "Pink Connect ID for the authenticated connection"
},
"text_format": {
"type": "string",
"description": "Format of the text response (plain by default). \"plain\" will return only the page body text. \"json\" and \"xml\" will return a json/xml with \"title\", \"description\" and \"content\" keys.",
"enum": [
"plain",
"xml",
"json"
]
},
"return_links": {
"type": "boolean",
"description": "[Works only with text_format=json] Return links from the page body text (false by default). Useful for building web crawlers."
},
"url": {
"type": "string",
"description": "URL of the target page."
},
"headers": {
"type": "object",
"description": "HTTP headers to pass to the target page. Can be specified either via a nested query parameter (...&headers[One]=value1&headers=[Another]=value2) or as a JSON encoded object (...&headers={\"One\": \"value1\", \"Another\": \"value2\"})."
},
"timeout": {
"type": "integer",
"description": "Maximum web page retrieval time in ms. Increase it in case of timeout errors (10000 by default, maximum is 30000)."
},
"js": {
"type": "boolean",
"description": "Execute on-page JavaScript using a headless browser (true by default)."
},
"js_timeout": {
"type": "integer",
"description": "Maximum JavaScript rendering time in ms. Increase it in case if you see a loading indicator instead of data on the target page."
},
"wait_for": {
"type": "string",
"description": "CSS selector to wait for before returning the page content. Useful for pages with dynamic content loading. Overrides js_timeout."
},
"proxy": {
"type": "string",
"description": "Type of proxy. Use `residential` if your site restricts traffic from datacenters, or `stealth` for the most heavily protected sites with advanced anti-bot detection (`datacenter` by default). Residential and stealth proxy requests are more expensive than datacenter, see the pricing page for details.",
"enum": [
"datacenter",
"residential",
"stealth"
]
},
"country": {
"type": "string",
"description": "Country of the proxy to use (US by default).",
"enum": [
"us",
"gb",
"de",
"it",
"fr",
"ca",
"es",
"ru",
"jp",
"kr",
"in",
"hk",
"tr"
]
},
"custom_proxy": {
"type": "string",
"description": "Your own proxy URL to use instead of our built-in proxy pool in \"http://user:password@host:port\" format (Smartproxy for example)."
},
"device": {
"type": "string",
"description": "Type of device emulation.",
"enum": [
"desktop",
"mobile",
"tablet"
]
},
"error_on_404": {
"type": "boolean",
"description": "Return error on 404 HTTP status on the target page (false by default)."
},
"error_on_redirect": {
"type": "boolean",
"description": "Return error on redirect on the target page (false by default)."
},
"js_script": {
"type": "string",
"description": "Custom JavaScript code to execute on the target page."
}
},
"required": [
"PCID",
"url"
]
}
```