Executing HTTP Requests

http_request Function

The http_request function allows developers to perform additional HTTP POST/GET requests during a page interaction, unlocking access to more data. By capturing these internal API calls, you can directly access key data in machine-readable formats like JSON, bypassing the need to parse HTML.

Note: render_flow may include multiple http_request steps, but only the first is free of charge. Starting from the 2nd http_request, the request will be classified and charged as a VX6 request.

Parameter	Required	Description
url	Required	URL | The target URL for the HTTP request
method	Optional (Default = GET)	Enum | The HTTP request method -GET, POST
headers	Optional	Object | Additional headers required for the request in JSON format
timeout	Optional	Integer | Controls the timeout if the internal HTTP request being performed in ms

Example request

cURL

curl -X POST 'https://api.webit.live/api/v1/realtime/web' \
--header 'Authorization: Basic <credential string>' \
--header 'Content-Type: application/json' \
--data-raw '{
    "url": "https://www.example.com",
    "render": true,
    "render_flow": [{
        "http_request": {
                "url": "https://www.example.com/api/data",
                "method": "POST",
                "headers": {...}
                "timeout": 5000
        }
    }]
}'

Like all Page Interactions, infinite scrolling is capped by the global 120-second session timeout, and will be terminated if this limit is reached.