Page Interactions allow users to perform operations on a webpage before the web data is collected and returned including clicking, typing, scrolling, and more.
This is useful (and sometimes necessary) in a variety of situations, such as one-page applications that use lazy loading and require scrolling in order to load the desired data. Another example includes E-commerce websites that require button clicks or zip code input to display a product price.
Currently, Page Interactions are synchronous, so operations are run sequentially one by one, and the overall sequence is limited to a maximum timeout of 120 seconds. Page Interactions are supported by real-time, asynchronous, and batch requests.
Using Page Interactions
To perform page interactions, set render to true and add a render_flow argument with a JSON list of the operations you’d like to perform, as in the example below:
Page Interactions will not function if rendering is not enabled in the request. Please be sure to set render:true in order for interactions to function correctly.
Response
When a request with Page Interactions is completed, an additional property named render:true is included that contains a log detailing the result of each interaction. For each interaction, the following fields are present:
Name – the interaction type (eg. scroll, wait_and_click, wait_and_type, etc).
Result – some operations may return information. If there is information to return, result will contain the returned value, otherwise result will be true.
Error – this field is only present if an error occurs, and details the type of error encountered.
Status – the status of the interaction. Interactions can have one of four statuses:
Status
Description
no-run
the interaction was not performed.
in-progress
The interaction was in progress when the global 120-second timeout was triggered.
done
the interaction finished successfully.
error
the interaction encountered an error.
The above request would produce the following response:
Multiple interactions can be chained together and performed sequentially. To do this, simply add additional steps to the render:true property. When chaining interactions, it’s important to consider the maximum overall execution time of all the requested interactions.
Each request has a maximum timeout of 120 seconds for the overall execution, which includes all render flow interactions such as delays, clicks, scrolls, and timeouts if and when they occur.
Additionally, if any particular interaction encounters an error, interactions that come after it will not be executed. The chain is halted, and the data from the webpage is returned normally.
Below is an example of a request with several interactions chained together in a sequence:
The most common errors that are likely to occur include execution errors or timeouts. Most interaction functions include an optional timeout property that allows users to cap their execution time.
If an error is encountered in a chain of interactions, interactions that come after the failed interaction will not be executed. The data is collected from the web resource normally, and returned in accordance with the parameters of the associated request.
Additionally, the error encountered is returned to the user for debugging purposes. In the below example, the two initial interactions were executed successfully, followed by a failed interaction, and finally an interaction that was not executed because of the previously failed interaction: