Infinite Scrolling

infinite_scroll Function

A common use case for Page Interactions is to scroll down pages that differ loading off-screen content until a user scrolls down to reveal it. Also known as "lazy loading", websites often do this to reduce page load times or as an alternative to pagination. Some examples include Facebook's newsfeed, Google News, and YouTube's search pages.

Page Interactions include a dedicated feature named infinite_scroll designed specifically to scroll through lazy loading web pages. It's controlled using three parameters:

Parameter	Required	Description
duration	Required when infinite_scroll is being used	Integer | A time interval in milliseconds during which scrolling should be continuously attempted.
loading_selector	Optional	String | An identifier of the HTML element that displays the loading/spinner indicator. This helps identify when loading starts and ends.
delay_after_scroll	Optional	Integer | The time delay, in ms, between scrolls.

Scrolling is initiated as soon as the page load event is fired.

Infinite Scroll was designed to be used in one of two main methods:

Identifying the loading element

Where possible, it is recommended to use this method, as it allows for more accurate scrolling operation. Use the loading_selector parameter to identify the loading indicator or spinner that is displayed while the page loads additional content, and refine the scrolling behavior with the delay_after_scroll and duration parameters.

Time-based scrolling

In cases where there is no loading indicator, the delay_after_scroll and duration parameters can be used to define and refine the scrolling action until it is well-timed for the particular webpage in question.

Example Request

cURL

curl -X POST 'https://api.webit.live/api/v1/realtime/web' \
--header 'Authorization: Basic <credential string>' \
--header 'Content-Type: application/json' \
--data-raw '{
    "url": "https://www.example.com",
    "render": true,
    "format": "json",
    "country": "US",
    "parse": true,
    "render_flow": [
        {
            "infinite_scroll": {
                "loading_selector": ".spinner",
                "delay_after_scroll": 2000,
                "duration": 15000
            }
        }    
    ]
}'

Like all Page Interactions, infinite scrolling is capped by the global 120-second session timeout, and will be terminated if this limit is reached.