LogoLogo
Nimble HomeLoginCreate an Account
  • Home
  • Quick Start Tutorials
    • Tutorial Library
      • Track SEO and SEM Ranking
      • Reddit as a Guerilla Marketing Strategy
  • Nimble Platform
    • Nimble Platform Overview
    • Online Pipelines
      • Supermarkets
        • ASDA
        • Tesco Groceries
        • Sainsbury’s
        • Morrisons
      • eCommerce
      • Restaurants
        • Yelp
        • Tabelog
        • Uber Eats Japan
        • Demaecan
        • Uber Eats US
      • Real Estate
        • Zillow
  • Nimble SDK
    • SDK Overview
    • Web API
      • Web API Overview
      • API Product Specs
      • Nimble Web API Quick Start Guide
        • Introduction
        • Nimble APIs Authentication
        • Real-time URL request
        • Delivery methods
        • Batch processing
        • Response codes
        • FAQs
      • Nimble Web API Functions
        • Realtime, Async & Batch Request
        • Geo Location Targeting
        • Javascript Rendering
        • Page Interaction
          • Wait (delay)
          • Wait for Selector
          • Wait and Click
          • Wait and Type
          • Scroll
          • Scroll to
          • Infinite Scrolling
          • Capturing Screenshots
          • Collecting Cookies
          • Executing HTTP Requests
          • Operation Reference
        • Network Capture
          • Filter by URL Matching
          • Filter By Resource Type
            • Real World Demo: Capturing Ajax Requests
          • Wait for Requests
          • Capturing XHR without Rendering
          • Operation Reference
        • Data Parsing
          • Parsing Templates
          • Merge Dynamic Parser
        • Custom Headers & Cookies
        • General Params
      • Vertical Endpoints
        • SERP API
          • Real-time search request
          • Getting local data
          • Browsing SERP pagination
          • Delivery methods
          • Batch Processing
          • Endpoints and Response Codes
        • Maps API
          • Searching for places
          • Getting information about a place
          • Collecting reviews
          • Delivery methods
          • Batch processing
          • Endpoints and Response Codes
    • Web Retrieval API
      • Web Retrieval API Overview
    • Proxy API
      • Nimble IP Overview
      • Nimble IP Quick Start Guide
        • Send a request
        • Nimble IP Autentication
        • Geotargeting and session control
        • Response codes
        • FAQs
      • Nimble IP Functions
        • Country/state/city geotargeting
        • Controlling IP rotation
        • Geo-sessions: longer, stickier, more accurate sessions
        • Using IPv6 Proxies
        • Response Codes
      • Integration Guides
        • Incogniton
        • Kameleo
        • VMLogin
        • AdsPower
        • FoxyProxy
        • Android
        • Multilogin
        • iOS
        • SwitchyOmega
        • Windows
        • macOS
        • Proxifier
        • MuLogin
        • Puppeteer
        • Selenium
        • Scrapy
    • Client Libraries
      • Installation
      • Quick Start
  • AI Agents
    • LangChain Integration
    • MCP Server
  • Technologies
    • Browserless Drivers
      • API Driver-Based Pricing
    • IP Optimization Models
    • AI Parsing Skills
  • Management Tools
    • Nimble Dashboard
      • Exploring the User Dashboard
      • Managing Pipelines
      • Reporting and Analytics
      • Account Settings
      • Experimenting with the Playground
      • Billing and history
    • Nimble Admin API
      • Admin API basics
      • Admin API reference
  • General
    • Onboarding Guide
      • Getting started with Nimble's User Dashboard
      • Nimble IP Basics
      • Nimble API Basics
      • Helpful Resources
    • FAQs
      • Account Settings and Security
      • Billing and Pricing
      • Tools and Integrations
      • Nimble API
      • Nimble IP
    • Deprecated APIs
      • E-commerce API
        • E-commerce API Authentication
        • Real-time product request
        • Real-time product search request
        • Delivery methods
        • Batch Processing
        • Endpoints and Response Codes
      • Unlocker Proxy Overview
        • Unlocker Proxy Quick Start Guide
          • Real-time request
          • FAQs
        • Unlocker Proxy FAQ
Powered by GitBook
On this page
  • URL Filtering
  • Exact Match Filtering
  • Containing a Match Filtering
  • Example Request
  • Example Response
  1. Nimble SDK
  2. Web API
  3. Nimble Web API Functions
  4. Network Capture

Filter by URL Matching

URL Filtering

Requests can be captured by using the network_capture field, which receives a JSON array that can include one or more filters. Each filter specifies which requests should be collected for later inspection.

The below example demonstrates two simple network capture filters, and how they are performed:

Parameter
Required
Description

url.type

Optional (default = exact)

Enum: |exact, contains - choosing which type of URL filtering to apply

url.value

Required when filtering by URL

String | The URL or URL portion to match to.

method

Optional (default = Any)

Enum | HTTP method should be filtered for. Examples: GET, POST, PUT.

Exact Match Filtering

In the this example, we ask to capture requests that match the URL we've provided exactly and that only use the GET method. Exact matches are very precise and can be used when you've determined exactly which network request you'd like to capture.

Containing a Match Filtering

In the this example, we ask to capture all .css requests by searching for requests to URLs that include ".css". The request method has been intentionally left out to capture requests using any method.

Using containing filters helps broaden our network captures, and can be very useful when needing all files of a certain type (css, js, etc.), a particular request that includes dynamic variables, or when the exact request is still not fully known.

Example Request

curl -X POST 'https://api.webit.live/api/v1/realtime/web' \
--header 'Authorization: Basic <credential string>' \
--header 'Content-Type: application/json' \
--data-raw '{
    "url": "https://www.example.com/",
    "render": true,
    "network_capture": [
            {
                "method": "GET",
                "url": {
                    "type": "exact",
                    "value": "https://i5.walmartimages.com/a14/capabilities.min.js"
                }
            },
            {
                "url": {
                    "type": "contains",
                    "value": ".css"
                }
            }
        ]
}'

Example Response

In the example response above, we see that network capture returns:

  • filter - the filter that was defined initially.

  • result - the requests found that match the filter

    • request - the headers and metadata of the request.

    • response - the headers and data of the response.

      • body - under response, body will contain the contents of the captured resource.

{
...
"network_capture": {
    "filter":
    {
        "method": "GET",
        "url":
        {
            "type": "exact",
            "value": "https://i5.walmartimages.com/a14/capabilities.min.js"
        }
    },
    "results":
    [
        {
            "request":
            {
                "method": "GET",
                "url": "https://i5.walmartimages.com/a14/capabilities.min.js",
                "headers":
                {
                    "sec-ch-ua": "\"Chromium\";v=\"112\", \"Google Chrome\";v=\"112\", \"Not:A-Brand\";v=\"99\"",
                    "referer": "https://www.example.com/",
                    "accept-language": "en,en;q=0.9",
                    "sec-ch-ua-mobile": "?0",
                    "user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/112.0.0.0 Safari/537.36",
                    "sec-ch-ua-platform": "\"Windows\""
                }
            },
            "response":
            {
                "status": 200,
                "status_text": "",
                "headers":
                {
                    "accept-ranges": "bytes",
                    "cache-control": "max-age=2702",
                    "content-length": "538511",
                    "content-type": "application/javascript",
                    "date": "...",
                    "etag": "\"6a6c1c29ddb3737a4398ff1f7b1063bf\"",
                    "last-modified": "...",
                    "strict-transport-security": "max-age=15552000; includeSubDomains; preload",
                    "vary": "Access-Control-Request-Headers,Access-Control-Request-Method,Origin",
                    "x-amz-request-id": "tx00000000000002e8d4651-0064943c55-6d194055-nyc3c",
                    "x-hw": "1687437271.dop007.ml1.t,1687437271.cds219.ml1.hn,1687437271.cds216.ml1.c",
                    "x-rgw-object-type": "Normal"
                },
                "serialization": "none"
            },
            "body": "..."
        },
        {
        "filter": {
                "url": {
                    "type": "contains",
                    "value": ".css"
                }
            },
            "results": [
                {
                    "request": {
                        "method": "GET",
                        "url": "https://www.example.com/wp-content/cache/min/1/10069917ea6c78bacd530335b1684679.css",
                        "headers": {
                            "sec-ch-ua": "\"Chromium\";v=\"112\", \"Google Chrome\";v=\"112\", \"Not:A-Brand\";v=\"99\"",
                            "referer": "https://www.example.com/",
                            "accept-language": "en,en;q=0.9",
                            "sec-ch-ua-mobile": "?0",
                            "user-agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/112.0.0.0 Safari/537.36",
                            "sec-ch-ua-platform": "\"Windows\""
                        }
                    },
                    "response": {
                        "status": 200,
                        "status_text": "",
                        "headers": {
                            "cache-control": "max-age=31536000",
                            "content-encoding": "gzip",
                            "content-length": "81027",
                            "content-type": "text/css",
                            "date": "Thu, 22 Jun 2023 12:34:30 GMT",
                            "etag": "\"647713da-13c83\"",
                            "expires": "Fri, 21 Jun 2024 12:34:30 GMT",
                            "last-modified": "Wed, 31 May 2023 09:31:06 GMT",
                            "server": "nginx",
                            "vary": "Accept-Encoding"
                        },
                        "serialization": "none",
                        "body": "..."
		    }
                }
            ]
        }
    ]
}
}
PreviousNetwork CaptureNextFilter By Resource Type

Last updated 8 months ago