Web Search Agents

Nimble Web Search Agents (WSA) are intelligent agents that search, extract, and structure real-time knowledge from any website. Describe what data you need, and Nimble builds an agent that delivers structured results at scale — no CSS selectors, no scraping expertise, no maintenance headaches. A growing library of pre-built agents covers popular sites like Amazon, Google, and Walmart with zero configuration. For any other website, create a custom agent in seconds using natural language.

Try Nimble Studio

Create a Web Search Agent for any website — no coding required

Install Plugin

Use Nimble directly in Claude Code or Cursor

Quick Start

Example Request

from nimble_python import Nimble

nimble = Nimble(api_key="YOUR-API-KEY")

result = nimble.agent.run(
    agent="amazon_pdp",
    params={
        "asin": "B0DLKFK6LR"
    }
)

parsed = result.data.parsing["parsed"]
print(f"Product: {parsed['product_title']}")
print(f"Price: ${parsed['web_price']}")

Example Response

{
  "url": "https://www.amazon.com/dp/B08N5WRWNW",
  "task_id": "b1fa7943-cba5-4ec2-a88c-4d2d6799c794",
  "status": "success",
  "data": {
    "parsing": {
      "asin": "B08N5WRWNW",
      "product_title": "Apple AirPods Pro (2nd Generation)",
      "brand": "Apple",
      "web_price": 249.0,
      "list_price": 279.0,
      "average_of_reviews": 4.7,
      "number_of_reviews": 125432,
      "availability": true
    }
  },
  "status_code": 200
}

How it works

Select an agent and provide inputs

Choose an agent by name (e.g., amazon_pdp, google_search) and pass the required parameters like product ID, search query, or URL

Nimble handles everything

The agent fetches the page, handles anti-bot protection, and extracts data using battle-tested selectors maintained by Nimble

Receive structured data

Get clean, normalized JSON with consistent field names - ready to use in your application immediately

Any website. Structured data. One API.

Every Web Search Agent — whether pre-built or custom — works through the same simple API. Pick a site, describe what you need, and get production-ready structured data.

Pre-Built Agents

\ A growing library of agents for popular websites — maintained by Nimble 24/7, auto-healing when sites change. Browse the Gallery.

Custom Agents

\ Create an agent for any website using Nimble Studio or the Nimble plugin in your IDE. Describe what you need in plain English.

Parameters

Supported input parameters:

agent - Required

agent

string

required

The name of the pre-built agent to use. Each agent is designed for a specific platform or data type.Popular agents:

amazon_pdp - Amazon product pages
amazon_serp - Amazon search results
google_search - Google search results
google_maps_search - Google Maps locations
walmart_pdp - Walmart products
chatgpt - ChatGPT prompt results
perplexity - Perplexity prompt results

Browse all agents →

params - Required

params

object

required

Agent-specific parameters that tell the agent what data to fetch. Each agent has different requirements.Common param types:

Product IDs (ASINs, SKUs)
Search queries
URLs or usernames
Page numbers for pagination

Example:

{
  "asin": "B08N5WRWNW"
}

localization

boolean

default:"false"

Enable location-based pricing and availability. Required when passing zip_code or store_id in params.Only available for agents that support localization (check agent details).Example:

{
  "agent": "amazon_pdp",
  "localization": true,
  "params": {
    "asin": "B08N5WRWNW",
    "zip_code": "90210"
  }
}

formats

array

default:"[]"

Output formats to include in the response alongside data.parsing. By default, only structured parsed data is returned.Available formats:

html - Raw HTML source of the extracted page
markdown - Clean markdown version of the page
headers - HTTP response headers as a key-value object under data.headers
links - All URLs found on the page as an array under data.links

Example:

"formats": ["html", "markdown", "links"]

Breaking change: data.html is no longer returned by default. To receive HTML in the response, pass "html" in the formats array.

Each agent has unique parameter requirements. Check the Agent Gallery for exact parameters for each agent.

Usage

E-commerce product extraction

Extract product data from Amazon, Walmart, and other retailers:

from nimble_python import Nimble

nimble = Nimble(api_key="YOUR-API-KEY")

# Amazon product
result = nimble.agent.run(
    agent="amazon_pdp",
    params={
        "asin": "B08N5WRWNW"
    }
)

parsed = result.data.parsing["parsed"]
print(f"Product: {parsed['product_title']}")
print(f"Price: ${parsed['web_price']}")
print(f"Rating: {parsed['average_of_reviews']}")

Search results extraction

Get search results from Google, Amazon, and other platforms:

from nimble_python import Nimble

nimble = Nimble(api_key="YOUR-API-KEY")

# Amazon search
result = nimble.agent.run(
    agent="amazon_serp",
    params={
        "keyword": "wireless headphones"
    }
)

products = result.data.parsing["parsed"]
print(f"Found {len(products)} products")
for product in products:
    print(f"- {product['product_name']}: ${product['price']}")

Google Maps extraction

Find businesses and locations:

from nimble_python import Nimble

nimble = Nimble(api_key="YOUR-API-KEY")

result = nimble.agent.run(
    agent="google_maps_search",
    params={
        "query": "coffee shops near Times Square"
    }
)

parsed = result.data.parsing["parsed"]
for place in parsed["entities"]["SearchResult"]:
    print(f"- {place['title']} ({place['rating']} stars)")

LLM platform extraction

Get responses from AI platforms like ChatGPT and Perplexity:

from nimble_python import Nimble

nimble = Nimble(api_key="YOUR-API-KEY")

result = nimble.agent.run(
    agent="chatgpt",
    params={
        "prompt": "What are the best practices for web scraping?"
    }
)

parsed = result.data.parsing["parsed"]
print(parsed["response"])

Localized extraction

Get location-specific pricing and availability:

from nimble_python import Nimble

nimble = Nimble(api_key="YOUR-API-KEY")

# Get New York pricing
result = nimble.agent.run(
    agent="amazon_pdp",
    localization=True,
    params={
        "asin": "B08N5WRWNW",
        "zip_code": "10001"
    }
)

parsed = result.data.parsing["parsed"]
print(f"Price in NYC: ${parsed['web_price']}")

Async & Batch

Run agent extractions in the background or submit multiple agent requests at once. Both modes return immediately with a task or batch ID — no waiting while Nimble processes the work.

Async Example
Batch Example

from nimble_python import Nimble

nimble = Nimble(api_key="YOUR-API-KEY")

response = nimble.agent.run_async(
    agent="amazon_pdp",
    params={"asin": "B0DLKFK6LR"}
)

print(f"Task created: {response.task_id}")
# Poll GET /v1/tasks/{task_id} until state == "success"

from nimble_python import Nimble

nimble = Nimble(api_key="YOUR-API-KEY")

response = nimble.agent.batch(
    inputs=[
        {"params": {"keyword": "iphone 15"}},
        {"params": {"keyword": "iphone 16"}},
        {"params": {"keyword": "iphone 17"}},
    ],
    shared_inputs={
        "agent": "amazon_serp",
    }
)

print(f"Batch created: {response.batch_id} ({response.batch_size} tasks)")
# Poll GET /v1/batches/{batch_id}/progress until completed == true

Use async or batch when you need to:

Run agents without blocking your application
Process multiple agent requests in parallel
Deliver results to cloud storage (S3 / GCS) automatically
Receive webhook notifications when tasks complete
Integrate agents into scheduled or queued workflows

How it works

Submit a request

Send a POST request to the async or batch endpoint. The API returns immediately with a task_id (async) or batch_id (batch) — no waiting for the agent to finish.

Nimble processes in the background

The agent runs asynchronously. For batch, each input becomes an independent task processed in parallel.

Receive results your way

Choose how results are delivered:

Polling — check task status and fetch results on demand
Webhook — get notified automatically when a task completes
Cloud storage — results saved directly to your S3 or GCS bucket

For setup instructions and code examples, see the Callbacks & Delivery guide.

Async

Submit a single agent request and receive a task_id immediately. Retrieve results via polling, webhook, or cloud storage.

POST https://sdk.nimbleway.com/v1/agents/async

Parameters

Accepts all Agents API parameters, plus async-specific delivery options:

agent (required) — The agent to run
params (required) — Agent-specific input parameters
localization — Enable location-based data
formats — Output formats: html, markdown, headers

Async-specific parameters:

storage_type

string

Storage provider for results. When specified, results are saved to your cloud storage instead of Nimble’s servers.Options: s3 (Amazon S3), gs (Google Cloud Storage)

storage_url

string

Bucket path where results will be saved. Results are stored as {task_id}.json at the specified location.Format: s3://your-bucket/path/prefix/

storage_compress

boolean

default:"false"

Compress results with GZIP before saving. When true, results are saved as {task_id}.json.gz.

storage_object_name

string

Custom filename for the stored object instead of the default task ID.Example: "my-custom-name" saves as my-custom-name.json

callback_url

string

Webhook URL to receive a POST request when the task completes. Nimble sends task metadata (without result data) to this URL when the agent finishes.Example: https://your-api.com/webhook/complete

The endpoint returns immediately with a task ID:

{
  "status": "success",
  "task": {
    "id": "8e8cfde8-345b-42b8-b3e2-0c61eb11e00f",
    "state": "pending",
    "created_at": "2026-01-24T12:36:24.685Z",
    "modified_at": "2026-01-24T12:36:24.685Z",
    "input": {}
  }
}

Status & Results

When polling, the typical flow is:

Poll GET /v1/tasks/{task_id} until state: "success"
Call GET /v1/tasks/{task_id}/results to retrieve the extracted data

Task states:

State	Description
`pending`	Task queued, waiting to start
`success`	Extraction complete, results available
`error`	Extraction failed

Batch

Submit up to 1,000 agent requests in a single request. Each input runs as an independent async task. Use shared_inputs to set the agent and common settings — individual items in inputs can override params per item.

POST https://sdk.nimbleway.com/v1/agents/batch

Parameters

inputs — Required

inputs

array

required

Array of per-item inputs. Supports up to 1,000 items per batch. agent is not set here — it must be in shared_inputs and applies to all items.Each item supports:

params — Agent-specific inputs for this item (e.g. asin, keyword). Merged with shared_inputs.params — keys in inputs[i].params take priority on conflicts.
localization — Override the shared localization setting for this item
formats — Override the shared formats setting for this item

"inputs": [
  { "params": { "keyword": "iphone 15" } },
  { "params": { "keyword": "iphone 16" }, "localization": true }
]

shared_inputs — Required

shared_inputs

object

required

Shared configuration applied to all items in the batch. Each batch runs a single agent — agent is required here and cannot be set per item.

agent (required) — The agent to run for all items
params — Default agent params and delivery options, merged with each item’s params. Per-item values win on conflicts.
localization — Default localization setting for all items (overridable per item)
formats — Default output formats for all items (overridable per item)

Delivery options (set inside params):

storage_type — s3 or gs
storage_url — bucket path for results
storage_compress — GZIP compress results
callback_url — webhook on completion

"shared_inputs": {
  "agent": "amazon_serp",
  "params": {
    "storage_type": "s3",
    "storage_url": "s3://your-bucket/results/",
    "callback_url": "https://your-api.com/webhooks/complete"
  },
  "formats": ["markdown"]
}

Examples

Example 1: Batch multiple keywords

Run the same agent for multiple search terms with S3 delivery:

response = nimble.agent.batch(
    inputs=[
        {"params": {"keyword": "iphone 15"}},
        {"params": {"keyword": "samsung galaxy s24"}},
        {"params": {"keyword": "pixel 9"}},
    ],
    shared_inputs={
        "agent": "amazon_serp",
        "params": {
            "storage_type": "s3",
            "storage_url": "s3://your-bucket/results/",
            "callback_url": "https://your-api.com/webhooks/batch-complete",
        },
    }
)

print(f"Batch ID: {response.batch_id}")
print(f"Tasks submitted: {response.batch_size}")

Example 2: Batch multiple product ASINs

Extract product details for many ASINs at once:

response = nimble.agent.batch(
    inputs=[
        {"params": {"asin": "B0DLKFK6LR"}},
        {"params": {"asin": "B08N5WRWNW"}},
        {"params": {"asin": "B09XS7JWHH"}},
    ],
    shared_inputs={
        "agent": "amazon_pdp",
        "params": {
            "callback_url": "https://your-api.com/webhooks/batch-complete",
        },
    }
)

print(f"Batch ID: {response.batch_id}")

The endpoint returns immediately with a batch_id and the initial task list:

{
  "batch_id": "4b0a90bf-c951-42e4-95b3-a95a65ba69fc",
  "batch_size": 3,
  "tasks": [
    { "id": "task-001-uuid", "state": "pending", "batch_id": "4b0a90bf-..." },
    { "id": "task-002-uuid", "state": "pending", "batch_id": "4b0a90bf-..." },
    { "id": "task-003-uuid", "state": "pending", "batch_id": "4b0a90bf-..." }
  ]
}

Status & Results

When polling, the typical flow is:

Poll /v1/batches/{batch_id}/progress until completed: true
Fetch /v1/batches/{batch_id} to get all task IDs and states
For each success task, call GET /v1/tasks/{task_id}/results

Batch states:

State	Description
`pending`	Task queued, waiting to start
`in_progress`	Task is currently being processed
`success`	Extraction complete, results available
`error`	Extraction failed

Poll for batch completion

Call /v1/batches/{batch_id}/progress repeatedly until completed: true. This is a lightweight endpoint — use it for polling.

GET https://sdk.nimbleway.com/v1/batches/{batch_id}/progress

{
  "id": "4b0a90bf-c951-42e4-95b3-a95a65ba69fc",
  "completed": false,
  "completed_count": 47,
  "progress": 0.47,
  "completed_at": null
}

Fetch the full batch details

Once completed: true, fetch the batch details to get all task IDs, states, and download URLs.

GET https://sdk.nimbleway.com/v1/batches/{batch_id}

{
  "id": "4b0a90bf-c951-42e4-95b3-a95a65ba69fc",
  "completed": true,
  "completed_count": 3,
  "progress": 1.0,
  "tasks": [
    {
      "id": "task-001-uuid",
      "state": "success",
      "download_url": "https://sdk.nimbleway.com/v1/tasks/task-001-uuid/results"
    },
    {
      "id": "task-002-uuid",
      "state": "error",
      "error": "Connection timeout"
    }
  ]
}

Retrieve results per task

Iterate over the task list and call GET /v1/tasks/{task_id}/results for each success task.

import requests

batch = requests.get(
    f"https://sdk.nimbleway.com/v1/batches/{batch_id}",
    headers={"Authorization": "Bearer YOUR-API-KEY"}
).json()

for task in batch["tasks"]:
    if task["state"] == "success":
        result = requests.get(
            task["download_url"],
            headers={"Authorization": "Bearer YOUR-API-KEY"}
        ).json()
        print(result["data"]["parsing"])

Agent Gallery

Browse pre-built agents maintained by Nimble for popular platforms:

Explore Full Gallery

Browse all agents with interactive documentation and live testing

E-commerce

Agent	Platform	Description
`amazon_pdp`	Amazon	Product details, pricing, reviews
`amazon_serp`	Amazon	Search results with products
`walmart_pdp`	Walmart	Product details and pricing
`walmart_search`	Walmart	Search results
`target_pdp`	Target	Product details
`best_buy_pdp`	Best Buy	Product details

Search Engines

Agent	Platform	Description
`google_search`	Google	Search results with snippets
`google_maps_search`	Google Maps	Business listings and locations
`google_search_aio`	Google	AI Overview results

LLM Platforms

Agent	Platform	Description
`chatgpt`	ChatGPT	Prompt responses
`perplexity`	Perplexity	Search + AI responses
`gemini`	Google Gemini	Prompt responses
`grok`	Grok	Prompt responses

Agent	Platform	Description
`tiktok_account`	TikTok	Account profiles and videos
`facebook_page`	Facebook	Page information
`youtube_shorts`	YouTube	Short-form videos

Create Custom Agents

Can’t find an agent for your target website? Create your own using Nimble Studio - no coding required.

Open Nimble Studio

Go to the Nimble Studio in Nimble Platform

Provide URL and describe your needs

Enter the website URL and describe what data you need in plain English:“Extract product name, price, rating, and all review comments”

AI creates your agent

Our AI analyzes the page and builds an extraction agent automatically - no CSS selectors needed

Test and refine

Preview extracted data and refine your description if needed

Use via API

Your custom agent is available via the same Agent API with your chosen name

Custom agent example

from nimble_python import Nimble

nimble = Nimble(api_key="YOUR-API-KEY")

# Use your custom agent just like a public one
result = nimble.agent.run(
    agent="my_custom_store_pdp",  # Your custom agent name
    params={
        "url": "https://www.nike.com/t/air-max-90-mens-shoes"
    }
)

parsed = result.data.parsing["parsed"]
print(f"Product: {parsed['product_name']}")
print(f"Price: ${parsed['price']}")

Pre-built agents are maintained by Nimble 24/7 and auto-heal when sites change. Custom agents can be updated anytime in Nimble Studio. For popular sites, check the Gallery first — a pre-built agent may already exist.

Response Fields

Field	Type	Description
`url`	string	The URL that was extracted
`task_id`	string	Unique identifier for the request
`status`	string	`success` or `failed`
`data.parsing`	object	Structured extracted data (always returned)
`data.html`	string	Raw HTML — only included when `formats: ["html"]`
`data.markdown`	string	Markdown — only included when `formats: ["markdown"]`
`data.headers`	object	HTTP response headers — only included when `formats: ["headers"]`
`status_code`	number	HTTP status code from target

Use cases

Price Monitoring

Track prices across Amazon, Walmart, and other retailers with consistent data formats

Product Research

Gather comprehensive product data from major e-commerce platforms

Search Tracking

Monitor Google, Bing, and marketplace search results for SEO and visibility

Competitive Intelligence

Extract competitor data from any website using public or custom agents

Agents vs other tools

What you need	Use
Data from popular sites (Amazon, Google, etc.)	Public Agents - browse gallery
Data from sites not in the gallery	Custom Agents - create in Studio
Data from specific URLs (expert users)	Extract - full control with CSS selectors
Data from entire website	Crawl
Search web + extract content from results	Search

Web Search Agents work with any website. Pre-built agents cover popular sites with zero configuration. For everything else, create a custom agent in seconds — or install the Nimble plugin and let your AI coding assistant handle it.

Next steps

Try Nimble Studio

Create a Web Search Agent for any website — see the value in minutes

Install Plugin

Use Nimble in Claude Code or Cursor — your AI assistant builds agents for you

Agent Gallery

Browse pre-built agents for popular sites

API Reference

Explore the Agents API for direct integration

Introduction

Web Tools

Guides

SDKs

Admin

Try Nimble Studio

Install Plugin

​Quick Start

​Example Request

​Example Response

​How it works

​Any website. Structured data. One API.

Pre-Built Agents

Custom Agents

​Parameters

​Usage

​E-commerce product extraction

​Search results extraction

​Google Maps extraction

​LLM platform extraction

​Localized extraction

​Async & Batch

​How it works

​Async

​Parameters

​Status & Results

​Batch

​Parameters

​Examples

​Status & Results

​Poll for batch completion

​Fetch the full batch details

​Retrieve results per task

​Agent Gallery

Explore Full Gallery

​E-commerce

​Search Engines

​LLM Platforms

​Social Media

​Create Custom Agents

​Custom agent example

​Response Fields

​Use cases

Price Monitoring

Product Research

Search Tracking

Competitive Intelligence

​Agents vs other tools

​Next steps

Try Nimble Studio

Install Plugin

Agent Gallery

API Reference

Quick Start

Example Request

Example Response

How it works

Any website. Structured data. One API.

Parameters

Usage

E-commerce product extraction

Search results extraction

Google Maps extraction

LLM platform extraction

Localized extraction

Async & Batch

How it works

Async

Parameters

Status & Results

Batch

Parameters

Examples

Status & Results

Poll for batch completion

Fetch the full batch details

Retrieve results per task

Agent Gallery

E-commerce

Search Engines

LLM Platforms

Social Media

Create Custom Agents

Custom agent example

Response Fields

Use cases

Agents vs other tools

Next steps