Callbacks & Delivery

When using async operations like /extract/async, /extract/batch, /agent/async, or /crawl, you have three flexible options for receiving your results. Choose the method that best fits your infrastructure and workflow.

Polling

Pull results on-demand using task IDs

Callbacks

Receive push notifications when tasks complete

Cloud Delivery

Automatic delivery to your S3 or GCS bucket

Option 1: Polling (Pull)

The simplest approach - submit your async request, receive a task ID, and poll for results when ready.

Submit async request

Send a request to the async endpoint. You’ll receive a task or crawl ID to track your request.

Extract
Agent
Crawl
Batch

from nimble_python import Nimble

nimble = Nimble(api_key="YOUR-API-KEY")

response = nimble.extract_async(
    url="https://www.nimbleway.com",
    render=True,
    formats=["html", "markdown"]
)

task_id = response.task_id
print(f"Task submitted: {task_id}")

import Nimble from '@nimble-way/nimble-js';

const nimble = new Nimble({ apiKey: "YOUR-API-KEY" });

const response = await nimble.extractAsync({
  url: "https://www.nimbleway.com",
  render: true,
  formats: ["html", "markdown"]
});

const taskId = response.task_id;
console.log(`Task submitted: ${taskId}`);

curl -X POST 'https://sdk.nimbleway.com/v1/extract/async' \
--header 'Authorization: Bearer YOUR-API-KEY' \
--header 'Content-Type: application/json' \
--data-raw '{
    "url": "https://www.nimbleway.com",
    "render": true,
    "formats": ["html", "markdown"]
}'

from nimble_python import Nimble

nimble = Nimble(api_key="YOUR-API-KEY")

response = nimble.agent.run_async(
    agent="amazon_pdp",
    params={"asin": "B0DLKFK6LR"}
)

task_id = response.task_id
print(f"Task submitted: {task_id}")

import Nimble from '@nimble-way/nimble-js';

const nimble = new Nimble({ apiKey: "YOUR-API-KEY" });

const response = await nimble.agent.runAsync({
  agent: "amazon_pdp",
  params: { asin: "B0DLKFK6LR" }
});

const taskId = response.task_id;
console.log(`Task submitted: ${taskId}`);

curl -X POST 'https://sdk.nimbleway.com/v1/agent/async' \
--header 'Authorization: Bearer YOUR-API-KEY' \
--header 'Content-Type: application/json' \
--data-raw '{
    "agent": "amazon_pdp",
    "params": { "asin": "B0DLKFK6LR" }
}'

from nimble_python import Nimble

nimble = Nimble(api_key="YOUR-API-KEY")

result = nimble.crawl.run(
    url="https://www.nimbleway.com",
    limit=50
)

crawl_id = result.crawl_id
print(f"Crawl started: {crawl_id}")

import Nimble from '@nimble-way/nimble-js';

const nimble = new Nimble({ apiKey: "YOUR-API-KEY" });

const result = await nimble.crawl.run({
  url: "https://www.nimbleway.com",
  limit: 50
});

const crawlId = result.crawl_id;
console.log(`Crawl started: ${crawlId}`);

curl -X POST 'https://sdk.nimbleway.com/v1/crawl' \
--header 'Authorization: Bearer YOUR-API-KEY' \
--header 'Content-Type: application/json' \
--data-raw '{
    "url": "https://www.nimbleway.com",
    "limit": 50
}'

Submit up to 1,000 URLs in one request. Each URL becomes its own task, tracked under a shared batch_id.

from nimble_python import Nimble

nimble = Nimble(api_key="YOUR-API-KEY")

response = nimble.extract_batch(
    inputs=[
        {"url": "https://www.example.com/page1"},
        {"url": "https://www.example.com/page2"},
        {"url": "https://www.example.com/page3"},
    ],
    shared_inputs={
        "render": True,
        "country": "US",
        "formats": ["markdown"],
    }
)

batch_id = response.batch_id
print(f"Batch submitted: {batch_id} ({response.batch_size} tasks)")

import Nimble from '@nimble-way/nimble-js';

const nimble = new Nimble({ apiKey: "YOUR-API-KEY" });

const response = await nimble.extractBatch({
  inputs: [
    { url: "https://www.example.com/page1" },
    { url: "https://www.example.com/page2" },
    { url: "https://www.example.com/page3" },
  ],
  shared_inputs: {
    render: true,
    country: "US",
    formats: ["markdown"],
  },
});

const batchId = response.batch_id;
console.log(`Batch submitted: ${batchId} (${response.batch_size} tasks)`);

curl -X POST 'https://sdk.nimbleway.com/v1/extract/batch' \
--header 'Authorization: Bearer YOUR-API-KEY' \
--header 'Content-Type: application/json' \
--data-raw '{
    "inputs": [
        {"url": "https://www.example.com/page1"},
        {"url": "https://www.example.com/page2"},
        {"url": "https://www.example.com/page3"}
    ],
    "shared_inputs": {
        "render": true,
        "country": "US",
        "formats": ["markdown"]
    }
}'

Check status

Poll the status endpoint to monitor progress.

Extract / Agent
Batch
Crawl

import time

while True:
    my_task = nimble.tasks.get(task_id)
    print(f"Status: {my_task.task.state}")

    if my_task.task.state == "success":
        break
    elif my_task.task.state == "error":
        print(f"Task failed: {my_task.task.error}")
        break

    time.sleep(15)

while (true) {
  const myTask = await nimble.tasks.get(taskId);
  console.log(`Status: ${myTask.task.state}`);

  if (myTask.task.state === "success") break;
  if (myTask.task.state === "error") {
    console.log(`Task failed: ${myTask.task.error}`);
    break;
  }

  await new Promise(resolve => setTimeout(resolve, 15000));
}

curl 'https://sdk.nimbleway.com/v1/tasks/{task_id}' \
--header 'Authorization: Bearer YOUR-API-KEY'

# Response: { "task": { "id": "...", "state": "success", ... } }

Use the lightweight progress endpoint for polling. Once completed is true, fetch task details.

import time

while True:
    progress = nimble.batches.progress(batch_id)
    print(f"Progress: {progress.completed_count} / {progress.total} ({progress.progress:.0%})")

    if progress.completed:
        break

    time.sleep(15)

# Get all task IDs and states
batch = nimble.batches.get(batch_id)
for task in batch.tasks:
    print(f"Task {task.id}: {task.state}")

while (true) {
  const progress = await nimble.batches.progress(batchId);
  console.log(`Progress: ${progress.completed_count} / ${progress.total} (${Math.round(progress.progress * 100)}%)`);

  if (progress.completed) break;

  await new Promise(resolve => setTimeout(resolve, 15000));
}

// Get all task IDs and states
const batch = await nimble.batches.get(batchId);
batch.tasks.forEach(task => {
  console.log(`Task ${task.id}: ${task.state}`);
});

# Lightweight progress check
curl 'https://sdk.nimbleway.com/v1/batches/{batch_id}/progress' \
--header 'Authorization: Bearer YOUR-API-KEY'

# Full batch details (when complete)
curl 'https://sdk.nimbleway.com/v1/batches/{batch_id}' \
--header 'Authorization: Bearer YOUR-API-KEY'

Crawl has its own status endpoint that shows overall progress and individual page tasks.

import time

while True:
    my_crawl = nimble.crawl.status(crawl_id)
    print(f"Status: {my_crawl.status}")

    if status.state == "succeeded":
        break
    elif status.state == "failed":
        print(f"Task failed: {status.error}")
        break

    time.sleep(2)

while (true) {
  const myCrawl = await nimble.crawl.status(crawlId);
  console.log(`Status: ${myCrawl.status}`);

  if (status.state === "succeeded") break;
  if (status.state === "failed") {
    console.log(`Task failed: ${status.error}`);
    break;
  }

  await new Promise(resolve => setTimeout(resolve, 2000));
}

curl 'https://sdk.nimbleway.com/v1/crawl/{crawl_id}' \
--header 'Authorization: Bearer YOUR-API-KEY'

# Response: { "crawl": { "status": "running", "completed": 10, "total": 50, "tasks": [...] } }

Retrieve results

Once complete, fetch the full results.

Extract
Agent
Crawl
Batch

results = nimble.tasks.results(task_id)

print(f"HTML length: {len(results.data.html)}")
print(f"Markdown length: {len(results.data.markdown)}")

const results = await nimble.tasks.results(taskId);

console.log(`HTML length: ${results.data.html.length}`);
console.log(`Markdown length: ${results.data.markdown.length}`);

curl 'https://sdk.nimbleway.com/v1/tasks/{task_id}/results' \
--header 'Authorization: Bearer YOUR-API-KEY'

results = nimble.tasks.results(task_id)

parsed = results.data.parsing["parsed"]
print(f"Product: {parsed['product_title']}")
print(f"Price: ${parsed['web_price']}")

const results = await nimble.tasks.results(taskId);

const parsed = results.data.parsing.parsed;
console.log(`Product: ${parsed.product_title}`);
console.log(`Price: $${parsed.web_price}`);

curl 'https://sdk.nimbleway.com/v1/tasks/{task_id}/results' \
--header 'Authorization: Bearer YOUR-API-KEY'

Fetch results for each completed task in the crawl.

# my_crawl["tasks"] from step #2 contains list of task IDs from status response
for task in my_crawl["tasks"]:
    if task.state == "success":
        task_result = nimble.tasks.results(task.id)

        print(f"URL: {task_result.url}")
        print(f"HTML length: {len(task_result.data.get('html', ''))}")

// batch.tasks from step #2 contains task objects with id and state
for (const task of myCrawl.tasks) {
  if (task.state === "success") {
    const results = await nimble.tasks.results(task.id);

    console.log(`URL: ${results.url}`);
    console.log(`HTML length: ${results.data?.html?.length || 0}`);
  }
}

# Get results for each task_id from the crawl status response
curl 'https://sdk.nimbleway.com/v1/tasks/{task_id}/results' \
--header 'Authorization: Bearer YOUR-API-KEY'

Once the batch is complete, retrieve results for each successful task using the task IDs from GET /v1/batches/{batch_id}.

# batch.tasks from step #2 — iterate and fetch each result
for task in batch.tasks:
    if task.state == "success":
        result = nimble.tasks.results(task.id)
        print(f"Markdown: {result.data.markdown[:200]}")
    elif task.state == "error":
        print(f"Task {task.id} failed: {task.error}")

// batch.tasks from step #2 — iterate and fetch each result
for (const task of batch.tasks) {
  if (task.state === "success") {
    const result = await nimble.tasks.results(task.id);
    console.log(`Markdown: ${result.data.markdown.substring(0, 200)}`);
  } else if (task.state === "error") {
    console.log(`Task ${task.id} failed: ${task.error}`);
  }
}

# Fetch results for each task_id from the batch details response
curl 'https://sdk.nimbleway.com/v1/tasks/{task_id}/results' \
--header 'Authorization: Bearer YOUR-API-KEY'

Example Results

{
    "url": "https://www.nimbleway.com/blog/post",
    "task_id": "ec89b1f7-1cf2-40eb-91b4-78716093f9ed",
    "status": "success",
    "task": {
        "id": "ec89b1f7-1cf2-40eb-91b4-78716093f9ed",
        "state": "success",
        "created_at": "2026-02-09T23:15:43.549Z",
        "modified_at": "2026-02-09T23:16:39.094Z",
        "account_name": "your-account"
    },
    "data": {
        "html": "<!DOCTYPE html>...",
        "markdown": "# Page Title\n\nContent...",
        "headers": { ... }
    },
    "metadata": {
        "query_time": "2026-02-09T23:15:43.549Z",
        "query_duration": 1877,
        "response_parameters": {
            "input_url": "https://www.nimbleway.com/blog/post"
        },
		"driver": "vx6"
    },
    "status_code": 200
}

Polling endpoints reference

API	Submit	Check Status	Get Results
Extract	`POST /v1/extract/async`	`GET /v1/tasks/{task_id}`	`GET /v1/tasks/{task_id}/results`
Agent	`POST /v1/agent/async`	`GET /v1/tasks/{task_id}`	`GET /v1/tasks/{task_id}/results`
Batch	`POST /v1/extract/batch`	`GET /v1/batches/{batch_id}/progress`	`GET /v1/tasks/{task_id}/results` (per task)
Crawl	`POST /v1/crawl`	`GET /v1/crawl/{crawl_id}`	`GET /v1/tasks/{task_id}/results` (per page)

To list all tasks across your account, use GET /v1/tasks (supports cursor and limit for pagination). To list all batches, use GET /v1/batches.

Option 2: Webhooks (Push)

Get notified automatically when your tasks complete. Perfect for event-driven architectures.

Submit request with callback URL

Include callback_url (or callback object for crawl) in your async request.

Extract
Agent
Crawl

from nimble_python import Nimble

nimble = Nimble(api_key="YOUR-API-KEY")

response = nimble.extract_async(
    url="https://www.nimbleway.com",
    render=True,
    formats=["html", "markdown"],
    callback_url="https://your-server.com/webhooks/nimble"
)

task_id = response.task_id
print(f"Task submitted: {task_id}")
print("Results will be POSTed to your callback URL when ready")

import Nimble from '@nimble-way/nimble-js';

const nimble = new Nimble({ apiKey: "YOUR-API-KEY" });

const response = await nimble.extractAsync({
  url: "https://www.nimbleway.com",
  render: true,
  formats: ["html", "markdown"],
  callback_url: "https://your-server.com/webhooks/nimble"
});

const taskId = response.task_id;
console.log(`Task submitted: ${taskId}`);

curl -X POST 'https://sdk.nimbleway.com/v1/extract/async' \
--header 'Authorization: Bearer YOUR-API-KEY' \
--header 'Content-Type: application/json' \
--data-raw '{
    "url": "https://www.nimbleway.com",
    "render": true,
    "formats": ["html", "markdown"],
    "callback_url": "https://your-server.com/webhooks/nimble"
}'

from nimble_python import Nimble

nimble = Nimble(api_key="YOUR-API-KEY")

response = nimble.extract_async(
    url="https://www.nimbleway.com",
    render=True,
    formats=["html", "markdown"],
    callback_url="https://your-server.com/webhooks/nimble"
)

task_id = response.task_id
print(f"Task submitted: {task_id}")
print("Results will be POSTed to your callback URL when ready")

import Nimble from '@nimble-way/nimble-js';

const nimble = new Nimble({ apiKey: "YOUR-API-KEY" });

const response = await nimble.agent.runAsync({
  agent: "amazon_pdp",
  params: {
		asin: "B0DLKFK6LR",
		callback_url: "https://your-server.com/webhooks/nimble"
  },
});

const taskId = response.task_id;
console.log(`Task submitted: ${taskId}`);

  curl -X POST 'https://sdk.nimbleway.com/v1/extract/async' \
--header 'Authorization: Bearer YOUR-API-KEY' \
--header 'Content-Type: application/json' \
--data-raw '{
    "url": "https://www.nimbleway.com",
    "render": true,
    "formats": ["html", "markdown"],
    "callback_url": "https://your-server.com/webhooks/nimble"
}'

Crawl uses a callback object for advanced webhook options.

from nimble_python import Nimble

nimble = Nimble(api_key="YOUR-API-KEY")

result = nimble.crawl.run(
    url="https://www.nimbleway.com",
    limit=100,
    callback={
        "url": "https://your-server.com/webhooks/nimble",
        "headers": {
            "X-Custom-Auth": "your-secret-token"
        },
        "events": ["completed", "failed"]
    }
)

print(f"Crawl started: {result.crawl_id}")

import Nimble from '@nimble-way/nimble-js';

const nimble = new Nimble({ apiKey: "YOUR-API-KEY" });

const result = await nimble.crawl.run({
  url: "https://www.nimbleway.com",
  limit: 100,
  callback: {
    url: "https://your-server.com/webhooks/nimble",
    headers: {
      "X-Custom-Auth": "your-secret-token"
    },
    events: ["completed", "failed"]
  }
});

console.log(`Crawl started: ${result.crawl_id}`);

curl -X POST 'https://sdk.nimbleway.com/v1/crawl' \
--header 'Authorization: Bearer YOUR-API-KEY' \
--header 'Content-Type: application/json' \
--data-raw '{
    "url": "https://www.nimbleway.com",
    "limit": 100,
    "callback": {
        "url": "https://your-server.com/webhooks/nimble",
        "headers": { "X-Custom-Auth": "your-secret-token" },
        "events": ["completed", "failed"]
    }
}'

Receive webhook notification

Nimble sends a POST to your callback URL when complete:

{
  "task": {
    "id": "8e8cfde8-345b-42b8-b3e2-0c61eb11e00f",
    "state": "success",
    "status_code": 200,
    "created_at": "2026-01-24T12:36:24.685Z",
    "modified_at": "2026-01-24T12:36:24.685Z",
    "input": {},
    "api_type": "extract"
  }
}

Webhook configuration options

API	Parameter	Type	Description
Extract	`callback_url`	string	Your callback URL
Agent	`params.callback_url`	string	Your callback URL
Crawl	`callback.url`	string	Your callback URL
	`callback.headers`	object	Custom headers for authentication
	`callback.metadata`	object	Custom data included in payload
	`callback.events`	array	Filter events: `started`, `page`, `completed`, `failed`

Option 3: Cloud Delivery (Async API)

Applies to async API requests - /extract/async, /extract/batch, /agent/async, /crawl. For connecting a Job to your storage, see Job Connections.

Automatically deliver results directly to your cloud storage bucket.

Amazon S3

Deliver to any S3 bucket in your AWS account

Google Cloud Storage

Deliver to any GCS bucket in your GCP project

Configure bucket permissions (one-time)

Grant Nimble’s service account write access to your bucket.

Amazon S3
Google Cloud Storage

Nimble Service User ARN:

arn:aws:iam::744254827463:user/webit-uploader

Add this bucket policy:

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "NimbleCloudDelivery",
      "Effect": "Allow",
      "Principal": {
        "AWS": "arn:aws:iam::744254827463:user/webit-uploader"
      },
      "Action": [
        "s3:PutObject",
        "s3:PutObjectACL",
        "s3:GetBucketLocation"
      ],
      "Resource": [
        "arn:aws:s3:::YOUR_BUCKET_NAME",
        "arn:aws:s3:::YOUR_BUCKET_NAME/*"
      ]
    }
  ]
}

Replace YOUR_BUCKET_NAME with your actual bucket name.

KMS-Encrypted Buckets

For KMS-encrypted buckets, add this to your KMS key policy:

{
  "Sid": "NimbleKMSAccess",
  "Effect": "Allow",
  "Principal": {
    "AWS": "arn:aws:iam::744254827463:user/webit-uploader"
  },
  "Action": [
    "kms:Encrypt",
    "kms:Decrypt",
    "kms:ReEncrypt*",
    "kms:GenerateDataKey*",
    "kms:DescribeKey"
  ],
  "Resource": "*"
}

Nimble Service Account:

[email protected]

Navigate to your bucket in the Google Cloud Console
Click on the Permissions tab
Click Grant Access
Add principal: [email protected]
Assign role: Storage Object Creator
Click Save

Submit request with storage config

Include storage_type and storage_url in your request.

Cloud delivery parameters

Parameter	Type	Description
`storage_type`	`s3` \| `gs`	Cloud provider
`storage_url`	string	Bucket path with prefix (e.g., `s3://bucket/prefix/`)
`storage_compress`	boolean	Enable GZIP compression
`storage_object_name`	string	Custom filename (default: task ID)

Extract
Agent

from nimble_python import Nimble

nimble = Nimble(api_key="YOUR-API-KEY")

response = nimble.extract_async(
    url="https://www.nimbleway.com",
    render=True,
    formats=["html", "markdown"],
    storage_type="s3",
    storage_url="s3://your-bucket/nimble-results/",
    storage_compress=True,
    storage_object_name="my-result"
)

task_id = response.task_id
print(f"Results will be saved to: s3://your-bucket/nimble-results/my-result.json.gz")

import Nimble from '@nimble-way/nimble-js';

const nimble = new Nimble({ apiKey: "YOUR-API-KEY" });

const response = await nimble.extractAsync({
  url: "https://www.nimbleway.com",
  render: true,
  formats: ["html", "markdown"],
  storage_type: "s3",
  storage_url: "s3://your-bucket/nimble-results/",
  storage_compress: true,
  storage_object_name: "my-result"
});

const taskId = response.task_id;
console.log(`Results will be saved to: s3://your-bucket/nimble-results/my-result.json.gz`);

curl -X POST 'https://sdk.nimbleway.com/v1/extract/async' \
--header 'Authorization: Bearer YOUR-API-KEY' \
--header 'Content-Type: application/json' \
--data-raw '{
    "url": "https://www.nimbleway.com",
    "render": true,
    "formats": ["html", "markdown"],
    "storage_type": "s3",
    "storage_url": "s3://your-bucket/nimble-results/",
    "storage_compress": true,
    "storage_object_name": "my-result"
}'

from nimble_python import Nimble

nimble = Nimble(api_key="YOUR-API-KEY")

response = nimble.extract_async(
    url="https://www.nimbleway.com",
    render=True,
    formats=["html", "markdown"],
    storage_type="gs",
    storage_url="gs://your-bucket/nimble-results/",
    storage_object_name="my-result"
)

task_id = response.task_id
print(f"Results will be saved to: gs://your-bucket/nimble-results/my-result.json")

import Nimble from '@nimble-way/nimble-js';

const nimble = new Nimble({ apiKey: "YOUR-API-KEY" });

const response = await nimble.extractAsync({
  url: "https://www.nimbleway.com",
  render: true,
  formats: ["html", "markdown"],
  storage_type: "gs",
  storage_url: "gs://your-bucket/nimble-results/",
  storage_object_name: "my-result"
});

const taskId = response.task_id;
console.log(`Results will be saved to: gs://your-bucket/nimble-results/my-result.json`);

curl -X POST 'https://sdk.nimbleway.com/v1/extract/async' \
--header 'Authorization: Bearer YOUR-API-KEY' \
--header 'Content-Type: application/json' \
--data-raw '{
    "url": "https://www.nimbleway.com",
    "render": true,
    "formats": ["html", "markdown"],
    "storage_type": "gs",
    "storage_url": "gs://your-bucket/nimble-results/",
    "storage_object_name": "my-result"
}'

from nimble_python import Nimble

nimble = Nimble(api_key="YOUR-API-KEY")

response = nimble.agent.run_async(
    agent="amazon_pdp",
    params={
        "asin": "B0DLKFK6LR",
        "storage_type": "s3",
        "storage_url": "s3://your-bucket/nimble-results/",
        "storage_compress": True,
        "storage_object_name": "my-result"
    }
)

task_id = response.task_id
print(f"Results will be saved to: s3://your-bucket/nimble-results/my-result.json.gz")

import Nimble from '@nimble-way/nimble-js';

const nimble = new Nimble({ apiKey: "YOUR-API-KEY" });

const response = await nimble.agent.runAsync({
  agent: "amazon_pdp",
  params: {
    asin: "B0DLKFK6LR",
    storage_type: "s3",
    storage_url: "s3://your-bucket/nimble-results/",
    storage_compress: true,
    storage_object_name: "my-result"
  }
});

const taskId = response.task_id;
console.log(`Results will be saved to: s3://your-bucket/nimble-results/my-result.json.gz`);

curl -X POST 'https://sdk.nimbleway.com/v1/agent/async' \
--header 'Authorization: Bearer YOUR-API-KEY' \
--header 'Content-Type: application/json' \
--data-raw '{
    "agent": "amazon_pdp",
    "params": {
        "asin": "B0DLKFK6LR",
        "storage_type": "s3",
        "storage_url": "s3://your-bucket/nimble-results/",
        "storage_compress": true,
        "storage_object_name": "my-result"
    }
}'

from nimble_python import Nimble

nimble = Nimble(api_key="YOUR-API-KEY")

response = nimble.agent.run_async(
    agent="amazon_pdp",
    params={
        "asin": "B0DLKFK6LR",
        "storage_type": "gs",
        "storage_url": "gs://your-bucket/nimble-results/",
        "storage_object_name": "my-result"
    }
)

task_id = response.task_id
print(f"Results will be saved to: gs://your-bucket/nimble-results/my-result.json")

import Nimble from '@nimble-way/nimble-js';

const nimble = new Nimble({ apiKey: "YOUR-API-KEY" });

const response = await nimble.agent.runAsync({
  agent: "amazon_pdp",
  params: {
    asin: "B0DLKFK6LR",
    storage_type: "gs",
    storage_url: "gs://your-bucket/nimble-results/",
    storage_object_name: "my-result"
  }
});

const taskId = response.task_id;
console.log(`Results will be saved to: gs://your-bucket/nimble-results/my-result.json`);

curl -X POST 'https://sdk.nimbleway.com/v1/agent/async' \
--header 'Authorization: Bearer YOUR-API-KEY' \
--header 'Content-Type: application/json' \
--data-raw '{
    "agent": "amazon_pdp",
    "params": {
        "asin": "B0DLKFK6LR",
        "storage_type": "gs",
        "storage_url": "gs://your-bucket/nimble-results/",
        "storage_object_name": "my-result"
    }
}'

Results delivered automatically

When complete, results are written to your bucket as {task_id}.json (or .json.gz if compressed).

Comparison

Feature	Polling	Webhooks	Cloud Delivery
Setup complexity	None	Requires endpoint	Requires bucket setup
Real-time notifications	No (you poll)	Yes	No
Automatic storage	No	No	Yes
Best for	Simple integrations, testing	Event-driven apps	Data pipelines ETLs
Infrastructure needed	None	Web server	Cloud storage bucket

Combining methods

You can combine delivery methods for redundancy:

from nimble_python import Nimble

nimble = Nimble(api_key="YOUR-API-KEY")

# Receive webhook AND store in S3
response = nimble.extract_async(
    url="https://www.nimbleway.com",
    formats=["html", "markdown"],
    callback_url="https://your-server.com/webhooks/nimble",
    storage_type="s3",
    storage_url="s3://your-bucket/results/"
)

import Nimble from "@nimble-way/nimble-js";

const nimble = new Nimble({ apiKey: "YOUR-API-KEY" });

// Receive webhook AND store in S3
const response = await nimble.extractAsync({
  url: "https://www.nimbleway.com",
  formats: ["html", "markdown"],
  callback_url: "https://your-server.com/webhooks/nimble",
  storage_type: "s3",
  storage_url: "s3://your-bucket/results/",
});

curl -X POST 'https://sdk.nimbleway.com/v1/extract/async' \
--header 'Authorization: Bearer YOUR-API-KEY' \
--header 'Content-Type: application/json' \
--data-raw '{
    "url": "https://www.nimbleway.com",
    "formats": ["html", "markdown"],
    "callback_url": "https://your-server.com/webhooks/nimble",
    "storage_type": "s3",
    "storage_url": "s3://your-bucket/results/"
}'

Best Practices

Polling

Check status first - Use /tasks/{id} before fetching full results - Use reasonable intervals - Poll every 2-5 seconds, not continuously - Handle rate limits - Implement retry logic for 429 responses - Set timeouts - Most tasks complete within seconds to minutes

Webhooks

Use HTTPS - Always use secure endpoints - Verify authenticity - Use custom headers for authentication - Respond quickly - Return 200 OK immediately, process async - Handle retries - Nimble retries failed deliveries

Cloud Delivery

Use prefixes - Organize by date, project, or type - Enable compression - Use storage_compress: true for large files - Set lifecycle policies - Auto-delete old files to manage costs - Use custom names - storage_object_name for meaningful filenames

Next Steps

Async Extract

Learn about async extraction options

Crawl API

Deep website crawling with async delivery

Agent Gallery

Browse available search agents

Rate Limits

Understand API rate limit

Introduction

Web Search Agents

Web Tools

Guides

SDKs

Admin

Polling

Callbacks

Cloud Delivery

Option 1: Polling (Pull)

Submit async request

Check status

Retrieve results

Polling endpoints reference

Option 2: Webhooks (Push)

Submit request with callback URL

Receive webhook notification

Webhook configuration options

Option 3: Cloud Delivery (Async API)

Amazon S3

Google Cloud Storage

Configure bucket permissions (one-time)

Submit request with storage config

Cloud delivery parameters

Results delivered automatically

Comparison

Combining methods

Best Practices

Next Steps

Async Extract

Crawl API

Agent Gallery

Rate Limits

Polling

Callbacks

Cloud Delivery

​Option 1: Polling (Pull)

Submit async request

Check status

Retrieve results

​Polling endpoints reference

​Option 2: Webhooks (Push)

Submit request with callback URL

Receive webhook notification

​Webhook configuration options

​Option 3: Cloud Delivery (Async API)

Amazon S3

Google Cloud Storage

Configure bucket permissions (one-time)

Submit request with storage config

​Cloud delivery parameters

Results delivered automatically

​Comparison

​Combining methods

​Best Practices

​Next Steps

Async Extract

Crawl API

Agent Gallery

Rate Limits

Option 1: Polling (Pull)

Polling endpoints reference

Option 2: Webhooks (Push)

Webhook configuration options

Option 3: Cloud Delivery (Async API)

Cloud delivery parameters

Comparison

Combining methods

Best Practices

Next Steps