Data Enrichment

-- Web search → rows of (title, description, url, content)
SELECT title, url
FROM   nimble_integration.tools.nimble_search('AI agent frameworks news', 5, 'news');

-- Enrich a table of URLs and persist as a governed Delta table
CREATE TABLE my_catalog.web.page_text AS
SELECT u.id, x.url, x.content
FROM   my_catalog.web.urls u,
       LATERAL nimble_integration.tools.nimble_extract(u.url) x;

​

What you get

• Search and extract as first-class SQL. nimble_search(query, …) returns rows of title, description, url, content; nimble_extract(url, …) returns a row of url, format, content (markdown / html / links). Both hit the live web through Nimble at query time.
• Results land as governed Delta tables. Wrap any call in CREATE TABLE … AS and the output is a managed Unity Catalog table — Delta time travel, column/table lineage, and GRANT-based access control come for free.
• Every Web Search Agent, one function. nimble_agent_run(agent, params_json) runs any agent (google_maps_search, amazon_serp, linkedin_company_details, …) and returns the parsed payload as a navigable VARIANT. nimble_agent_list() enumerates the catalog.
• Per-row error isolation. Each tool yields zero rows on failure instead of raising, so a batch CREATE TABLE … AS over thousands of inputs is never aborted by one bad row.
• Genie-native. Databricks Genie registers table functions as tools, so each function is directly registrable as a Genie agent tool — the function and column COMMENTs are the spec the LLM reads.
• Stays inside your Databricks account. Only authorized outbound calls hit Nimble. The API key lives in a Databricks secret scope and is injected server-side — it never appears in a function signature or at a call site.

​

Prerequisites

• A Databricks workspace with a serverless SQL warehouse.
• Privilege to create a catalog/schema (or an existing schema you own) plus CREATE FUNCTION on it.
• The Databricks CLI v0.205+, authenticated (databricks auth login).
• A Nimble API key (sign up free).

​

Install in Databricks

databricks secrets create-scope nimble
databricks secrets put-secret  nimble api_key   # paste the raw token, then Ctrl-D
databricks secrets put-acl     nimble users READ

CREATE CATALOG IF NOT EXISTS nimble_integration
  COMMENT 'Nimble web-data integration: UC table functions wrapping Nimble APIs and agents.';

CREATE SCHEMA IF NOT EXISTS nimble_integration.tools
  COMMENT 'Table functions wrapping Nimble APIs / agents. Callable from Genie, agents, dashboards.';

CREATE SCHEMA IF NOT EXISTS nimble_integration.recipes
  COMMENT 'Sample inputs and end-to-end recipe outputs (Delta tables) built on the tools schema.';

WH=<your-serverless-warehouse-id>   # databricks warehouses list

python3 databricks/helpers/deploy_sql.py --file databricks/01_setup.sql --warehouse "$WH"
for f in databricks/tools/*.sql; do
    python3 databricks/helpers/deploy_sql.py --file "$f" --warehouse "$WH"
done

-- Web search → expect > 0 rows
SELECT count(*) AS n
FROM   nimble_integration.tools.nimble_search('AI agents news', 5);

-- Page extraction → expect a large character count
SELECT length(content) AS md_len
FROM   nimble_integration.tools.nimble_extract('https://www.nimbleway.com');

​

The tools

​

Common workflows

​

Land web data as a governed Delta table

CREATE TABLE my_catalog.web.company_pages AS
SELECT c.company_id,
       c.homepage,
       x.content            AS homepage_markdown,
       current_timestamp()  AS fetched_at
FROM   my_catalog.crm.companies c,
       LATERAL nimble_integration.tools.nimble_extract(c.homepage) x;

​

Local business discovery at scale

CREATE OR REPLACE TABLE nimble_integration.recipes.local_businesses AS
SELECT
    q.query,
    q.category,
    biz:position::int                          AS position,
    biz:title::string                          AS name,
    biz:business_category[0]::string           AS business_category,
    biz:street_address::string                 AS street_address,
    biz:city::string                           AS city,
    biz:review_summary:overall_rating::double  AS rating,
    coalesce(biz:review_summary:review_count::int,
             biz:number_of_reviews::int)       AS review_count,
    biz:price_level::string                    AS price_level,
    biz:place_information:website_url::string  AS website,
    biz:business_status::string                AS business_status,
    biz:sponsored::boolean                     AS sponsored,
    current_timestamp()                        AS enriched_at
FROM nimble_integration.recipes.location_queries q,
     LATERAL nimble_integration.tools.nimble_agent_run(
         'google_maps_search',
         to_json(named_struct('query', q.query))
     ) a,
     LATERAL variant_explode(a.parsing:entities:SearchResult) AS e(pos, key, biz)
WHERE a.status = 'success';

​

Structured per-row enrichment with any agent

SELECT
    p.sku,
    a.parsing:brand::string                    AS brand,
    a.parsing:product_title::string            AS title,
    a.parsing:web_price::decimal(10, 2)        AS price,
    a.parsing:availability::boolean            AS in_stock,
    a.parsing:average_of_reviews::double       AS rating
FROM   my_catalog.crm.products p,
       LATERAL nimble_integration.tools.nimble_agent_run(
           'amazon_pdp',
           to_json(named_struct('asin', p.amazon_asin))
       ) a
WHERE  p.amazon_asin IS NOT NULL
  AND  a.status = 'success';

​

Schedule recurring enrichment

​

Fallback: http_request() without the networking preview

CREATE CONNECTION IF NOT EXISTS nimble_api TYPE HTTP
OPTIONS (
    host         'https://sdk.nimbleway.com',
    port         '443',
    base_path    '/',
    bearer_token secret('nimble', 'api_key')
);

​

Roll out across your organization

• Tune for your Nimble rate-limit tier. Each tool call is one upstream request — one per input row. For high-cardinality inputs (thousands of rows), watch for 429s; because failures yield zero rows, a single retry pass over the rows that produced no output is the typical recovery. Nimble’s rate-limits page lists per-tier ceilings.
• Right-size the warehouse. A small serverless warehouse handles daily refreshes over a few hundred input rows. For larger enrichments (10K+ rows), step up the size only for the duration of the scheduled job.
• Surface failures separately. Because tools yield zero rows on failure, an input row with no matching output row is a failed call — left-join the input table against the result to find the gaps and audit them, rather than grepping job logs.
• Don’t ship guesses. Field names inside parsing differ per agent. Inspect the shape once per agent (SELECT parsing FROM nimble_agent_run(<agent>, …)) before pinning a projection into a dbt model or production view.

​

Use it from Databricks Genie

​

Resources