Skip to main content

​
Overview

Map is like having a bird’s-eye view of any website. Give it a URL, and it automatically discovers and lists related URLs on that site with helpful context like titles and descriptions. This context enables AI agents to intelligently decide which pages to scrape, making data collection smarter and more efficient. Think of it as an instant site explorer that answers: “What pages exist, and what’s on them?”

​
How it works

1

You provide a starting URL

Give Map any website URL - homepage, subdomain, or specific page
2

Map discovers all pages

Automatically finds URLs from sitemaps and by scanning internal links on the site
3

Returns complete URL list

Get a comprehensive list of every discoverable page, ready to use for crawling or analysis

​
When to use Map

AI Agent Planning

Give AI agents context about available URLs with titles and descriptions to intelligently decide which pages to scrape

Smart Data Collection

Discover all pages with metadata before scraping, enabling targeted extraction from only relevant URLs

Competitive Research

See what pages competitors have - product listings, blog posts, landing pages, and more

Site Audits

Quickly inventory all pages for SEO audits, quality checks, or content analysis

​
Real-world examples

Scenario: Your AI agent needs to extract product data from an e-commerce site but should only scrape relevant product pages.How Map helps:
  • Discovers all URLs with titles and descriptions
  • AI agent reads the context (title: “Laptop Pro 15”, description: “High-performance laptop…”)
  • Agent intelligently decides which URLs contain product data vs navigation/legal pages
  • Only scrapes relevant product URLs, saving costs and time
Result: AI-powered smart scraping that automatically filters out irrelevant pages based on URL context.
Scenario: You want to understand a competitor’s product catalog structure.How Map helps:
  • Discover all product category pages
  • Find hidden product pages not linked from the homepage
  • Identify promotional landing pages
  • Map out the entire site structure before detailed scraping
Result: Complete visibility into their online catalog without manual browsing.
Scenario: You’re tracking content strategy across multiple news sites.How Map helps:
  • Get a complete list of articles and sections
  • Discover new content categories as they’re added
  • Identify archive structures and content organization
  • Plan targeted crawling for specific content types
Result: Systematic content discovery at scale.
Scenario: Analyzing website architecture for SEO optimization.How Map helps:
  • Visualize complete site hierarchy
  • Identify orphaned pages (pages not well-linked)
  • Discover deep pages buried in site structure
  • Understand internal linking patterns
Result: Comprehensive site architecture insights in seconds.
Scenario: You need to scrape data from a large website efficiently.How Map helps:
  • Get complete URL inventory before crawling
  • Filter for specific sections or page types
  • Plan crawl budget and prioritization
  • Avoid wasting resources on irrelevant pages
Result: Smarter, more efficient data collection.

​
What you get

When you use Map, you receive:
  • URLs with context - Each URL includes optional title and description for AI reasoning
  • Complete discovery - Every discoverable page on the site
  • Fast results - Most sites mapped in seconds
  • Smart filtering - Subdomain control and comprehensive sitemap + link discovery

​
Map vs. other tools

What you needUse this
URLs with context for AI planningMap
Data from popular sites (Amazon, Google, etc.)Public Agent - maintained by Nimble
Data from sites not in the galleryCustom Agent - create with natural language
Data from specific URLs (expert users)Extract
Search web + extract content from resultsSearch
Data from entire websiteCrawl
Map is the starting point - Use it to discover URLs with context, enabling AI agents to intelligently decide which pages to scrape with Crawl or Extract.

​
Key capabilities

Lightning fast

Most websites mapped in under 10 seconds

No maintenance

Works on any website without custom configuration

Highly accurate

Combines multiple discovery methods for comprehensive results

​
Simple example

Input: Amazon product URL with template 
{
	"url": "http://www.example-store.com"
}
**Output: **List of related URLs 
[
    {
      "url": "https://example.com/page1",
      "title": "Example Page 1",
      "description": "This is an example description for page 1."
    },
    {
      "url": "https://example.com/page2"
    },
    {
      "url": "https://example.com/page3"
    },
    {
      "url": "https://example.com/page3/subpage1"
    },
    ... (and so on)
  ]
No clicking, no browsing - just instant visibility into the entire site structure.

​
Next steps

Quick Start

Get started with code examples and basic usage

API Reference

Explore all parameters, options, and advanced features