Website Crawler

Map site URLs, scrape richer page data, or crawl a bounded website section with Firecrawl through NanoGPT billing.

Operation

URL

Limit

Sitemap

Search filter

Advanced JSON override

Request

{
  "operation": "map",
  "url": "https://docs.firecrawl.dev",
  "sitemap": "include",
  "limit": 10
}

API

curl -sS \
  -X POST \
  -H "Authorization: Bearer $NANOGPT_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{"operation":"map","url":"https://docs.firecrawl.dev","sitemap":"include","limit":10}' \
  "https://nano-gpt.com/api/v1/firecrawl"

Website Crawler for web and API workflows

Map sites, scrape pages, and crawl bounded website sections with Firecrawl through NanoGPT. The same workflow is available from the web app and the /api/v1/firecrawl API endpoint.

Common uses

Build site maps for research and migrations
Extract clean Markdown or HTML from a known URL
Crawl bounded documentation, help, or product sections

Data returned

Discovered URLs
Markdown, HTML, links, images, screenshots, and summaries
Page metadata and crawl status details