Advanced APIs

Map & Crawl

Map discovers the URLs on a site and classifies them. Crawl reads many pages and returns their content.

Map

Map explores a site from a seed URL and returns discovered links with a page type and priority score. It responds synchronously when limit is 50 or fewer, otherwise it returns a job to poll.

POST/map

Request

curl https://api.ohsee.tech/map \
  -H "Authorization: Bearer ohsee-<your-key>" \
  -H "Content-Type: application/json" \
  -d '{ "url": "https://example.com", "limit": 50 }'

200 OK (limit <= 50)

{
  "request_id": "1a2b3c4d",
  "status": "succeeded",
  "credits_charged": 5,
  "response_time": 2.30,
  "url": "https://example.com",
  "urls": [
    { "url": "https://example.com/pricing", "title": "Pricing", "path": "/pricing", "page_type": "pricing", "priority_score": 0.9 }
  ],
  "count": 1
}

Map parameters

Field	Type	Default	Description
`url`required	string	-	Seed URL to map from.
`limit`	int	50	Maximum URLs to discover (1 to 500).
`max_depth`	int	2	How many link levels to follow (0 to 5).
`include_patterns`	string[]	[]	Only keep URLs matching these regex patterns.
`exclude_patterns`	string[]	[]	Drop URLs matching these regex patterns.

Crawl

Crawl reads many pages starting from a seed URL and returns cleaned content per page. It is always asynchronous and returns a job_id. Pass a schema to structure each page.

POST/crawl

Request

curl https://api.ohsee.tech/crawl \
  -H "Authorization: Bearer ohsee-<your-key>" \
  -H "Content-Type: application/json" \
  -d '{ "url": "https://example.com/docs", "limit": 20, "format": "markdown" }'

GET /jobs/{id}

{
  "job_id": "c1d2e3f4-...",
  "kind": "crawl",
  "status": "succeeded",
  "credits": { "reserved": 20, "charged": 12, "refunded": 8 },
  "result": {
    "url": "https://example.com/docs",
    "pages_extracted": 12,
    "failed_pages": [],
    "pages": [{ "url": "https://example.com/docs/intro", "content": "# Intro ..." }]
  }
}

Field	Type	Description
`result.pages_extracted`	int	Number of pages successfully read.
`result.failed_pages`	object[]	Pages that failed, with an error.
`result.pages`	object[]	Per page: `url`, `content`, and `structured` when a schema is used.

Crawl parameters

Field	Type	Default	Description
`url`required	string	-	Seed URL to crawl from.
`limit`	int	20	Maximum pages to read (1 to 200).
`max_depth`	int	2	How many link levels to follow (0 to 5).
`format`	string	markdown	Page format: `markdown` or `text`.
`schema`	object	null	Structure each page against this schema.
`include_patterns`	string[]	[]	Only crawl URLs matching these patterns.
`exclude_patterns`	string[]	[]	Skip URLs matching these patterns.

Credits

Map costs 1 credit per 10 URLs found (rounded up). Crawl costs 1 credit per 5 pages (rounded up), or 1 credit per page when a schema is provided. You are charged for pages actually read.

Use small limits first and /estimate to size larger crawls.

Research

Domain intelligence