CrawlHawk API

Crawl any website, extract products, contacts or clean markdown — from your own code. Job-based and asynchronous: start a job, poll or receive a webhook, download the results.

Authentication

Every request carries your API key as a Bearer token. Create keys on your Account → API page.

curl https://api.crawlhawk.com/v1/account \
  -H "Authorization: Bearer chk_live_…"

POST/v1/estimate

Returns the per-unit rate, whether the site needs the protected tier, and a page estimate read from the sitemap when there is one. Starts nothing and spends no credits.

Body

curl -X POST https://api.crawlhawk.com/v1/estimate \
  -H "Authorization: Bearer chk_live_…" \
  -H "Content-Type: application/json" \
  -d '{"url":"https://example.com","mode":"links"}'

Response 200

{
  "url": "string",
  "mode": "links",
  "credit_per_unit": 0,
  "credit_per_1000": 0,
  "protected": false,
  "requires_protected_tier": false,
  "protected_marker": "string",
  "estimated_pages": 0,
  "estimated_credits": 0,
  "credits_available": 0
}

GET/v1/jobs

List the account's runs

curl https://api.crawlhawk.com/v1/jobs \
  -H "Authorization: Bearer chk_live_…"

Response 200

{
  "items": [
    {
      "job_id": "string",
      "mode": "links",
      "status": "queued",
      "url": "string",
      "counts": {},
      "credits_used": 0,
      "credit_per_unit": 0,
      "protected": false,
      "batch": 0,
      "batch_ready": false,
      "next_batch": 0,
      "external": {},
      "source": "string",
      "created_at": "2026-08-21T10:00:00Z",
      "finished_at": "2026-08-21T10:00:00Z",
      "error": {}
    }
  ],
  "total": 0,
  "page": 0,
  "page_size": 0
}

POST/v1/jobs

Start a run

Parameters

Body

curl -X POST https://api.crawlhawk.com/v1/jobs \
  -H "Authorization: Bearer chk_live_…" \
  -H "Content-Type: application/json" \
  -d '{"mode":"links"}'

Response 202

{
  "job_id": "string",
  "status": "queued",
  "mode": "links",
  "url": "string",
  "created_at": "2026-08-21T10:00:00Z"
}

GET/v1/jobs/{id}

Read a run’s status

Parameters

curl https://api.crawlhawk.com/v1/jobs/583d8118-… \
  -H "Authorization: Bearer chk_live_…"

Response 200

{
  "job_id": "string",
  "mode": "links",
  "status": "queued",
  "url": "string",
  "counts": {
    "pages": 0,
    "products": 0,
    "results": 0
  },
  "credits_used": 0,
  "credit_per_unit": 0,
  "protected": false,
  "batch": 0,
  "batch_ready": false,
  "next_batch": 0,
  "external": {},
  "source": "string",
  "created_at": "2026-08-21T10:00:00Z",
  "finished_at": "2026-08-21T10:00:00Z",
  "error": {}
}

POST/v1/jobs/{id}/confirm

Resolves both machine-side stops: a protected-tier escalation waiting on `awaiting_confirmation`, and a run parked on a batch boundary (`{"continue_batch": true}`).

Parameters

Body

curl -X POST https://api.crawlhawk.com/v1/jobs/583d8118-…/confirm \
  -H "Authorization: Bearer chk_live_…"

Response 200

{
  "job_id": "string",
  "mode": "links",
  "status": "queued",
  "url": "string",
  "counts": {
    "pages": 0,
    "products": 0,
    "results": 0
  },
  "credits_used": 0,
  "credit_per_unit": 0,
  "protected": false,
  "batch": 0,
  "batch_ready": false,
  "next_batch": 0,
  "external": {},
  "source": "string",
  "created_at": "2026-08-21T10:00:00Z",
  "finished_at": "2026-08-21T10:00:00Z",
  "error": {}
}

POST/v1/jobs/{id}/abort

Stop a run

Parameters

curl -X POST https://api.crawlhawk.com/v1/jobs/583d8118-…/abort \
  -H "Authorization: Bearer chk_live_…"

Response 200

{
  "job_id": "string",
  "mode": "links",
  "status": "queued",
  "url": "string",
  "counts": {
    "pages": 0,
    "products": 0,
    "results": 0
  },
  "credits_used": 0,
  "credit_per_unit": 0,
  "protected": false,
  "batch": 0,
  "batch_ready": false,
  "next_batch": 0,
  "external": {},
  "source": "string",
  "created_at": "2026-08-21T10:00:00Z",
  "finished_at": "2026-08-21T10:00:00Z",
  "error": {}
}

GET/v1/jobs/{id}/results

The shape follows the mode. This is the endpoint an agent should read; `export` exists for handing a file to a human.

Parameters

curl https://api.crawlhawk.com/v1/jobs/583d8118-…/results \
  -H "Authorization: Bearer chk_live_…"

Response 200

{
  "job_id": "string",
  "mode": "string",
  "items": [
    {}
  ],
  "total": 0,
  "page": 0,
  "page_size": 0,
  "has_more": false
}

POST/v1/jobs/{id}/export

Returns a signed, single-use URL. The URL carries no API key, so it can be passed on to whatever needs the file.

Parameters

Body

curl -X POST https://api.crawlhawk.com/v1/jobs/583d8118-…/export \
  -H "Authorization: Bearer chk_live_…" \
  -H "Content-Type: application/json" \
  -d '{"format":"xlsx"}'

Response 200

{
  "url": "string",
  "format": "xlsx",
  "expires_at": "2026-08-21T10:00:00Z"
}

GET/v1/account

Credit balance, key metadata and the documented ceilings

curl https://api.crawlhawk.com/v1/account \
  -H "Authorization: Bearer chk_live_…"

Response 200

{
  "account_id": "string",
  "credits": 0,
  "top_up_url": "string",
  "key": {},
  "auth_method": "api_key",
  "scopes": [
    "string"
  ],
  "keys_active": 0,
  "webhook_failing": false,
  "limits": {}
}

GET/v1/webhooks

List the account’s webhook endpoints

curl https://api.crawlhawk.com/v1/webhooks \
  -H "Authorization: Bearer chk_live_…"

Response 200

[
  {
    "id": "string",
    "url": "string",
    "events": [
      "job.completed"
    ],
    "active": false,
    "created_at": "2026-08-21T10:00:00Z",
    "last_success_at": "2026-08-21T10:00:00Z",
    "failing_since": "2026-08-21T10:00:00Z",
    "last_error": "string"
  }
]

POST/v1/webhooks

The signing secret is returned once, here. Every delivery carries an X-CrawlHawk-Signature of `t=<unix>,v1=<hmac-sha256 of "t.body">` computed with it.

Body

curl -X POST https://api.crawlhawk.com/v1/webhooks \
  -H "Authorization: Bearer chk_live_…" \
  -H "Content-Type: application/json" \
  -d '{"url":"https://example.com"}'

Response 201

{
  "id": "string",
  "url": "string",
  "events": [
    "job.completed"
  ],
  "active": false,
  "created_at": "2026-08-21T10:00:00Z",
  "last_success_at": "2026-08-21T10:00:00Z",
  "failing_since": "2026-08-21T10:00:00Z",
  "last_error": "string",
  "secret": "string"
}

DELETE/v1/webhooks/{id}

Remove a webhook endpoint

Parameters

curl -X DELETE https://api.crawlhawk.com/v1/webhooks/583d8118-… \
  -H "Authorization: Bearer chk_live_…"

Webhooks

Crawls take minutes, so register a URL on your account instead of polling. You receive job.completed · job.failed · job.awaiting_confirmation · job.batch_ready events, each HMAC-signed with your webhook secret in the X-CrawlHawk-Signature header. Failed deliveries are retried with exponential backoff.

Errors

Errors are structured and machine-readable. The code is stable; messages may change.

Billing & auto top-up

Calls spend the credits on the account the key belongs to. Credits never expire and there is no subscription. When a job would cost more than the balance covers, the API answers 402 with the exact required and available amounts and a top_up_url — so an integration can surface the number rather than fail silently.

Balance and the published ceilings are readable at any time from GET /v1/account.