Skip to content
looot docs
Esc
↑↓navigate↵open⌘Jpreview
On this page

Results web crawl

Run job:web.crawl.results through the looot API: 2 providers, from $0 per call. Inputs, prices, output shape and code for curl, JavaScript and Python.

Results web crawl.

Job id web.crawl.results, in Enrichment (Web data). As of 2026-10-07, 2 providers serve it through 2 tools. Run job:web.crawl.results and looot picks one of them; with fallback on, a miss moves on to the next. See Jobs.

Inputs

Fields the job’s tools take, most shared first. Any key you send passes through unchanged.

Name Type Taken by Required by Description
cursor integer 2 of 2 0 Optional integer representing the index to start fetching content from. Useful to paginate until all URLs are fetched. Start with 0, then provide response['cursor'] value of the last request.
limit integer 2 of 2 0 Optional integer to limit the number of results returned. Recommended 10-50 results at a time. Paginated using cursor. Maximum 10MB of content can be fetched in a single request.
crawl_id string 1 of 2 1 The ID of the crawl to retrieve the list of URLs for.
formats array 1 of 2 0 Deprecated: Use /retrieve endpoint with retrieve_id. Array of formats to fetch (e.g., [“html”, “markdown”]).
id string 1 of 2 1
search_query string 1 of 2 0 An optional search query to sort the results by relevance. Uses the original search_query by default if provided.

Providers and prices

Provider Tool Price
Olostep olostep-crawl-pages $0 per call
ScrapeGraphAI scrapegraphai-crawl-pages $0 per call

A call that fails at the provider costs $0. See What’s free.

Output

The run’s result holds the provider’s answer. Signed in, looot inspect olostep-crawl-pages prints its fields.

Run it

Every call needs your API token in LOOOT_TOKEN; Sign in shows how to get one. Each run also needs a new idempotency key, so a retry never pays twice. With wait: 30 the answer comes back inline when the run ends within 30 seconds. Otherwise you get the running run back: poll GET /v1/runs/<runId>.

Example input with placeholder values:

curl -X POST "https://api.looot.ai/v1/runs" \
  -H "Authorization: Bearer $LOOOT_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: $(uuidgen)" \
  -d '{"endpointId":"job:web.crawl.results","input":{"crawl_id":"<crawl_id>"},"wait":30}'
const response = await fetch("https://api.looot.ai/v1/runs", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.LOOOT_TOKEN}`,
    "Content-Type": "application/json",
    "Idempotency-Key": crypto.randomUUID(),
  },
  body: JSON.stringify({
    endpointId: "job:web.crawl.results",
    input: {
      crawl_id: "<crawl_id>",
    },
    wait: 30,
  }),
});
const run = await response.json();
console.log(run.status, run.result);
import os
import uuid

import requests

response = requests.post(
    "https://api.looot.ai/v1/runs",
    headers={
        "Authorization": f"Bearer {os.environ['LOOOT_TOKEN']}",
        "Idempotency-Key": str(uuid.uuid4()),
    },
    json={
        "endpointId": "job:web.crawl.results",
        "input": {
            "crawl_id": "<crawl_id>",
        },
        "wait": 30,
    },
    timeout=90,
)
run = response.json()
print(run["status"], run.get("result"))

To pin one provider, send its tool id as endpointId instead of job:web.crawl.results. The shared input names above are for the job: run only; a tool id takes that tool’s own field names, listed on its page, and refuses the shared ones when they differ. For olostep-crawl-pages that is crawl_id.

Was this page helpful?