Steel API / Scrape a web page to markdown or HTML

Steel API tool steel-scrape on looot: input fields, $0.005 per call, output shape, and code to run it with curl, JavaScript or Python.

Loads a URL in Steel’s browser and returns its content. Send url. format picks the content formats (html by default, markdown among the others); screenshot and pdf add those files. delay waits up to 10 s before scraping. Returns content in the requested formats, links and metadata (title, description, language, status code).

Tool id: steel-scrape
Provider: Steel API
Job: Scrape a web page into markdown (web.scrape.markdown)
Price: $0.005 per call. A call that fails at the provider costs $0.

Inputs

Name	Type	Required	Description
`delay`	number	no	Delay before scraping (in milliseconds). Example: 2000
`format`	array	no	Desired format(s) for the scraped content. Default is `html`.
`pdf`	boolean	no	Include a PDF in the response
`projectId`	string	no	Project to execute the scrape in.
`region`	string	no	The desired region for the action to be performed in
`screenshot`	boolean	no	Include a screenshot in the response
`url`	string	yes	URL of the webpage to scrape. Example: “https://www.example.com”
`useProxy`	boolean	no	Use a Steel-provided residential proxy for the scrape

Output

Shape of the run’s result, checked against 2 real answers:

{ links: { url, text }[], content: { html, markdown }, metadata: { ogUrl, title, author, jsonLd: unknown[], favicon, ogImage, ogTitle, keywords, language, timestamp, urlSource, ogSiteName, statusCode, description, modifiedTime, articleAuthor, ogDescription, publishedTime } }

Run it

Every call needs your API token in LOOOT_TOKEN; Sign in shows how to get one. Each run also needs a new idempotency key, so a retry never pays twice. With wait: 30 the answer comes back inline when the run ends within 30 seconds. Otherwise you get the running run back: poll GET /v1/runs/<runId>.

Example input with placeholder values:

curl -X POST "https://api.looot.ai/v1/runs" \
  -H "Authorization: Bearer $LOOOT_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: $(uuidgen)" \
  -d '{"endpointId":"steel-scrape","input":{"url":"https://www.example.com","delay":2000},"wait":30}'

const response = await fetch("https://api.looot.ai/v1/runs", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.LOOOT_TOKEN}`,
    "Content-Type": "application/json",
    "Idempotency-Key": crypto.randomUUID(),
  },
  body: JSON.stringify({
    endpointId: "steel-scrape",
    input: {
      url: "https://www.example.com",
      delay: 2000,
    },
    wait: 30,
  }),
});
const run = await response.json();
console.log(run.status, run.result);

import os
import uuid

import requests

response = requests.post(
    "https://api.looot.ai/v1/runs",
    headers={
        "Authorization": f"Bearer {os.environ['LOOOT_TOKEN']}",
        "Idempotency-Key": str(uuid.uuid4()),
    },
    json={
        "endpointId": "steel-scrape",
        "input": {
            "url": "https://www.example.com",
            "delay": 2000,
        },
        "wait": 30,
    },
    timeout=90,
)
run = response.json()
print(run["status"], run.get("result"))

To let looot pick among every provider of this job instead, send job:web.scrape.markdown as endpointId; see the job page.