Skip to content
looot docs
Esc
↑↓navigate↵open⌘Jpreview
On this page

Olostep / Map website URLs

Olostep tool olostep-map on looot: input fields, $0.0018 per call, output shape, and code to run it with curl, JavaScript or Python.

This endpoint allows users to get all the urls on a certain website. It can take up to 120 seconds for complex websites. For large websites, results are paginated using cursor-based pagination. Large sites can take up to two minutes.

  • Tool id: olostep-map
  • Provider: Olostep
  • Job: Sitemap web (web.sitemap)
  • Price: $0.0018 per call. A call that fails at the provider costs $0.

Inputs

Name Type Required Description
cursor string no OPTIONAL: Pagination cursor from a previous response. When provided, returns the next set of URLs from where the previous request left off due to response size limit.
exclude_urls array no URL path patterns to exclude using glob syntax. For example: /careers/**. Excluded URLs will supersede included URLs.
include_subdomain boolean no Include subdomains of the given URL. true by default.
include_urls array no URL path patterns to include using glob syntax. For example: /blog/** to only include blog URLs. Only URLs matching these patterns will be returned.
search_query string no An optional search query to sort the links by search relevance.
top_n number no An optional number to limit to only top n links for a search query.
url string yes The URL of the website for which you want the links

Output

The run’s result holds the provider’s answer. Signed in, looot inspect olostep-map prints its fields.

Run it

Every call needs your API token in LOOOT_TOKEN; Sign in shows how to get one. Each run also needs a new idempotency key, so a retry never pays twice. With wait: 30 the answer comes back inline when the run ends within 30 seconds. Otherwise you get the running run back: poll GET /v1/runs/<runId>.

Example input with placeholder values:

curl -X POST "https://api.looot.ai/v1/runs" \
  -H "Authorization: Bearer $LOOOT_TOKEN" \
  -H "Content-Type: application/json" \
  -H "Idempotency-Key: $(uuidgen)" \
  -d '{"endpointId":"olostep-map","input":{"url":"https://example.com"},"wait":30}'
const response = await fetch("https://api.looot.ai/v1/runs", {
  method: "POST",
  headers: {
    Authorization: `Bearer ${process.env.LOOOT_TOKEN}`,
    "Content-Type": "application/json",
    "Idempotency-Key": crypto.randomUUID(),
  },
  body: JSON.stringify({
    endpointId: "olostep-map",
    input: {
      url: "https://example.com",
    },
    wait: 30,
  }),
});
const run = await response.json();
console.log(run.status, run.result);
import os
import uuid

import requests

response = requests.post(
    "https://api.looot.ai/v1/runs",
    headers={
        "Authorization": f"Bearer {os.environ['LOOOT_TOKEN']}",
        "Idempotency-Key": str(uuid.uuid4()),
    },
    json={
        "endpointId": "olostep-map",
        "input": {
            "url": "https://example.com",
        },
        "wait": 30,
    },
    timeout=90,
)
run = response.json()
print(run["status"], run.get("result"))

To let looot pick among every provider of this job instead, send job:web.sitemap as endpointId; see the job page.

Was this page helpful?