--- title: "TinyFish Fetch API / Fetch and extract content from URLs" description: "TinyFish Search API tool tinyfish-fetch on looot: input fields, $0 per call, output shape, and code to run it with curl, JavaScript or Python." sidebar: hidden: true --- {/* Generated by scripts/generate-api-reference.mjs from data/api-reference.json. Do not edit. */} Fetches web pages, renders JavaScript-heavy pages when needed, and returns clean extracted content in your preferred format. Submit up to 10 URLs, get back structured content. Per-URL failures appear in `errors[]` and do not fail the entire request. **Per-URL error codes** (in `errors[].error`): - `target_http_error`, target server returned a non-2xx HTTP status other than 404/410; the raw status code is in `errors[].status` - `page_not_found`, target URL returned HTTP 404 or 410; the raw status code is in `errors[].status` - `target_unreachable`, connection refused, TLS failure, DNS failure, or other network error - `timeout`, request timed out - `proxy_error`, proxy tunnel failure - `bot_blocked`, bot-challenge page detected (Cloudflare, etc.) - `empty_content`, page loaded but no extractable text was found - `login_required`, the target redirected to a login or account wall; retrying without credentials will not help - `content_too_large`, the document exceeds the 20MB size limit for generic documents (PDF and CSV have their own 50MB budgets); not retried in the browser - `invalid_url`, malformed URL or SSRF-blocked address - `invalid_redirect_url`, redirect target rejected before fetch - `conditional_unsupported`, conditional requests (`if_none_match` / `if_modified_since`) are supported on the fast path only; this URL requires browser rendering - `selector_not_matched`, no elements matching any `include_selectors` entry remained after `exclude_selectors` was applied; the error carries `unmatched_selectors` plus `candidate_selectors` retry hints (a partial miss is not an error, it's reported on the result's `unmatched_selectors`) - `selector_unsupported`, `include_selectors` / `exclude_selectors` sent for a URL that resolves to a direct PDF/CSV download (no HTML to scope) - **Tool id:** `tinyfish-fetch` - **Provider:** [TinyFish Search API](/providers/tinyfish) - **Job:** [Post fetch](/reference/jobs/fetch-post) (`fetch.post`) - **Price:** $0 per call. A call that fails at the provider costs $0. ## Inputs | Name | Type | Required | Description | | --- | --- | --- | --- | | `exclude_selectors` | array | no | Array of CSS selectors (1-20 entries, each 1-1000 characters) for elements to remove before extraction, applied before `include_selectors` scopes what remains, so it also prunes inside selected regions. Entries may themselves use CSS comma-grouping. Entries that match nothing are a no-op, never an error, but URLs that resolve to direct PDF/CSV downloads fail with `selector_unsupported`. Invalid CSS selector syntax is rejected with a 422. Applied post-fetch: caching and routing are unchanged. | | `format` | string | no | Output format for extracted content. "markdown" (default) is ideal for LLM consumption. "html" returns cleaned semantic HTML. "json" returns a structured document tree. Example: "markdown" | | `if_modified_since` | string | no | Last-Modified validator from a prior fetch of this URL, forwarded verbatim as the If-Modified-Since header on the origin request. Only valid with a single URL, combining with a batch of URLs returns a 400. tf-fetch does not persist validators; the caller owns replaying them. Example: "Wed, 21 Oct 2015 07:28:00 GMT" | | `if_none_match` | string | no | ETag validator from a prior fetch of this URL, forwarded verbatim as the If-None-Match header on the origin request. Only valid with a single URL, combining with a batch of URLs returns a 400. tf-fetch does not persist validators; the caller owns replaying them. Example: "W/\"abc123\"" | | `image_links` | boolean | no | Extract all image URLs (<img src>) from each page. Useful for finding visual content or media assets. Image links are returned as absolute URLs in the image_links array of each result. Example: false | | `include_etag_and_last_modified` | boolean | no | Opt-in to receiving `etag` / `last_modified` validators (and `not_modified` detection) on each result. Defaults to false, tf-fetch omits these fields unless requested. Independent of `if_none_match` / `if_modified_since`: works with a single URL or a batch. Example: true | | `include_selectors` | array | no | Array of CSS selectors (1-20 entries, each 1-1000 characters) that scope extracted content (`text`, `links`, `image_links`) to elements matching ANY entry, concatenated in document order. Tag selectors cover semantic sections (`main`, `article`, `nav`); entries may themselves use CSS comma-grouping. Selected content is returned verbatim in the requested format (scripts/styles stripped), automatic boilerplate removal is bypassed. Page-level metadata (`title`, `description`, `language`, `autho... | | `links` | boolean | no | Extract all outbound links (<a href>) from each page. Useful for discovering related pages or navigating to specific content. Links are returned as absolute URLs in the links array of each result. Example: false | | `page_metadata` | boolean | no | Return page-head metadata for each page in the page_metadata object of each result: canonical URL, favicon, robots directive, generator, viewport, keywords, all Open Graph (og), Twitter card (twitter), and article tags, and remaining named meta tags under `other`. Useful for SEO/technical audits and link-preview generation. Example: false | | `per_url_timeout_ms` | integer | no | Wall-clock timeout budget in milliseconds applied independently to each URL. If one URL exceeds this budget, it returns a per-URL timeout error while other URLs in the same request continue. Example: 45000 | | `purpose` | string | no | Why these URLs are being fetched, the underlying goal or task the content will be used for. Used to better tailor fetching and extraction to your intent. Example: "Compare pricing tiers across vendors for a procurement report" | | `ttl` | integer | no | Caller freshness tolerance in seconds for the cached entry. Omit (default) for unlimited tolerance, any cached entry is acceptable. Set to 0 to prefer a live fetch; a cached entry is still served if the origin's Cache-Control: max-age covers its age, or the host is in the small allowlist of operator-pinned never-expire domains. Set to N > 0 to accept a cached entry whose age is below N; the upstream Cache-Control: max-age and the never-expire allowlist may extend (never shorten) t... Example: 0 | | `urls` | array | yes | Array of URLs to fetch (1-10). All URLs are fetched in parallel. Each URL is processed independently, if one fails, others still return successfully. Errors are reported per-URL in the errors array. | ## Output Shape of the run's `result`, checked against 2 real answers: ```txt { errors: unknown[], results: { url, text, title, author, format, language, fina... ... ``` The shape is cut here. Signed in, `looot inspect tinyfish-fetch` prints all of it. ## Run it Every call needs your API token in `LOOOT_TOKEN`; [Sign in](/get-started/sign-in#use-the-token-in-scripts-and-agents) shows how to get one. Each run also needs a new idempotency key, so a retry never pays twice. With `wait: 30` the answer comes back inline when the run ends within 30 seconds. Otherwise you get the running run back: poll `GET /v1/runs/`. Example input with placeholder values: ```bash curl curl -X POST "https://api.looot.ai/v1/runs" \ -H "Authorization: Bearer $LOOOT_TOKEN" \ -H "Content-Type: application/json" \ -H "Idempotency-Key: $(uuidgen)" \ -d '{"endpointId":"tinyfish-fetch","input":{"urls":["https://example.com","https://example.org"],"ttl":0,"links":false,"format":"markdown","purpose":"Compare pricing tiers across vendors for a procurement report","image_links":false,"if_none_match":"W/\"abc123\"","page_metadata":false,"exclude_selectors":[".comments",".newsletter-signup"],"if_modified_since":"Wed, 21 Oct 2015 07:28:00 GMT","include_selectors":["article"],"per_url_timeout_ms":45000,"include_etag_and_last_modified":true},"wait":30}' ``` ```js JavaScript const response = await fetch("https://api.looot.ai/v1/runs", { method: "POST", headers: { Authorization: `Bearer ${process.env.LOOOT_TOKEN}`, "Content-Type": "application/json", "Idempotency-Key": crypto.randomUUID(), }, body: JSON.stringify({ endpointId: "tinyfish-fetch", input: { urls: ["https://example.com", "https://example.org"], ttl: 0, links: false, format: "markdown", purpose: "Compare pricing tiers across vendors for a procurement report", image_links: false, if_none_match: "W/\"abc123\"", page_metadata: false, exclude_selectors: [".comments", ".newsletter-signup"], if_modified_since: "Wed, 21 Oct 2015 07:28:00 GMT", include_selectors: ["article"], per_url_timeout_ms: 45000, include_etag_and_last_modified: true, }, wait: 30, }), }); const run = await response.json(); console.log(run.status, run.result); ``` ```python Python import os import uuid import requests response = requests.post( "https://api.looot.ai/v1/runs", headers={ "Authorization": f"Bearer {os.environ['LOOOT_TOKEN']}", "Idempotency-Key": str(uuid.uuid4()), }, json={ "endpointId": "tinyfish-fetch", "input": { "urls": ["https://example.com", "https://example.org"], "ttl": 0, "links": False, "format": "markdown", "purpose": "Compare pricing tiers across vendors for a procurement report", "image_links": False, "if_none_match": "W/\"abc123\"", "page_metadata": False, "exclude_selectors": [".comments", ".newsletter-signup"], "if_modified_since": "Wed, 21 Oct 2015 07:28:00 GMT", "include_selectors": ["article"], "per_url_timeout_ms": 45000, "include_etag_and_last_modified": True, }, "wait": 30, }, timeout=90, ) run = response.json() print(run["status"], run.get("result")) ``` To let looot pick among every provider of this job instead, send `job:fetch.post` as `endpointId`; see [the job page](/reference/jobs/fetch-post).