Guides

Rendering & tiers

Siftzo has two ways to fetch a page. The render parameter decides which one is used, and whether Siftzo may switch between them.

The two tiers

Tiers
TierWhat it doesCredits
httpAn HTTP client with the TLS and HTTP/2 fingerprint of Chrome. Fast and cheap. Returns the HTML the server sends, without running JavaScript.1
browserGoogle Chrome in a fresh profile for each task. Runs JavaScript, can wait for elements and take screenshots. Images, fonts and media are not loaded, to keep it fast, except images and fonts for a screenshot.10

Both tiers send traffic through residential proxies (see Proxies). You pay for the tier that produced the result, and only when it succeeded.

The render parameter

Render modes
ValueBehaviour
auto (default)Starts on the tier that usually works for this host, which is http unless the host has been blocking it. If the HTTP attempt is blocked, the next attempt uses the browser.
neverOnly http. Retries stay on HTTP. Cannot be combined with wait_for, screenshot or iframes.
alwaysOnly browser. Use it for pages that build their content with JavaScript.

A request with wait_for, screenshot or iframes always starts on the browser, also with auto.

With auto, an HTTP answer that is only the empty shell of a JavaScript app (an empty <div id="root"> and almost no text) is rendered in the browser instead, and the domain then starts on the browser. With never you get the shell as the site sent it.

Retries and escalation

Each request gets at most 3 attempts, all within its timeout_ms.

  • Blocked (an anti-bot challenge, an access-denied page, a captcha): with auto on HTTP, the next attempt moves to the browser. Otherwise it retries on the same tier.
  • Proxy error, unreachable target or timeout of one attempt: retried on the same tier.
  • Success, or a response over 10 MB: no more attempts.

Without a session, every attempt uses a new exit IP. With a session, all attempts use the session's IP.

The browser tier has two engines. Google Chrome is the default. When Chrome is blocked, the next attempt uses a fallback engine based on Firefox, which presents a different browser fingerprint, once per request; a later attempt goes back to Chrome. Hosts where Chrome keeps getting blocked and the fallback gets through start on the fallback. Both engines support wait_for, screenshot and iframes, cost the same and are reported as browser in the tier field. A fallback attempt takes at most 45 seconds.

One attempt on HTTP can take at most 30 seconds, one on the browser at most 60 seconds. A new attempt is only started if there is enough time left (3 seconds for HTTP, 10 for the browser). If you need a browser and set a short timeout, give it at least 15 seconds.

Per-host strategy

With auto, Siftzo keeps a running success rate per host and tier. When HTTP succeeds less than half the time on a host, new requests to that host start on the browser and skip the failed HTTP attempt. Every 20th request to such a host tries HTTP again, so hosts that relax their protection go back to the cheaper tier. This memory expires after 7 days without traffic.

What counts as blocked

Siftzo looks at the status code, headers and body for the signatures of common anti-bot systems: challenge pages, access-denied pages with a reference ID, captcha interstitials and the other block pages common protection services show. A blocked page is never returned as a result and never billed. If every attempt is blocked you get 424 target_blocked.

A 429 Too Many Requests from the target also counts as a block, since it means the exit IP is being rate limited. A plain error page from the site itself (a 500, or a 403 without an anti-bot signature) is not a block. It is returned as a result with its status code, and it is not billed either (see billing rules).

wait_for

A CSS selector. The browser waits until an element matching it is on the page before it captures the HTML, within the request timeout.

If the element never appears, the attempt is retried on a new exit IP while the timeout allows. When no attempt finds it, the request fails with 408 timeout (the wait_for selector did not appear within timeout_ms) and costs nothing. A selector that cannot be parsed gives 400 invalid_request.

Request body
{
  "url": "https://quotes.toscrape.com/js/",
  "render": "always",
  "wait_for": ".quote"
}

actions

Steps the browser performs on the loaded page, in order, before it captures the HTML (and the screenshot). Use them to open what a click reveals, run a search, or load a list that grows as it is scrolled. At most 20 per request. A request with actions always runs on the browser.

typeFieldsWhat it does
clickselectorClicks the first element matching the CSS selector, once it is visible.
typeselector, textReplaces the content of an input or textarea with the text (at most 1,000 characters).
presskeyPresses a key, for example Enter, Escape or Control+A.
scrollto or pixelsto: "bottom" scrolls to the end and keeps going while the page grows (up to 8 rounds); to: "top" scrolls back. pixels scrolls down by that much, or up when negative.
waitmsWaits 1 to 10,000 milliseconds.
wait_forselectorWaits until an element matching the selector is on the page.
Request body
{
  "url": "https://example.com/search",
  "actions": [
    { "type": "type", "selector": "input[name=q]", "text": "tractor" },
    { "type": "press", "key": "Enter" },
    { "type": "wait_for", "selector": ".result" },
    { "type": "scroll", "to": "bottom" }
  ]
}

Each step gets up to 10 seconds, all within timeout_ms. If a step cannot complete, the steps after it do not run and the attempt is retried. When no attempt completes them, the request fails with 408 timeout and a message naming the step, for example actions[1] (click): element did not appear or could not be used in time, and costs nothing. Actions run after the top-level wait_for. They cannot run scripts.

screenshot

Set screenshot: true to get a PNG of the browser viewport in the screenshot field, base64-encoded. The HTML is still returned in target.body.

For a screenshot, the browser also loads the page's stylesheets, images and fonts, so it shows the page as a visitor sees it. Video, audio and known ad and tracking services stay blocked. A screenshot costs the same credits as any other browser request.

Add screenshot_full_page: true to capture the whole page instead of the viewport. Lazy-loaded images (loading="lazy") are loaded first. Images that a script loads only when you scroll may still be missing.

  • The viewport is a desktop browser window, about 1920 pixels wide.
  • A full-page screenshot is at most 10,000 pixels tall. A taller page is cut off.
  • A screenshot is at most 4 MB. A full-page screenshot that is larger is cut shorter until it fits, and a viewport screenshot that is larger is left out: screenshot is then null.
Node.js
import { writeFile } from "node:fs/promises";

const res = await fetch("https://api.siftzo.com/v1/scrape", {
  method: "POST",
  headers: { "X-API-Key": process.env.SIFTZO_API_KEY, "Content-Type": "application/json" },
  body: JSON.stringify({ url: "https://example.com", screenshot: true, screenshot_full_page: true }),
});
const result = await res.json();
await writeFile("page.png", Buffer.from(result.screenshot, "base64"));

iframes

Content in an <iframe> is a separate document, so it is not part of the page's HTML: embedded listings, booking widgets, maps and forms from another domain are missing from target.body. Set iframes: true and the browser copies each frame's content into the HTML before it is returned.

  • Each frame's body follows its iframe in a <div data-siftzo-iframe="frame URL">. The iframe itself stays.
  • Frames from other domains, srcdoc frames and frames filled by JavaScript are included. Nested frames sit inside their parent's copy.
  • Links and sources are made absolute against the frame's own URL. Scripts and styles are left out.
  • Lazy-loaded iframes on the page are loaded too. Overlays (an iframe in a position: fixed element, such as a chat widget or a popup) and hidden frames are left out. At most 50 frames, 5 levels deep, and a few seconds per page: empty frames and frames that do not answer in time are left out.
  • With format: "markdown", each frame's content is converted as if its own URL was scraped, and is part of the page's Markdown. Content in open shadow roots is included. A screenshot shows the page as it looks, without the copies.
  • Ad and tracking frames are blocked as always, so they stay empty. The 10 MB limit applies to the page with its frames.
Request body
{
  "url": "https://example.com/contact",
  "iframes": true,
  "format": "markdown"
}

Before a request runs, Siftzo sets aside the most it can cost: 10 credits for auto and always, 1 for never. If you are close to your monthly limit, render: "never" requests can still run when auto ones are refused.