Anyone can extract a page once. The hard part is the Tuesday a site ships a redesign and your export quietly arrives with 40 rows instead of 1,200. ScrapeWise runs four fallback extraction modes, watches for drift, and tells you when a run is thin — instead of handing you a clean-looking empty file.
This is the failure that actually hurts. The job runs, the export lands on schedule, and the file is a third of its usual size. Nothing alerts, because from the pipeline's point of view nothing failed — and by the time someone notices the gap, the pages that would have filled it are gone.
A scraper pinned to a CSS path stops the day the class names change. One pinned to structured markup stops the day the site removes it. A single method means every layout change is an outage rather than a degraded run.
Adding a site means a developer writes, tests and deploys another script, then owns it forever. The backlog is not the first scraper — it is the twentieth, all of which still need maintaining.
JavaScript rendering, IP reputation, rate limits and challenge pages block most scrapers at the door. Handling them properly is infrastructure work that has nothing to do with the data you actually wanted.
A script that works on a sample needs proxy rotation, concurrency control, retry logic and request budgeting to survive at volume. That is the part teams discover after committing to the approach.
Per 1,000 pages scraped — pay for pages collected, not for a seat or a plan
Ready-made site endpoints, plus any public site you point the scraper at
Free requests on every new account — run a real scrape before committing to anything
Select the fields you want from any public page visually, or declare them as a schema with types. No Python, no Selenium, no developer ticket.
Rendering, pagination, infinite scroll and access are handled for you, and four extraction modes are tried in order so a layout change degrades the run instead of ending it.
Structured JSON, CSV or Excel, with the extraction mode and run status attached — so your pipeline can load a clean run and refuse a flagged one.
This is the actual sequence behind the run above — a 1,204-page retailer that shipped a redesign overnight, and why the export still arrived complete.
The scraper first reads the page's structured markup, which is the most reliable source when it exists. The redesign removed it. Under a single-method scraper this is the end of the run and the start of an empty export. Here it is a recorded mode failure on 1,204 pages, which is information rather than silence — and the run continues down the ladder.
DOM extraction does not depend on the exact class names surviving; it locates fields by their relationship to stable anchors — the price near the buy control, the title in the primary heading, the availability text in the offer block. The remap succeeds on 1,161 of 1,204 pages. The 43 that fail are the ones whose prices are now written into the DOM by JavaScript after load.
Those 43 are re-fetched with full rendering, and their prices resolve. Mode 4, the visual read, is never needed. The run finishes at 1,204 rows against 1,198 expected from the previous run, so the row-count guard passes. Had it come back at 400, the run would have been flagged before the export shipped — because a thin file that looks clean is the expensive failure, not a loud one.
Scrapers rarely fail on day one. They fail quietly in week six, and every one of these is a place where the output still looks valid while being wrong.
Four modes run as a ladder: structured markup, DOM selectors, rendered layout, and a visual read as the last resort. Which one produced each field is recorded, so a site degrading from mode 1 to mode 3 over a month is visible as a trend rather than as a surprise outage.
Anchors move, class names change, blocks get reordered. Drift is measured against the previous successful run per site, so a partial change surfaces as a warning on the fields it touched instead of as a silent null in a column nobody checks.
Every run is compared against the last one for the same scope. A run that returns materially fewer rows, or nulls a field that was populated yesterday, is flagged before delivery. This is the guard that turns a quiet empty file into an alert.
Challenge pages, JavaScript-dependent content, IP reputation and per-site rate limits decide whether a page can be read at all. Handled as infrastructure rather than as a per-scraper problem, so adding the twentieth site costs what the first one did.
Every run lands in one of three states, and the state ships with the data. A scraper that reports only success and failure is the one that hands you a third of a dataset marked success.
Row count and field population are consistent with the previous run for the same scope. The export is delivered, with the extraction mode that answered recorded per field.
A higher mode failed and a lower one recovered the data. The run is complete and usable, and the degradation is reported — because a site that has dropped to mode 3 is a site worth watching before it drops off the ladder entirely.
Row count or field coverage fell materially against the previous run. The run is flagged rather than delivered as if nothing happened, so you find out from an alert instead of from a report that looked fine until someone summed it.
You point at the pages and name the fields. You receive structured rows, plus the run metadata that tells you whether to trust them.
Category URLs, search result pages, a sitemap or an explicit URL list. Pagination and infinite scroll are traversed for you.
retailer-c.com/c/toolssitemap.xmlURL list (CSV)The fields you want from each page, selected visually or declared as a schema. Types are enforced, so a value that will not fit is surfaced instead of stringified.
title, price:decimal, availability:enum, sku, image_urlOn by default. A run whose row count or field coverage drops materially against the previous one is flagged rather than delivered. This is the single setting that prevents a silent partial export.
Automatic is cheaper and faster: plain fetches for pages that do not need a browser, full rendering only for the pages whose content is written by JavaScript.
Scheduled runs are diffed against each other, which is what makes drift detection and the row-count guard possible at all.
One row per page, plus run metadata, 11 columns each
Delivered as JSON over the REST API, through the MCP server, or as CSV and Excel exports after each scheduled run. Run status travels with the data, so a pipeline can refuse a flagged run instead of loading it.
One row per page, each carrying which extraction mode answered and whether the run was clean — which is what lets a downstream job decide whether to load it.
| run_id | source_url | title | price | availability | extraction_mode | row_status |
|---|---|---|---|---|---|---|
| RUN-4417 | retailer-c.com/p/1180 | Cordless drill 18V kit | 219.00 EUR | in_stock | 2 · DOM | complete |
| RUN-4417 | retailer-c.com/p/1181 | Impact driver 18V | 164.90 EUR | in_stock | 2 · DOM | complete |
| RUN-4417 | retailer-c.com/p/1182 | Angle grinder 125mm | 89.00 EUR | backorder | 3 · Rendered | degraded — JS-only price |
| RUN-4417 | retailer-c.com/p/1183 | Rotary hammer SDS+ | — | — | — | flagged — 403 after 3 retries |
| RUN-4417 | — run summary — | 1,204 pages | 1,198 previous | — | ladder 1→3 | complete — guard passed |
Structured markup, DOM selectors, rendered layout and a visual read run as a ladder. A site removing its markup costs you a rung, not a run — and the mode that answered is recorded per field, so degradation is visible long before it becomes an outage.
Each run is compared with the last stable one for the same site. When anchors move or a field starts returning nulls, you get a warning naming the fields that changed — instead of a column that quietly went empty six weeks ago.
Row count and field coverage are checked against the previous run before anything ships. A scrape that comes back at a third of its usual size is flagged, because the empty file that arrives on schedule is the failure that actually costs money.
ScrapeWise is the data scraper business teams use to pull structured rows from any public website — with four fallback extraction modes, drift detection and a guard that holds a run back when it comes up thin.
Common questions about using ScrapeWise as a data scraper.
Two things, and neither is the first run. Four extraction modes are tried as a ladder, so a site removing its structured markup costs a rung rather than the whole run. And every run is diffed against the previous one, so a scrape that comes back thin is flagged instead of delivered — which matters because the expensive failure is not a scraper that errors, it is one that quietly returns a third of the data on schedule.