A model is only as honest as the rows behind it. ScrapeWise turns public web pages into typed, schema-constrained records where every field carries the URL and timestamp it came from — and a field the page never published stays null instead of becoming a plausible guess.
An LLM asked to pull a field from a page will almost always return something. A missing lead time comes back as a confident number, indistinguishable in the output from one the page actually published. Downstream, that value is trained on or cited to a customer, and the failure is silent by construction.
Navigation, cookie banners, recommendation carousels and footer link farms are most of a page's tokens and none of its meaning. Embedded raw, they push real content out of chunks and make near-identical pages look identical to a vector index.
When a model cites a price, someone eventually has to answer where it came from and when. If the record does not carry its source URL and extraction timestamp, that question has no answer and the whole chain becomes unverifiable after the fact.
A RAG index built three months ago answers today's pricing question with three-month-old figures and no hedge. Freshness is not a nice-to-have in grounded retrieval; without a per-record timestamp there is no way to decline to answer.
Prices arriving as "€1.219,00", "1219.00" and "from 1219" in the same column do not break the scraper. They break the aggregation three steps later, long after the page that produced them has changed.
Per 1,000 pages extracted — corpus collection stops being the expensive part of the pipeline
Ready-made site endpoints, plus any public source you point us at
Free requests on every new account — inspect the typed rows before you wire up ingestion
Give us the sources and the fields you need with the type each must be. Extraction, rendering and the inference policy are agreed before anything is collected.
Page furniture is separated from content, values are coerced against the page's locale, absent fields stay null, uncoercible ones are surfaced, and duplicates are collapsed by content hash.
JSON or CSV over the REST API, through the MCP server, or written to your object store — ready for embedding, fine-tuning or agent grounding, with every field traceable to its source.
This is the actual sequence behind the grounding record above — a German retailer product page turned into a record an agent can be trusted to cite, including the field that does not exist.
Navigation, cookie consent, recommendation carousels, review pagination and footer links are identified and discarded before anything is read. What survives is the region of the page that actually describes the product. This matters twice: the noise is most of the token budget, and two pages that differ only in their recommendation carousel are otherwise near-identical to an embedding model.
The schema says price is a decimal in a stated currency, availability is one of a fixed enum, warranty_months is an integer. "1.219,00 €" is parsed against German conventions to 1219.00 EUR rather than 1.21900. Anything that cannot be coerced to its declared type is rejected at extraction, where the page is still available to check, instead of silently entering the corpus as a string.
Each field carries the source URL, the region of the page it was read from and the extraction timestamp — so a model citing the price can be traced to the page and the hour. The fourth field, lead_time_days, is not published anywhere on this page. It comes back null. That is the whole design: a null is a fact about the source, and a plausible integer in its place would be indistinguishable from a real one.
Pipelines for AI rarely fail at collection. They fail at the boundary between a web page and a typed record — and every one of those failures produces output that looks correct.
Boilerplate is stable across a site and dominates token counts, which means it both wastes context and makes distinct pages look similar in vector space. Identifying the content region is the first operation, not a cleanup pass afterwards.
A declared schema is what turns extraction into something that can fail loudly. Decimal separators, thousands separators, currency position and unit suffixes are resolved against the page's locale, and a value that will not coerce is rejected rather than stringified.
Every field carries the URL it was read from, the page region, and when. This is what makes a cited answer auditable and what lets a retrieval layer decline to answer from a record older than its freshness policy.
Syndicated articles, paginated variants and the same product description across regional domains inflate a corpus and skew whatever is trained on it. Content hashing plus near-duplicate detection collapses them while keeping every source URL that carried the content.
Every field in every record lands in one of three states, and the states are visible in the output. A pipeline that cannot tell them apart is a pipeline that will eventually cite a guess.
The value was present and is carried with the URL, the page region it came from and the extraction timestamp. This is the only state a model should be allowed to cite without a hedge.
The page does not publish this field. It comes back null with a reason, never as a plausible value. Downstream that means a retrieval layer can say it does not know, which is the single most valuable behaviour a grounded system has.
A value was found but will not fit its declared type: a price reading "from 1219", a date in an unresolvable format, an enum value outside the allowed set. Surfaced for review with the raw string and the URL, so a schema or locale problem is caught while the page is still there to inspect.
You define the sources and the schema. You receive typed records where every field carries its provenance, absent fields are null, and uncoercible ones are surfaced rather than swallowed.
Sitemaps, category paths, search URLs or an explicit URL list. Dynamic and JavaScript-rendered pages are handled the same as static ones.
sitemap.xmlretailer-b.de/c/power-toolsURL list (CSV)The fields you want and the type each one must be. Anything that cannot be coerced is rejected at extraction rather than entering the corpus as a string.
price:decimal(EUR), availability:enum, warranty_months:int, published_at:dateOn by default and the reason this page exists. Absent fields return null with a reason. Turning it off lets a model fill gaps, which is occasionally useful and should always be a deliberate choice.
Content hashing plus near-duplicate detection across syndicated, paginated and regional copies. Every source URL that carried the content is retained on the canonical record.
Records carry an extraction timestamp, so a retrieval layer can apply a freshness policy and decline to answer from data older than it allows.
One row per field per record, 12 columns each
Delivered as JSON or CSV over the REST API, through the MCP server, or written straight to your object store. Field-level rows and assembled records arrive together, so an embedding job and an audit query can read the same extraction.
One row per field, each carrying its type, its provenance and its status — which is what lets an agent cite a value and a reviewer check it. Note the null and the rejection: both are results, not failures.
| record_id | field_name | raw_value | coerced_value | declared_type | source_region | field_status |
|---|---|---|---|---|---|---|
| R-88214 | price | 1.219,00 € | 1219.00 EUR | decimal | json-ld offer | sourced |
| R-88214 | availability | Auf Lager | in_stock | enum | json-ld offer | sourced |
| R-88214 | warranty_months | 24 Monate | 24 | integer | spec table | sourced |
| R-88214 | lead_time_days | — | null | integer | — | absent — not published |
| R-88215 | price | ab 1.219,00 € | — | decimal | listing block | rejected — range, not a price |
Extraction runs against a schema with declared types, so values are coerced against the page's own locale and anything that will not fit is rejected while the page is still available to inspect. Aggregation stops failing three steps downstream.
Fields the page does not publish come back null with a reason, never as a confident guess. This is what lets a grounded system decline to answer — the single behaviour that separates a citable pipeline from a persuasive one.
Source URL, page region and extraction timestamp travel with each value, so a cited answer can be traced to the page and the hour it was read, and a retrieval layer can enforce a freshness policy instead of answering from a stale index.
Stop embedding navigation menus and stop shipping inferred values as facts. ScrapeWise delivers typed, deduplicated records where every field knows which URL it came from and when — and where a field the page never published stays null.
Everything you need to know about extracting web data for AI training, RAG and agent grounding with ScrapeWise.
An LLM asked for a field will nearly always return one. Where the page does not publish it, you get a plausible value that is indistinguishable in the output from a real one. ScrapeWise extracts against a declared schema: absent fields come back null with a reason, uncoercible ones are surfaced for review, and the same page produces the same record on every run.