Which scraper returns useful content?
The same pages. Different scrapers. A comparison of how much content comes back, how long it takes, and where each provider works best.
Updating daily. Last completed run: Sep 30, 2026, 6:18 AM UTC. Next scheduled: Oct 1, 2026, 12:15 AM UTC. Inspect runs
Which scrapers deliver?
Target: 3,000 URLs in one daily independent run. The same frozen set goes to every provider; shortages remain visible.
How often does each scraper return the page’s main content? These URLs come from public feeds and registries.
Latest completed daily run.
3000 unique URLs · 90 domains · 8 page types · 12000 recorded page tests. Collected Sep 30, 2026 (UTC).
| Provider | Useful | Time |
|---|---|---|
| 85.6%2568 / 3000 pages | 3.19s | |
| 84.9%2546 / 3000 pages2 unresolved · up to 84.9% | 2.02s | |
| 79.3%2378 / 3000 pages1 unresolved · up to 79.3% | 1.28s | |
| 66.3%1990 / 3000 pages1 unresolved · up to 66.4% | 0.78s |
Useful = pages with matching, substantive content / all selected page tests. Partial content can qualify. Time = median request duration. Scoring details
Confirmed useful content
Share of all page tests, including unresolved · higher is better
Unresolved results
Share of all page tests · lower is better
Quality over time
One point per provider, per UTC day. Each daily run targets 3,000 URLs, shared by every scraper. The leaderboard above shows the latest completed daily run.
One day recorded; a trend needs more than one day. Up to 90 days of completed results under the same scoring protocol. Missing days remain gaps. Earlier small batches are labeled and replaced by that day’s daily run once it finishes. Rates show confirmed coverage of the full sample, including unresolved and unattempted tests. Changes can reflect account availability and page mix as well as the provider.
Different pages, different results
An average hides the hard parts. Compare useful-content rates across page types to find the fit for your workload.
| Page type | Context | Exa | Keenable | Firecrawl |
|---|---|---|---|---|
| Commerce | 96% | 91% | 61% | 97% |
| News | 97% | 80% | 89% | 85% |
| Community | 86% | 85% | 18% | 77% |
| Jobs | 100% | 100% | 99% | 99% |
| Legal & government | 43% | 41% | 47% | 63% |
| Software & docs | 79% | 82% | 80% | 78% |
| Finance | 98% | 98% | 90% | 97% |
| Research | 86% | 57% | 48% | 83% |
Same selected page set as above. Each cell is the share of page tests with useful content.
What does one scraper miss?
Two providers can have the same score and recover different pages. These pairs show where their coverage overlaps.
Contextvs.Exa- Both
- 2297
- Context only
- 270
- Exa only
- 81
- Neither
- 351
1 unresolved · 3000 matched page tests
Contextvs.Keenable- Both
- 1945
- Context only
- 622
- Keenable only
- 45
- Neither
- 387
1 unresolved · 3000 matched page tests
Contextvs.Firecrawl- Both
- 2414
- Context only
- 152
- Firecrawl only
- 132
- Neither
- 300
2 unresolved · 3000 matched page tests
Exavs.Keenable- Both
- 1904
- Exa only
- 474
- Keenable only
- 86
- Neither
- 534
2 unresolved · 3000 matched page tests
Exavs.Firecrawl- Both
- 2242
- Exa only
- 134
- Firecrawl only
- 303
- Neither
- 318
3 unresolved · 3000 matched page tests
Keenablevs.Firecrawl- Both
- 1901
- Keenable only
- 87
- Firecrawl only
- 644
- Neither
- 365
3 unresolved · 3000 matched page tests
Who returns the best output?
Compare the fresh responses directly. Jev compares anonymous outputs for useful detail, coverage and cleanliness. Identical text ties automatically. Extra length alone does not win.
314 comparable pages · 186 with only one useful response · 244 with none · 2256 unresolved · 0 pending
| Provider | Best or tied | Share | Only useful |
|---|---|---|---|
| Context | 1of 314 compared | 0.3% | 58 |
| Firecrawl | 80of 314 compared | 25.5% | 93 |
| Exa | 152of 314 compared | 48.4% | 24 |
| Keenable | 82of 314 compared | 26.1% | 11 |
Best among the observed outputs, not verified ground truth. Ties count for each winning provider. Pages with missing, uncertain, cached or non-contemporaneous evidence do not enter this comparison. The useful-content score above still includes every selected page.
Latency & cost
Timing starts when the API request begins and ends after the complete response arrives. It includes provider failures, excludes judging and account/configuration errors, and uses the same 30-second deadline. P95 is unstable for small samples.
Median latency
Seconds, all timed requests · lower is better
Tail latency
95th percentile, seconds · lower is better
Report pricing
The price of 1,000 fresh, base-rate pages on a named plan. Like NEEDLE, the default is the largest published self-service tier. Change a plan to compare its rate.
ContextFirecrawl| Provider & selected plan | Base / 1k pages | Projected / 1k useful |
|---|---|---|
| ContextScale · monthly | $0.499 | $0.583 |
| FirecrawlScale · monthly | $0.749 | UnknownNeeds resolved results and useful pages |
| ExaPay as you go | $1.000 | UnknownNeeds resolved results and useful pages |
| KeenableLive fetch | Unknown | Unknown |
Subscription rates assume the full included allowance is used. Annual rates use the exact yearly payment ÷ 12, not rounded advertised monthly prices. Rates are rounded to $0.001. Projected useful-page cost = base rate ÷ useful-content rate, assuming one billable base page per attempt; it is not a measured charge.
Changing pricing does not change the measured results or imply they were collected on that plan. Free allowances, promotions and negotiated quotes are excluded. Keenable’s live-fetch price is unverified. Checked September 30, 2026. Plans, budgets & billing evidence · Download prices
A fresh supply of pages
Public feeds, sitemaps and registries supply the independent sample. NEEDLE supplies changing queries, which Context Search turns into a separate stream of URLs. Both are deduplicated before collection.
100 of 107 source endpoints succeeded at their latest poll. Failed sources remain in the registry and are retried on their next scheduled poll.
| Category | Inventory | First seen, 24h | Domains |
|---|---|---|---|
| Commerce | 7,304 | 7,304 | 5 |
| Community | 867 | 867 | 20 |
| Finance | 473 | 473 | 2 |
| Jobs | 1,694 | 1,694 | 4 |
| Legal & government | 482 | 482 | 13 |
| News | 1,040 | 1,040 | 25 |
| Research | 1,400 | 1,400 | 5 |
| Software & docs | 924 | 924 | 19 |
“First seen” means new to this collector, not necessarily newly published. The first poll includes existing inventory. Category and domain counts do not establish scraping difficulty. Shortfalls are retained, without substituting easier categories.
Frequently asked questions
Appendix
Fresh retrieval
Every selected page is requested fresh, with one attempt and the same 30-second deadline. “Unreported” means the provider did not supply usable cache evidence. A reported cache hit remains unresolved, not a fresh-content success. Firecrawl also receives storeInCache=false.
| Provider | Request setting | Fresh | Unreported |
|---|---|---|---|
| Context | maxAgeMs = 03000/3000 requests verified | 29440 cache hits | 56 |
| Firecrawl | maxAge = 03000/3000 requests verified | 00 cache hits | 3000 |
| Exa | maxAgeHours = 03000/3000 requests verified | 28190 cache hits | 181 |
| Keenable | live = true3000/3000 requests verified | 00 cache hits | 3000 |
Unresolved results
Context: All selected results resolved.
Firecrawl: 1 judge error; 1 content unresolved
Exa: 1 content unresolved
Keenable: 1 judge error
Pricing methodology
NEEDLE’s pricing catalog stores a price, plan and check date for each engine and describes its comparison as the biggest self-serve tier, excluding enterprise quotes. Trace uses that convention for scraping, with the same billing period across subscriptions. Search prices are not scraping prices. The plan and billing selections in Report pricing also apply below.
Budget for 3,000 pages a day
93,000 billable base pages in a 31-day month, per provider, on your selected plans. Estimates, not invoices. This includes unused capacity and whole top-up blocks; it differs from the full-allowance unit rate above.
| Provider & plan | Monthly spend | Blended / 1k pages |
|---|---|---|
| ContextScale | $499.00 | $5.366 |
| FirecrawlScale | $749.00 | $8.054 |
| ExaPay as you go | $93.00 | $1.000 |
| KeenableLive fetch | Unknown | Unknown |
Annual fees are amortized across 12 months; included credits still renew monthly. Assumes no opening balance, rollover, free credits or promotional discounts, and top-ups enabled. Failure billing and special URL charges can change the invoice. Excludes search, judging, hosting, taxes and extra formats. A plan’s throughput limits may prevent a five-minute run.
Reported usage
| Provider | USD subtotal | Credits subtotal | Reported / 1k useful |
|---|---|---|---|
| Context | Unknown0/3000 pages reported | 2,9443000/3000 pages reported | Unknown |
| Firecrawl | Unknown0/3000 pages reported | 2,9502950/3000 pages reported · partial | Unknown |
| Exa | $2.8193000/3000 pages reported | Unknown0/3000 pages reported | Unknown |
| Keenable | Unknown0/3000 pages reported | Unknown0/3000 pages reported | Unknown |
API receipts for the selected page set, not the account invoice. Missing receipts stay unknown, including timed-out requests that may still be billed. Credits are provider-specific; they are not converted to dollars at a hypothetical plan rate. Reported cost per useful page requires USD receipts for every selected page and fully resolved judgments. Subscription fees absent from receipts and operational costs are excluded.
Published plans
Included-credit rates and overage rates are different. The first assumes full use of the subscription; the second applies only after that allowance runs out. Free tiers are bounded allowances, not sustainable zero-cost tariffs.
Context
Markdown scrape · maxAgeMs=0 · 1 base credit/page · Official pricing · Billing rules · Checked 2026-09-30
| Plan & commitment | Credits / mo | Base / 1k pages | Extra / 1k credits |
|---|---|---|---|
| Free$0.00/mo1 concurrent | 1,000 | Limited free tier | Unavailable |
| Developer$19.00/mo10 concurrent | 7,500 | $2.533 | $2.20$2.20 per 1,000-credit block |
| Pro$99.00/mo100 concurrent | 125,000 | $0.792 | $1.80$1.80 per 1,000-credit block |
| Growth$299.00/mo250 concurrent | 500,000 | $0.598 | $1.40$1.40 per 1,000-credit block |
| Scale$499.00/mo500 concurrent | 1,000,000 | $0.499 | $1.00$1.00 per 1,000-credit block |
| EnterpriseCustom quote | — | Unknown | Unknown |
Current new-subscription prices. Annual billing saves two months on the base subscription. Overage is purchased in 1,000-credit blocks; auto top-up must be enabled. Most failures are unbilled, but 404 responses are billed. Browser actions and extra output formats cost more. Existing accounts may retain older rates and limits.
Firecrawl
Markdown scrape · maxAge=0 · parsers=[] · 1 base credit/page · Official pricing · Billing rules · Checked 2026-09-30
| Plan & commitment | Credits / mo | Base / 1k pages | Extra / 1k credits |
|---|---|---|---|
| Free$0.00/mo2 concurrent · 10 req/min | 1,000 | Limited free tier | Unavailable |
| Hobby$19.00/mo5 concurrent · 100 req/min | 5,000 | $3.800 | $5.00$5.00 per 1,000-credit block |
| Standard$99.00/mo25 concurrent · 500 req/min | 100,000 | $0.990 | $2.50$5.00 per 2,000-credit block |
| Growth$399.00/mo50 concurrent · 5,000 req/min | 500,000 | $0.798 | $2.00$5.00 per 2,500-credit block |
| Scale$749.00/mo100 concurrent · 10,000 req/min | 1,000,000 | $0.749 | $1.00$5.00 per 5,000-credit block |
| EnterpriseCustom quote | — | Unknown | Unknown |
Annual commitments use the exact published totals, not rounded monthly labels. Allowances renew monthly; Scale has one month of rollover. Extra credits come in $5 blocks and require pay-as-you-go to be enabled. Returned 403/404 pages still cost a credit; a scrape with no result is uncharged. Enhanced proxy escalation has no surcharge. PDF parsing is disabled in this benchmark. X/Twitter URLs have separate billing. Extra formats and PDF parsing can cost more. Concurrency is concurrent browsers; request rate limits also apply.
Exa
/contents · text only · maxAgeHours=0 · Official pricing · Checked 2026-09-30
| Plan & commitment | Credits / mo | Base / 1k pages | Extra / 1k credits |
|---|---|---|---|
| Pay as you goNo subscription | — | $1.000 | Same rate |
| EnterpriseCustom quote | — | Unknown | Unknown |
Text content is $1 per 1,000 pages, with no subscription minimum. Each additional content type is billed separately. Published $10 monthly free credits and the onboarding bonus depend on account eligibility and shared usage; they are not deducted from this gross comparison. Enterprise discounts are custom. These are Contents prices, not Search prices.
Keenable
/fetch · live=true · fetch.live SKU · Official pricing · Checked 2026-09-30
| Plan & commitment | Credits / mo | Base / 1k pages | Extra / 1k credits |
|---|---|---|---|
| Live fetchCustom quote | — | Unknown | Unknown |
The docs advertise a shared monthly allowance of 100,000 requests, but live fetch has a distinct SKU and organization-specific credit pricing. A public USD tariff for this mode is not established. Search-tier prices and the ordinary-fetch allowance cannot establish its live-fetch cost. This collector has no verified HTTP billing receipt; an account-specific live-fetch tariff is needed.
Methodology & data
Fresh pages. Inspired by NEEDLE, Trace collects URLs from changing public sources instead of relying on a fixed test set. Independent pages cover news, commerce, jobs, finance, research, government, software, and community sites.
Same task. Every reader receives the same selected URL with a 30-second deadline and a 250,000-character text limit. Their returned text, errors, and timings are saved. Each provider receives its fresh-fetch setting: maxAgeMs=0, maxAgeHours=0, maxAge=0, or live=true. Provider-reported cache evidence is tracked separately; origin freshness is not independently verified.
Content checks. The judge checks page identity and substantive content. The v3 protocol first checks a 12,000-character sample. If it finds no useful match, overlapping chunks cover the remaining returned text. A negative requires a completed full-text review; judge errors and ambiguous content remain unresolved. Earlier protocol results remain in the downloads and are excluded from v3 comparisons. If every unresolved result were useful, the score would rise by that count divided by the number of tests.
Scoring. A model grades the returned text without seeing the provider’s name. Factual accuracy has not been independently verified. Unresolved results stay in the denominator; timings include failed requests. The collected mix of sites affects the ranking. New runs compare fresh provider outputs directly without rendering a reference browser. Pairwise Jev preferences identify the best observed output, including ties. No useful output means unknown recoverability, not an impossible page. A comparison requires resolved content judgments for every provider, fresh-fetch requests, no reported cache hits and request starts within 120 seconds. Conflicting preferences remain unresolved. Providers with missing observations remain in the denominator. Entire batches invalidated by a collector fault are excluded for every provider, with reasons and raw results retained in the downloads.
Output comparison. Each useful output contributes up to 12,000 characters from the beginning, middle and end. Pairwise choices below 0.75 model confidence remain unresolved; this threshold is provisional. A winner must beat or tie every other useful output. This measures sampled preference, not factual accuracy or complete-page recall. Identical text ties without a model call; a sole useful response is reported separately.
Reproducibility. Each run freezes its URLs, provider set, configuration and code hashes before requests begin. Raw API responses and comparison receipts are retained privately. Historical browser captures remain in the archive. The public manifests and result hashes let others audit the selection and rerun the same URLs. Live pages can change.
Operated by Context, one of the providers being compared. Protocol · GitHub (private) · Download all data
Run receipts
- Sep 30, 2026 · 206 pages · needlecompleted · needle-funded-rerun-20260930
- Sep 30, 2026 · 3000 pages · independentcompleted · independent-funded-rerun-20260930
- Sep 30, 2026 · 206 pages · needlecompleted · needle-2026-09-30T043718.995385Z-75441a67
- Sep 30, 2026 · 3000 pages · independentcompleted · independent-20260930T043019.215901Z
- Sep 30, 2026 · 38 pages · needlecompleted · needle-2026-09-30T042534.076123Z-ae0017d8
- Sep 30, 2026 · 128 pages · independentcompleted · independent-20260930T041513.689199Z
- Sep 30, 2026 · 128 pages · independentcompleted · independent-20260930T035707.590283Z
Individual results
Individual results load when you reach this section.
Citation
If you use Trace in your work, please cite:
@misc{trace2026,
author = {Ryaboy, Michael and Bakour, Yahia},
title = {{Trace}: A Live Scraping Benchmark},
year = {2026},
url = {https://trace.2.28.61.111.sslip.io/},
note = {Contact: michael@context.dev, yahia@context.dev. Source repository currently private. Inspired by NEEDLE.}
}By Michael Ryaboy and Yahia Bakour at Context. Contact michael@context.dev or yahia@context.dev. GitHub repository (private; access required).
Inspiration
Trace is inspired by NEEDLE: A Live Open-Source Search Benchmark for AI Agents (2026), by Andrey Styskin, Matthias Petri and Ilya Gusev. Its fresh-query methodology and transparent comparisons informed this benchmark. Trace evaluates scraping; NEEDLE evaluates search.