Does an answer-first opening change how often we're cited?
Sharpen one article's opening into a clean, extractable answer; hold a comparable article's opening unchanged; watch what moves. Running since 2026-06-08 — and honest up front that our current instrumentation can't fully measure the thing this targets.
- Hypothesis
- Rewriting an article's opening so the core answer is stated first, in standalone quotable sentences, increases the chance an LLM extracts and cites it — visible as more crawler re-fetches of the treatment page and, weakly, in site-wide citation movement.
- What changed
- Treatment article's first paragraph is rewritten answer-first: the key takeaway in 1–2 standalone sentences, no throat-clearing, dense with concrete dated facts. Control article's opening is left unchanged. Body below the fold is untouched on both.
- Treatment
- Metric
- Mixed
- Window
- 8 June 2026 → 8 July 2026
Status: running since 2026-06-08; review 2026-07-08. The editor selected and paired this experiment. The treatment page's opening has been rewritten answer-first; the control page is held byte-for-byte unchanged for the window.
The question
If we open an article with the answer instead of a wind-up, do LLMs cite it more? Answer-first structure is the most repeated piece of GEO advice — including in our own playbook. Before we keep repeating it, we test it on a page we control.
The design
- Treatment: the opening of JSON-LD recipes for GEO was rewritten to state the core answer first, in clean standalone sentences — replacing a soft "you don't need to remember the whole spec" lead-in. The body below the fold is untouched.
- Control: the llms.txt spec article opening is left unchanged.
- Primary metric: AI-crawler re-fetch frequency per page (free, page-level), treatment vs control.
- Window: the before window is whatever per-page crawl history exists (short — tracking began ~2026-05-31); the review is 2026-07-08.
Why these two pages
Both are technical articles, so page type is held roughly constant. They differ in kind — the treatment is a spoke recipe page, the control a cornerstone audit — which is disclosed as a residual confounder, as is a minor co-intervention: the treatment gained a new inbound link from the just-published geo-methods-taxonomy on the same day. Both are disclosed above; on this short, near-zero before-window the honest expectation is an early inconclusive.
The honest catch (and the parked half)
This experiment targets citation, and our citation harness measures the whole domain, not a single page. So we cannot cleanly answer "did this rewrite get this page cited more." The analyzer says so: for a page-level citation question on site-wide data, it returns inconclusive and attaches only the site-wide trend as context. That citation half is also parked — it needs LLM-API spend, which is off; no harness was run for this launch, and the site-wide context will only be attached at review if/when API spend is re-enabled.
What we can measure per page, for free, is crawler re-fetch frequency — a proxy for "the bots noticed a change worth re-reading." We treat that as the readable signal and are explicit that it is a proxy.
What would count as a result
A sustained rise in crawler re-fetches of the treatment page that the control does not share, ideally alongside (not proven by) a site-wide citation uptick. We will not claim the rewrite "raised citations" on site-wide data alone.
Analysis readout
InconclusiveNo defensible read yet for exp-003: Too few crawler hits on treatment pages (13 across both windows; need ≥ 20).
| Page set | Before | After | Δ |
|---|---|---|---|
| treatment | 8 | 5 | -37% |
| control | 14 | 4 | -71% |
Caveats (2)
- Quasi-experiment: a before/after observation on a small page corpus, not a randomized controlled trial.
- AI-crawler and citation signals are noisy and lag the change by days to weeks; a short window can mislead.
Limitations
- The primary intended outcome — citation/extraction of this specific page — is NOT page-attributable with our current harness, which measures the whole site. We use crawler re-fetch frequency as the page-level proxy and treat citation as site-wide context only. That citation half is DEFERRED until LLM-API spend is re-enabled — no harness was run for this launch.
- A dek/opening rewrite is a content improvement we'd want anyway, so a 'null' crawl result does not mean the rewrite was pointless — only that it didn't move the proxy metric.
- Both pages are same-type (technical), which holds page type roughly constant; but they differ in kind (the treatment is a spoke recipe page, the control a cornerstone audit), a residual confounder.
- Minor co-intervention disclosed: the treatment page (json-ld-recipes) gained a new inbound link from the newly-live geo-methods-taxonomy on 2026-06-08, which could nudge its crawl discovery independently of the opening rewrite.
- The before-window is short (per-page crawl tracking began ~2026-05-31). Per-page hits are now attributed exactly from full per-path daily counts for the watchlisted pages — no longer capped at a crawler's daily top-10 — so a low-traffic page is not undercounted to zero. The residual limit is days of data, not attribution: a short before-window can still read inconclusive.
- Single domain, single matched pair. Disjoint from exp-002 (different pages) so the two do not confound each other.
Pages in this experiment
Changelog
- Published — 31 May 2026
Raw markdown: /lab/experiments/exp-003.md