Content Chunking and Passage Retrieval

Chunking and passage retrieval, defined

Chunking is the step in a retrieval pipeline where a document is cut into smaller units before it is indexed, so the system can return a relevant fragment instead of a whole file. Passage retrieval is the read side of the same idea: matching a query against those fragments and returning the ones that answer it. Both belong to information retrieval, the discipline concerned with finding material that satisfies a need from within a large collection.

In a vector-based system each chunk is converted into an embedding — a numeric representation of its meaning — and stored; at query time the query is embedded too and the nearest chunks come back. In a classic web search engine the corresponding idea is passage ranking: scoring individual sections of a page rather than only the page as a whole.

The consequence practitioners care about is genuine. If the retrievable unit is a passage rather than a URL, what matters is whether some span of your page answers a question on its own, not whether the page as a whole covers the topic. A page that addresses a question across four scattered sections may contain no span that answers it. A shorter page carrying one clean, self-contained passage may.

That consequence is real. Almost everything built on top of it — recommended word counts, chunk sizes, section lengths, answer-in-N-words targets — is documented by nobody, and the gap between those two sentences is the subject of this page.

Two mechanisms that get treated as one

The first source of confusion is that passage ranking and chunked retrieval are different systems doing different jobs, and advice derived from one and applied to the other will be wrong in both directions.

Passage ranking in a web index. Google indexes the page. A ranking system additionally scores sections within it, so a page can rank for a query answered in one buried paragraph even where the page overall is about something broader. Google's complete public description of it, from its ranking systems guide stamped 10 December 2025, is a single sentence:

Passage ranking is an AI system we use to identify individual sections or 'passages' of a web page to better understand how relevant a page is to a search.

Read the last clause carefully. The passage is used to understand how relevant the page is. The unit that ranks is still the page. This is not chunked retrieval.

Chunked vector retrieval. The document is cut into fragments, each fragment is embedded and stored independently, and retrieval returns fragments. Here the unit genuinely is the chunk, and the document survives only as a pointer attached to it. Enterprise retrieval-augmented generation works this way without ambiguity.

Google's AI Overviews and AI Mode are grounded on Google's web index, which is passage-ranking territory with a generative layer on top. Google has never described chunking in any AI-features documentation. Which mechanism governs an assistant that fetches a live URL is undocumented, and the honest answer is that a fetched page is converted to text and placed in a context window — which is neither of the two, but a third thing: whole-document reading under a token budget.

Practitioners routinely assert that AI Overviews retrieve chunks. The inference is reasonable, because that is how the enterprise systems they have built work. It is also undocumented, and it should be labeled that way.

Google's instruction is the opposite of the advice being sold

In its guide to optimizing for generative AI features, published 15 May 2026, Google includes a section it titled Mythbusting generative AI search: what you don't need to do — a list of things publishers are being told to do that Google says are unnecessary. One entry addresses this subject directly:

There's no requirement to break your content into tiny pieces for AI to better understand it.

Google adds that its systems are able to understand the nuance of multiple topics on a page. The only structural guidance it does offer is that people generally appreciate web pages organized by paragraphs and sections, with headings giving content a clear structure.

The framing is worth dwelling on. The justification Google gives for structure is that people appreciate it. In a document written to tell publishers how to optimize for machines, Google declines to make a machine-facing argument for sectioning at all. Practitioners are entitled to observe that format-based chunking respects existing document structure, and that headings therefore plausibly produce better-bounded fragments. That second half is inference. Keeping it separate from the first half is the difference between a credible page and a confident one.

Google naming something a myth is about as direct a refutation as this field produces. An entire genre of content — structure your pages for AI chunking, keep sections short so the machine can parse them — is being sold against an explicit statement that it is not required.

Where the recommended chunk sizes came from

Keep sections to 200 to 300 words. Answer in 40 to 60 words. Chunk at 500 tokens. These numbers circulate as though they were measurements of a search product. They are not, and their origin is traceable.

They come from default parameters in retrieval libraries — LangChain, LlamaIndex and their equivalents — and from agency posts citing each other. A library default is a configuration value chosen by a framework author to be a reasonable starting point for a developer building a private document search. It measures nothing whatsoever about Google, OpenAI, Anthropic, Perplexity or Microsoft.

No operator has published a chunk size, an overlap, a split strategy, an embedding model, a count of how many fragments enter a context window, or a reranker. Not one. Any recommendation phrased as keep sections under N words because that is how AI chunks content is stating a number no operator has published.

It may still be reasonable writing advice. Short, well-bounded sections are often better prose than long undifferentiated ones. But the justification is editorial, not technical, and presenting a library default as a platform specification is how the generative engine optimization market — GEO, the practice of trying to influence what AI answer systems cite — has been manufacturing false precision since 2024.

Does a smarter split retrieve better? One study says no

The whole structure-your-content-for-chunking genre rests on an unstated assumption: that better-bounded chunks retrieve better. Somebody tested it.

Qu, Tu and Bao published Is Semantic Chunking Worth the Computational Cost? on 16 October 2024. Semantic chunking splits a document at detected topic boundaries rather than at fixed intervals — the sophisticated version of the idea, and the one that most closely resembles what careful sectioning is supposed to achieve. They evaluated it against plain fixed-size chunking across three retrieval tasks: document retrieval, evidence retrieval and answer generation. Their conclusion was that the computational costs associated with semantic chunking are not justified by consistent performance gains.

That finding deserves far more attention from practitioners than it gets. In the one controlled comparison available, the smarter split did not reliably beat the dumb one. It is a general retrieval finding rather than a finding about any commercial search product, and that distinction has to travel with it — but it points away from sophistication, and it is the only published result pointing anywhere at all.

The comparison worth setting against it is the other direction of the same literature. Karpukhin and colleagues, in Dense Passage Retrieval for Open-Domain Question Answering on 10 April 2020, found that dense retrieval over passages beat a strong Lucene-BM25 system by 9% to 19% absolute on top-20 passage retrieval accuracy. That is a benchmark result on open-domain question answering, not a statement about any web index, and production systems generally run hybrid retrieval — dense plus sparse — precisely because neither alone is dependable. The literature supports passages as a unit. It does not support any particular way of cutting them.

The one thing that transfers cleanly to writing

Whatever the split strategy, a retrieved fragment is read without the rest of the page. Everything useful on this subject follows from that sentence, and none of it requires knowing anyone's chunk size.

A paragraph that opens as we saw above, this depends on several factors is uninterpretable alone. A paragraph that opens renewing a UK passport takes about three weeks by standard service is not. Each substantive passage should be independently intelligible — which is also, and separately, just good writing.

Two corollaries have real teeth:

• Antecedents kill passages. It, this, they and the above all resolve against text that may not travel with the fragment. Naming the subject again costs a few words and preserves the meaning of the span.
• Put the answer near the top of its own section. If a fragment is truncated at a token boundary, the front survives. A section that builds to its conclusion loses the conclusion.

This is the strongest practical advice available on the subject, and it should still carry an honest label. It follows from how every documented retrieval architecture works and it is consistently reported by practitioners, but no operator has confirmed it and no published experiment isolates it on a live search product. It is a well-reasoned, undocumented default. Saying so costs nothing.

What the silence means for anyone giving advice

The uniform silence across every operator is the single most important fact on this page, and it dictates how the subject should be discussed.

Google describes passage ranking in one sentence and describes nothing about how AI Overviews or AI Mode segment text. OpenAI, Anthropic, Perplexity and Microsoft all publish crawler documentation and none describes how a fetched or indexed document is divided for retrieval. Nothing about segmentation is measurable from outside, which means no claim about it is falsifiable from outside either — and unfalsifiable claims are exactly what an advice market fills a silence with.

A short list of errors follows directly:

• Quoting a library default as though it were Google's chunk size. It measures a framework's configuration, nothing more.
• Fragmenting a good page into micro-sections. Google explicitly lists this under what you do not need to do. It costs readability and buys nothing documented.
• Assuming thoroughness guarantees retrievability. A page that covers a topic completely across four sections may still contain no single span that answers a specific question about it.
• Opening a section with a back-reference. The most common way to make an otherwise good passage useless out of context.
• Presenting the self-contained-passage argument as documented. It is not. It is sound reasoning from published architecture, and labeling it accurately is what separates a reference from a pitch.

There is one last control question worth answering, because people keep asking it. There is no markup that makes a passage the extracted answer. Google's featured snippet documentation, stamped 10 December 2025, answers the question of how to mark up a passage for extraction with two words — you can't — and notes that its systems determine whether a page would make a good featured snippet and elevate it. The only publisher controls are suppressive: nosnippet, data-nosnippet and max-snippet. You can remove yourself from extraction. You cannot volunteer for it.

Frequently asked questions