返回资讯中心
外部精选
软件工程
#开发者实时流

RAG: Coming together at the Seams

Retrieval-augmented generation (RAG) answers questions by searching a collection of documents and passing the best-matching passages to a language model. Before a document can be searched, it is cut into pieces called c…

Hacker News 最新adventure-time2026/11/10 · 7 天前30 分钟阅读

以下正文同步自 Hacker News 最新,版权归原站所有,已转换为易读排版。

Retrieval-augmented generation (RAG) answers questions by searching a collection of documents and passing the best-matching passages to a language model. Before a document can be searched, it is cut into pieces called chunks, and every cut is a place where an idea can be split in two. This paper describes that boundary problem, reviews how current chunking strategies try to manage it, and identifies the gap they leave: a fact that is split across two large chunks may not be found at all, because neither half resembles the question closely enough. It then introduces Seams, a chunking strategy that indexes a small, separate search vector over every chunk boundary and links it to both chunks it straddles. Seams is designed for long evidence in long chunks: conceptual material, where meaning develops over several paragraphs and chunks larger than 512 tokens are preferred. Seams keeps every chunk clean and disjoint, duplicates no text, works within a fixed context budget, adds to an existing index without rebuilding it, and has only two settings of its own. Published results at smaller chunk sizes found little benefit from keeping split evidence whole, so the paper confines its claim to this setting. This paper is a proposal: it sets out how Seams works, the conditions under which it applies, what it costs, and how its benefit can be measured, which is left to future work.

A retrieval-augmented generation system keeps its knowledge outside the language model, in an ordinary collection of documents, and looks up the relevant passages each time a question arrives. The founding RAG paper by Lewis and colleagues paired a document retriever with a text generator and used Wikipedia as its knowledge source, splitting every article "into disjoint 100-word chunks, to make a total of 21M documents"[1, p. 4]. The retriever it relied on, Dense Passage Retrieval (DPR), made the same choice and split each article "into multiple, disjoint text blocks of 100 words"[2, p. 4]. Later systems kept the pattern: In-Context RALM, which simply places retrieved passages in front of an unmodified language model, used "non-overlapping passages of 100 words"[3, p. 5].

Today the most common method is still to cut documents "on a fixed number of tokens (e.g., 100, 256, 512)," where a token is a word or a piece of a word[5, p. 8]. The trade-off is well described in the RAG survey by Gao and colleagues: "Larger chunks can capture more context, but they also generate more noise," while smaller chunks "may not fully convey the necessary context"[5, p. 8].

Every fixed cut is made without looking at the text, so some cuts land in the middle of an idea. The same survey states the problem directly: chunking "leads to truncation within sentences"[5, p. 8]. A 2026 study names it boundary fragmentation, "where critical evidence is split across segments," and reports that it lowers retrieval recall[11]. Anthropic's guidance on its Contextual Retrieval method likewise warns that "the choice of chunk size, chunk boundary, and chunk overlap can affect retrieval performance"[7, p. 9].

How often does this happen? A simple model helps. If a piece of evidence is L tokens long and sits at a random position in text that is cut every C tokens, the chance that a cut falls inside it is roughly L ÷ C. Figure 1 plots this model. It is an illustration, not a measurement, but it shows the key point: the risk is small for short facts and large for long evidence such as procedures, arguments and tables.

a b c

a: 50 tokens, 512-token chunks ≈ 10% · b: 200 tokens, 512-token chunks ≈ 39% · c: 200 tokens, 1,024-token chunks ≈ 20%

C = 256 C = 512 C = 1,024

Figure 1. An illustrative model of boundary risk: the chance that a fixed cut splits a piece of evidence is about L ÷ C, where L is the evidence length and C is the chunk size. Long, conceptual evidence is at much greater risk than short facts. This is a geometric model, not measured data, and it shows how often a cut falls inside evidence, not how often a split causes evidence to be missed (see Section 2.3).

1.3 How current strategies optimize chunking

Practitioners and researchers have developed several ways to manage chunk size and chunk boundaries. Table 1 summarizes the main families and what the evidence says about each.

Gap 1: a split fact can be lost before anything is fetched. Neighbour expansion and small-to-big retrieval both repair context after a chunk has been found. They cannot help when a fact is split so evenly that neither half resembles the question closely enough to be retrieved. Overlap does address this case, because it keeps the fact whole inside one chunk, but in a large chunk that fact is surrounded by hundreds of unrelated tokens. Chroma's authors speculate that relevant information can become "diluted within chunks"[9]. The argument that dilution makes a split fact hard to find is this paper's reasoning, built on that suggestion.

Gap 2: overlap pays for recall with duplicated text. Overlap places the same text in two chunks, so neighbouring chunks are often retrieved together and the duplicate fills slots that could hold different evidence. Figure 2 shows Chroma's measurements. Overlap helped recall for small 250-token chunks, but at 400 tokens it reduced the share of useful text in the results for both splitters tested[9]. Chroma measures that share as precision Ω, defined "for the case that all chunks containing excerpt tokens are successfully retrieved," which "gives an upper bound on token efficiency given perfect recall"[9]. Chroma also notes that "reducing chunk overlap improves IoU scores, as this metric penalizes redundant information"[9]. IoU (intersection over union) compares the relevant text with the retrieved text and counts every retrieved token, so duplicated text lowers it[9].

Gap 3: the larger the chunk, the more each repair costs. To guarantee that evidence of a given length is never split, overlap must be at least that long. For conceptual material with long evidence, that means a large fraction of every chunk is duplicated. Neighbour expansion instead sends whole extra chunks to the model, and with large chunks each one adds a great deal of text. That matters because "overly long contexts can also lead LLM to the 'Lost in the middle' problem," in which a language model attends mostly to the beginning and end of its input[5, p. 10]. The RAGAS authors make the same point about retrieved context that is too long[4, p. 3].

Gap 4: meaning-based cutting does not reliably fix boundaries. Semantic chunking tries to put cuts in the right places, but on real documents it did not beat fixed-size chunking, and its advantage was clearest when topics changed abruptly[8]. Its results are also hard to separate from chunk size, because the same study notes that "precision rises significantly as chunk size decreases"[8].

Gap 5: boundaries are rarely measured directly. Much of the evidence above concerns chunk size or chunking method, not boundaries themselves. A 2026 benchmark of ten retrieval strategies indexed each document "as a single unit without chunking," and lists chunking-strategy ablations as future work[6, pp. 3, 8]. SCAR is a notable exception, because it built a set of 160 boundary-fragmented questions specifically to measure this failure[11].

Seams is a chunking strategy that keeps a document's ordinary, non-overlapping chunks and adds one small search vector over every boundary between adjacent chunks. Each of these seam vectors embeds the text that straddles the cut and is linked to both chunks it joins. A seam is used only to find chunks: when a seam matches a question better than either chunk beside it, both of those chunks are passed to the language model, and the seam's own text is never shown to the model or cited.

The name reflects the idea. A chunked document is like a garment cut into panels, and the seams are where the panels meet. Ordinary retrieval looks only at the panels. Seams adds a way to search the joins, so that evidence lying across a join can be found and then read in full.

Seams builds on documented patterns rather than replacing them. LangChain's MultiVectorRetriever already "supports multiple embeddings per parent document," so that a match on any small vector returns the larger document it belongs to[12]. Small2Big already searches small units and returns their surroundings[5, p. 8]. Seams applies the same principle at exactly one place, the boundary, and gives each small vector two parents instead of one. The design is new, so the conditions below are reasoned from the evidence in Section 1.

The closest broader alternative is to index sliding windows over the whole document. The RAG survey lists a "sliding window approach" among the indexing refinements of advanced RAG[5, pp. 3, 8]. A multi-vector version would embed, for example, 256-token windows at a stride of 128 tokens and link each window to the chunk or chunks it overlaps. That design covers chunk interiors as well as boundaries, and Seams is a special case of it: one window per boundary, placed so that it straddles the cut. The difference is cost. Windows at a stride of s tokens add about C ÷ s vectors to every chunk of C tokens, which is 8 extra vectors per 1,024-token chunk at a 128-token stride, or about nine times the vectors of the chunks alone. Seams adds one vector per boundary, which is about twice the chunk-only index. The argument for indexing only the boundaries is that evidence lying inside a chunk is already whole, so it can be found by the chunk's own vector. The extra interior windows mainly address a different problem, the dilution of short facts inside long chunks. Seams provides the boundary coverage with less than a quarter of those vectors, and Section 3.5 includes sliding windows among the comparisons for measuring its benefit.

Some documents carry their meaning in short, self-contained facts, such as a product catalogue or a list of error codes. Others build meaning gradually, across several paragraphs: research papers, policy documents, legal agreements, design documents and technical procedures. In conceptual material like this, the answer to a question is often an argument or a sequence of steps rather than a single sentence.

The evidence suggests that such material is better served by larger chunks. In NVIDIA's chunking benchmark, datasets of factual-lookup questions did best with chunks of 256 to 512 tokens, while analytical datasets "generally benefited from larger chunk sizes (1,024 tokens) or page-level chunking"[10]. Chroma's authors observe that "chunks which are too small fail to capture necessary context within a single unit"[9], and the RAG survey notes that coarser retrieval units "can provide more relevant information for the problem," at the cost of extra noise[5, p. 7].

Large chunks make the boundary problem harder, not easier, for three reasons. First, the evidence in conceptual material is long, so by the model in Figure 1 a substantial share of it still straddles a cut: a 200-token argument has about a one-in-five chance of being split even with 1,024-token chunks. Second, each repair is expensive at this size, because overlap must duplicate long stretches of text and neighbour expansion must send whole extra chunks of 1,024 tokens or more. Third, the dilution described in Gap 1 is at its worst, because a split fact inside an overlapping 1,024-token chunk shares that chunk with roughly a thousand other tokens. Seams is designed for exactly this situation: it searches the boundary with a small vector, where a straddling fact is concentrated, and expands to the two large chunks only when that search succeeds.

Seams is the recommended strategy when conditions 1, 3 and 7 hold and most of the others do. Each condition follows from the gaps in Section 1.

Two of the sources this paper relies on cut against its premise, and they should be faced directly. DPR, working with 100-word passages and

正文由 探How 从来源站点 RSS 同步,内容未经改写;遇到排版缺失或需要图片、视频时请以原文为准。

打开原文
去社区讨论 →