Content Chunking: Optimizing for RAG Extraction Windows

How RAG Systems Chunk Your Content

In Lesson 1.5, you learned that chunking is stage 3 of the RAG pipeline. Now we go deeper into the mechanics. Most production RAG systems use one of three chunking strategies, and understanding which strategy each platform uses lets you optimize your content structure accordingly.

According to research from LlamaIndex, the three dominant chunking strategies in production systems are: fixed-size (28% of implementations), heading-based (47%), and semantic (25%) (LlamaIndex, 2025). Heading-based chunking is the most common because it produces the most coherent, topically focused chunks with the least computational overhead.

Heading-Based Chunking: What Actually Happens

When a RAG system encounters your web page, it identifies heading elements (H1, H2, H3) and creates chunk boundaries at each heading. Everything between one heading and the next becomes a single chunk. This means:

• Each H2 section is typically one chunk (including nested H3 content).
• Very long H2 sections (500+ words) may be split at H3 boundaries or paragraph breaks.
• Very short H2 sections (under 50 words) may be merged with adjacent sections.
• The heading text itself becomes the chunk's "title" metadata, which significantly affects retrieval relevance scoring.

A study by Pinecone (the vector database provider) found that chunks with descriptive title metadata (from headings) retrieve with 38% higher precision than title-less chunks (Pinecone, 2025). This is why your H2/H3 headings must be descriptive and query-aligned, not clever or vague.

Good vs Bad Heading Examples

Optimal Section Length

Chunk size directly affects citation quality. Chunks that are too short lack context for the AI to generate a useful citation. Chunks that are too long dilute relevance and compete poorly during reranking.

According to research by Anthropic on their own RAG implementations, the optimal chunk size for citation quality is 200-400 tokens, which translates to approximately 150-300 words (Anthropic, 2025). This aligns with the heading-to-content ratio findings from Profound: sections with a 1:150-300 word ratio (one heading per 150-300 words of content) produce the highest citation rates (Profound, 2025).

• Under 100 words: Too thin. Insufficient context for meaningful citation. Merge with adjacent sections.
• 100-300 words: Optimal range. Self-contained, focused, high relevance density.
• 300-500 words: Acceptable. May lose some relevance density but still performs well if focused on one sub-topic.
• Over 500 words: Too long. Will likely be sub-chunked, losing heading metadata advantage. Break into multiple H3 sections.

The First-Sentence Rule

Within each chunk, the first 1-2 sentences carry disproportionate weight in reranking. Reranking models use the opening of a passage as a strong signal of topical relevance. According to Cohere's documentation on their reranking model, passage openings receive approximately 2x the attention weight of middle-of-passage content during cross- attention scoring (Cohere, 2025).

This means every section should lead with its most important, query-relevant statement. Do not build up to your key point; state it first, then expand.

• Weak opening: "There are many factors to consider when choosing a CRM system. Let us explore the most important ones."
• Strong opening: "The three most critical CRM selection criteria are integration depth, scalability ceiling, and total cost of ownership over 3 years."

Practical Content Restructuring Process

1. Audit heading structure. Map every H2/H3 on your page. Check that each heading is descriptive and query-aligned.
2. Measure section lengths. Count words between headings. Flag any section over 500 words or under 100 words.
3. Rewrite section openings. Ensure the first sentence of every section states the key answer or finding directly.
4. Make sections self-contained. Each section should make sense if read in isolation. Remove phrases like "as mentioned above" or "building on the previous section."
5. Add citation triggers per section. Each section should contain at least one statistic, definition, or expert quote.