Fixed-Size Chunking

Fixed-size chunking segments document content according to a configured length rather than identifying topic changes or constructing a hierarchy from headings. Length can be measured in characters, words, or tokens; token-based sizing is particularly useful for foundation-model applications because embedding and generation models impose token limits. Implementations may enforce exact windows or treat the configured size as a maximum while preserving sentence boundaries. The final chunk is commonly shorter than the configured size.

In a cloud retrieval pipeline, documents are extracted into text, normalized, segmented, embedded, and indexed. Each chunk becomes a searchable unit associated with its source document and metadata. At query time, the application retrieves relevant chunks and places selected passages into the foundation model's context. Chunking therefore determines what evidence the retriever can return independently; a relevant document is not enough if its individual chunks separate the information needed to answer the question.

For example, a custom splitter could create 500-token windows with 75 tokens of overlap. The first window covers token positions 1 through 500, and the next starts at position 426. This overlap helps preserve a definition or instruction that crosses a boundary. It also duplicates text across embeddings and retrieved passages, increasing index size and potentially wasting generation context. The stride is the chunk size minus the overlap, and overlap must remain smaller than the chunk size.

Managed ingestion systems can implement this strategy without custom splitting code. Amazon Bedrock Knowledge Bases supports fixed-size chunking configured with a maximum token count and an overlap percentage. AWS Lambda can implement custom segmentation when an application needs specific tokenization, normalization, metadata, or boundary behavior. A custom pipeline should use token counting appropriate to the embedding model rather than assuming a fixed number of characters equals a fixed number of tokens. For large files or processing that exceeds function execution limits, use an appropriately sized batch or container workload.

Smaller chunks can improve retrieval precision by isolating a specific fact, but may remove qualifications, headings, or surrounding explanation. Larger chunks retain more context but can combine unrelated topics and weaken the specificity of an embedding. A troubleshooting guide might work well with moderate fixed-size passages, while a legal policy could split an exception from the rule it modifies. Tables and source code also require care because arbitrary boundaries can separate labels from values or declarations from their use.

Fixed-size chunking is not semantic chunking, and adding overlap does not make it structure-aware. Hierarchical chunking instead maintains relationships between smaller retrieval units and larger parent sections, allowing retrieval of focused evidence with broader context. If questions routinely require an entire procedure or section, structure-aware or hierarchical processing may fit better than increasing the fixed window everywhere.

There is no universally optimal chunk size. Evaluate representative questions using retrieval recall, ranking quality, answer correctness, citation accuracy, latency, and cost. Keep the total prompt within the generation model's context budget, including instructions, history, retrieved passages, and reserved output space. Preserve document identifiers, source locations, version information, and access-control metadata on every chunk. Apply authorization filtering during retrieval; chunking itself neither enforces access control nor removes sensitive data. Version the splitter configuration and rebuild affected embeddings when segmentation changes so that stale and current chunks do not coexist unintentionally.