Sharding

Sharding is horizontal data partitioning across independently managed storage or compute resources. Instead of storing every record on one database instance, a system distributes records among shards using a partitioning rule. A shard usually contains the same logical kinds of records as other shards but owns a different subset. Shards and servers are not necessarily one-to-one: a server can host several shards, and shard replicas can reside on other servers.

The partitioning rule must match the application's access patterns. Hash partitioning spreads records using a hash of a key, which generally helps distribute load. Range partitioning groups adjacent values, which can make time-window or ordered queries efficient but can concentrate new writes on one shard. Directory-based routing maintains a mapping between records or tenants and their locations. For example, routing orders by customer identifier keeps one customer's orders together, but an unusually large customer can become a hotspot.

Sharding is particularly useful when a single database writer cannot provide enough write throughput or when the dataset exceeds practical single-instance capacity. Read-intensive applications may first benefit from caching or read replicas without introducing sharding complexity. Replication copies the same data; sharding distributes different data. Replicas can improve availability and read capacity, but they do not inherently distribute writes to distinct portions of a dataset. Many architectures combine both techniques.

Queries that specify the shard key can target a small part of the system. Queries without suitable routing information may require scatter-gather execution: contacting many shards and merging their results. This can increase latency, network traffic, and sensitivity to slow shards. Cross-shard joins, uniqueness constraints, and transactions also require special consideration. Some databases support distributed transactions, but support and coordination costs vary; sharding does not inherently require eventual consistency or eliminate transaction guarantees.

For vector search, each shard holds a subset of vectors and its associated search index structures. A semantic search request may query several shards, obtain local candidates, and merge them into a final ranked result. More shards can enable parallelism and reduce the data held by each shard, but excessive sharding adds coordination, metadata, memory, and scheduling overhead. Approximate nearest-neighbor search also requires tuning candidate counts and search effort; distributing vectors does not automatically preserve the same recall and latency characteristics.

In Amazon OpenSearch Service, an index has primary shards and can have replica shards. Primary-shard count determines how the index's data is partitioned, while replicas provide additional copies for availability and search capacity. Engineers should size shards using measured vector dimensions, document counts, index structures, ingestion rates, concurrency, and available memory rather than assuming that more shards always improve performance. Changes to primary-shard layout require supported operations such as splitting or reindexing, subject to index and engine constraints.

Multiple indexes can separate specialized domains, such as legal documents and product manuals, allowing different mappings, retention policies, and retrieval settings. A hierarchical retrieval design can first select relevant domains or collections and then search their vector indexes. This reduces fan-out only when the first stage reliably selects the necessary data; incorrect routing can exclude relevant results. Logical domain separation is not automatically physical sharding, and a hierarchical retrieval pipeline is distinct from the graph hierarchy inside an HNSW vector index.

Operationally, monitor per-shard throughput, latency, storage, memory pressure, and throttling rather than relying only on cluster averages. Plan for growth, rebalancing, backups, recovery, and capacity during node failures. Adding nodes helps only if the system can redistribute shards or workload effectively. Routing must also preserve tenant authorization and required data placement: a shard key or separate index is not, by itself, an access-control boundary.