Elasticsearch for System Design: Shards, Replicas, Write Path and Read Path

Elasticsearch calls itself "near real-time". That little word "near" hides a lot: a 1-second refresh, a write-ahead log, and a background merge process that quietly rewrites your data on disk. If you have ever added a product, searched for it straight away and seen nothing, you have met that word. Let's open the box: where a document goes when you save it, what happens when you search, and which four settings cause most of the pain in production.

We focus on Elasticsearch concepts that also apply to OpenSearch, its open-source fork, and skip security, Kibana, lifecycle management and pricing.

01The pieces: cluster, node, index, shard, replica

Picture a chain of libraries. The whole chain is the cluster. Each branch is a node. A collection, say "all fiction", is an index. It is too big for one room, so it is split into shelf sections spread across branches; each section is a shard. And so a fire in one branch doesn't destroy a section, the chain keeps a photocopy of every section in another branch: a replica.

Now the real definitions, one line each:

• Cluster: Elasticsearch servers working together with one shared view of the data.
• Node: one Elasticsearch process, usually one machine or container.
• Index: a named collection of documents, like products. Roughly what a table is in a database.
• Document: one JSON object you store and search, like one product.
• Primary shard: the slice of an index that accepts writes for its documents first.
• Replica shard: a full copy of a primary shard, kept on a different node.
• Replication group: a primary plus all of its replicas.
• Coordinating node: whichever node received your request; it routes the work and collects answers.

Nodes also have roles: master-eligible (manages the cluster state, meaning which indices exist and which shard lives where), data (holds shards, does the heavy work), ingest (runs pre-processing pipelines), and every node also acts as a coordinator. In a small cluster, every node does everything, and that is fine.

Here is a three-node cluster holding one index, products, with three primary shards and one replica each.

Notice the placement rule. A replica never lives on the same node as its primary. A copy in the same building doesn't protect you from that building burning down. Notice also that your client can talk to any node; that node becomes the coordinator for the request. Elastic calls this a primary-backup model, and its reading and writing documents page is the reference for the write and read paths below.

02Document routing: which shard gets my document?

Every document belongs to exactly one primary shard. When you save product p-42, the coordinator must pick the same shard every time, or you could never find it again. The trick is a hash: take a routing value (by default the document's _id), hash it, and take the remainder after dividing by the number of primaries. Like a coat check that hangs your coat on hook "ticket number mod 3": same ticket, same hook.

shard_num = hash(_routing) % num_primary_shards
# _routing defaults to the document _id

One caveat: the _routing field docs say this exact formula applies to indices created in Elasticsearch 9.4.0 and later. Older indices use a legacy variant that hashes into a fixed number of "routing shards" first. The idea is the same.

Here's the catch. Change the number of primaries and the remainder changes, so almost every document would belong to a different shard. That's why index.number_of_shards can only be set at index creation time. To get more primaries later, you create a new index with the Split API or a reindex. Replica count, by contrast, you can change any time.

Custom routing

You can set the routing value yourself, for example to a tenant ID. Then all of one tenant's documents land on one shard, and a search with the same routing value hits that one shard instead of all of them. The risk is the hot shard: one giant tenant makes one shard huge and busy while the others sit idle.

Discord does a version of this in its own application code, routing by server (a "guild") across many small clusters. In How Discord Indexes Trillions of Messages, they report that a single primary shard with no fan-out is fastest for most guilds; only the very largest get dedicated multi-shard indices.

03The write path: buffer, translog, refresh, flush, merge

This is the heart of the blog. Let's follow one index request (saving one document) all the way down. First, five quick definitions:

• Translog: a per-shard append-only log of every write, kept so acknowledged writes survive a crash.
• Refresh: turning the in-memory buffer into a new searchable segment.
• Flush: a Lucene commit (segments fsynced to disk) plus starting a fresh translog.
• Merge: combining small segments into bigger ones in the background.
• In-sync copies: the replicas the primary currently trusts to be up to date.

Here is the whole journey in one picture. Read the numbers in order.

1. Client to coordinator to primary. The coordinator uses routing to find the primary shard and forwards the request there.
2. Primary indexes and logs. The primary validates the request, indexes the document into Lucene's in-memory buffer, and appends the operation to its translog.
3. Primary forwards to replicas. It sends the operation to every in-sync replica in parallel. Each does the same: buffer plus translog.
4. Translog fsync, then ack. With the default durability setting, request, Elasticsearch only reports success after the translog has been fsynced on the primary and on every allocated replica. Now your write is durable.
5. Refresh. About once a second, the buffer is written out as a new segment into the filesystem cache and opened for search. No fsync here. Now your write is searchable.
6. Flush. Every so often, a real Lucene commit fsyncs the segments, and a new translog generation starts so the old one can be trimmed.
7. Merge. In the background, small segments are merged into bigger ones, and deleted documents are finally dropped.

Why the translog exists

A Lucene commit writes and fsyncs whole segment files. Doing that per document would crush throughput. The translog settings page says it plainly: commits are too expensive for every change, so each shard copy writes operations into a translog before acknowledging them, and replays it after a crash. That is a write-ahead log, and Elastic's Elasticsearch from the Top Down calls it exactly that. Cheap append on every write, expensive commit occasionally.

The numbers that matter

• Durability. index.translog.durability defaults to request: fsync before the ack. async fsyncs every sync_interval (default 5 seconds) instead: faster, but a crash can lose writes acknowledged since the last sync.
• Translog size. Background flushes keep it small so recovery stays fast: by default a flush triggers at 10 GB of unflushed operations, and the docs state the translog never exceeds 1% of the disk size.
• Refresh. By default, Elasticsearch refreshes every 1 second, but only on indices that received at least one search in the last 30 seconds. That rule, from the near real-time search page, is the whole meaning of "near".

Safe versus searchable

Here's the idea to walk away with. "Durable" and "searchable" are two different moments. After step 4, the document is safe on disk in every in-sync translog, but in no segment, so search can't see it. After step 5, it is searchable, though the new segment sits only in the filesystem cache. That's fine: the translog covers it until the next flush.

So when your product doesn't show up right after saving, nothing is broken. You searched in the gap between the ack and the next refresh. If one request truly needs to see its own write, the index API's refresh=wait_for option holds the response until the next refresh. Use it rarely: forcing refreshes creates lots of tiny segments.

Merges, the quiet rewriter

Segments are immutable, so an update is really "mark the old version deleted, write the new one into a new segment". Every refresh adds another small segment, and every search must visit all of them. So Lucene keeps merging small segments into bigger ones, dropping deleted documents along the way, with IO throttling so merges don't starve searches (see merge settings). This is the "quietly rewrites your data" from the hook.