Engineering Atlasby Abhishek Goyal

Tag

Search & storage series

10 diagram-led guides on Search & storage series by Abhishek Goyal.

  1. GuideNew

    How Search Works in System Design: The Full Journey of a Query

    Follow one search query, "cheap flights goa", through 14 stops: browser, DNS, CDN, load balancer, app server, Redis cache, Elasticsearch coordinator, shards, inverted index, page cache and SSD, then back. Learn the latency numbers that matter, how a new record reaches the index (WAL, CDC, Kafka, refresh), what breaks, and how to present search in an HLD interview.

    ~20 min read · Beginner → Senior · 4 diagrams

  2. Deep diveNewMembers

    Disk I/O and Durability for System Design: Page Cache, fsync and Why "Saved" Isn't Saved

    What really happens between write() and the disk: the page cache, dirty pages, fsync and directory fsync, HDD vs SSD, journaling filesystems, the 2018 fsyncgate bug, O_DIRECT, epoll vs io_uring, and how grep, find, locate and plocate search, with the durability trade-offs to name in an HLD interview.

    ~12 min read · Beginner → Senior · 6 diagrams

  3. Deep diveNewMembers

    Data Structures for System Design: B-Tree, LSM Tree, Bloom Filter, Inverted Index, Geo and Vector Indexes

    The data structures behind real databases and search engines, explained for system design interviews: B-Tree vs LSM tree, Bloom filters, tries and FSTs for autocomplete, the inverted index, Roaring bitmaps, geohash, quadtree, S2 and H3 for location search, and HNSW for vector search, with when to pick each.

    ~12 min read · Beginner → Senior · 8 diagrams

  4. Deep diveNew

    Database Reads and Writes for System Design: WAL, B-Trees, LSM Trees, Caching, Replication and Sharding

    Why sequential appends beat random writes by 6,000x, and how that shapes the database write path and read path: write-ahead logs, B-Tree vs LSM tree, indexes, cache-aside vs write-through vs write-back, group commit, replication lag, range vs hash sharding, hot keys and CQRS, with the trade-offs to name in an HLD interview.

    ~24 min read · Beginner → Senior · 6 diagrams

  5. Deep diveNew

    How Search Engines Work: Inverted Index, Text Analysis, TF-IDF and BM25

    How a full-text search engine finds and ranks results: the inverted index, text analysis (tokenising, stemming, stop words), postings lists and boolean queries, TF-IDF and BM25 scoring, Lucene segments and merges, and near-real-time refresh, explained from the book-index analogy up to system design interview answers.

    ~25 min read · Beginner → Senior · 6 diagrams

  6. Deep diveNewMembers

    Types of Search in System Design: Autocomplete, Fuzzy, Phrase, Geo, Faceted, Vector and Hybrid

    The seven kinds of search behind one search box, for system design interviews: autocomplete with edge n-grams and FSTs, fuzzy search with Levenshtein automata, phrase search with positions, geo search with cells and BKD trees, faceted search and why counts are approximate across shards, vector search with HNSW, and hybrid search merged with Reciprocal Rank Fusion.

    ~16 min read · Beginner → Senior · 5 diagrams

  7. Deep diveNewMembers

    Elasticsearch for System Design: Shards, Replicas, Write Path and Read Path

    How Elasticsearch works inside, for system design interviews: cluster, nodes, shards and replicas, document routing, the write path (translog, refresh, flush, merge) and why search is near real-time, query-then-fetch scatter-gather, mappings, shard sizing, CDC and outbox sync from your database, and the tuning mistakes that hurt in production.

    ~25 min read · Beginner → Senior · 5 diagrams

  8. Deep diveNewMembers

    Choosing a Search Engine: Elasticsearch vs Postgres vs Redis vs OpenSearch, Solr and Typesense

    How to choose a search engine in system design: what a separate search cluster really costs, real full-text search in Postgres with tsvector, GIN and pg_trgm, Redis Search and MongoDB Search, the Lucene family (Elasticsearch, OpenSearch, Solr), Typesense, Meilisearch and Vespa, and a decision flow you can defend in an interview.

    ~14 min read · Beginner → Senior · 3 diagrams

  9. Deep diveNewMembers

    How Google Search Works: Crawling, Indexing, Ranking and Serving at Scale

    How Google Search works, explained for system design interviews: a polite web crawler with a URL frontier and robots.txt, the inverted index, PageRank with the random surfer, why Google shards its index by document, serving with caches and index servers, and the move from batch rebuilds to Caffeine and Percolator.

    ~12 min read · Beginner → Senior · 7 diagrams

  10. Deep diveNewMembers

    Search System Design Case Studies: Discord, GitHub, LinkedIn, Uber, Pinterest and More

    Real-world search system design case studies for HLD interviews: how Discord, GitHub, LinkedIn, Uber, DoorDash, Yelp, Instacart, Pinterest, Twitter, Slack, eBay and others scaled Elasticsearch, built their own Lucene engines, added vector and hybrid search, and ranked results, with the patterns that repeat.

    ~18 min read · Beginner → Senior · 7 diagrams

More tags