Elasticsearch #

Elasticsearch is a distributed search and analytics engine built on top of Apache Lucene. Unlike a typical database, Elasticsearch is designed for one thing: finding data extremely fast and relevantly, even at the scale of billions of documents. It stores data in JSON format that’s fully indexed, so every field can become a search criterion without slow queries. In the TypeScript ecosystem, the official @elastic/elasticsearch client comes with comprehensive types, so you can write Elasticsearch queries with autocomplete and full type checking — minimizing errors that would otherwise only surface at runtime.

Installation #

npm install @elastic/elasticsearch

For TypeScript, no additional @types package is needed because the types are already included in the main package.

# verify the installation
npx tsc --version  # make sure TypeScript >= 4.5

Elasticsearch Basics #

Before writing code, it’s important to understand Elasticsearch terminology and its relational database equivalents:

Elasticsearch	Relational Database	Description
Index	Table	A collection of documents with similar data types
Document	Row	A single unit of data in JSON format
Field	Column	A property within a document
Mapping	Schema	The data type definition for every field
Shard	Partition	A slice of an index for data distribution
Replica	Backup	A copy of a shard for fault tolerance

The concept that most differs from a regular database is the inverted index — the way Elasticsearch stores data for searching. Instead of storing “document A contains word X”, Elasticsearch stores “word X exists in documents A, B, C”. This structure is what makes text search so fast.

flowchart LR
    A["Incoming document:\n'Laptop Gaming Asus'"] --> B[Analyzer]
    B --> C["Tokens:\n'laptop', 'gaming', 'asus'"]
    C --> D[(Inverted Index)]
    D --> E["'laptop' → doc1, doc5, doc9\n'gaming' → doc1, doc3, doc7\n'asus' → doc1, doc8"]

Connecting to Elasticsearch #

import { Client } from "@elastic/elasticsearch";

// connect to local Elasticsearch
const client = new Client({
  node: "http://localhost:9200",
});

// connect to Elastic Cloud or with authentication
const clientCloud = new Client({
  node: "https://my-deployment.es.us-east-1.aws.found.io",
  auth: {
    apiKey: process.env.ELASTIC_API_KEY!,
  },
  // or use username/password
  // auth: {
  //   username: "elastic",
  //   password: process.env.ELASTIC_PASSWORD!,
  // },
  tls: {
    rejectUnauthorized: false, // development only
  },
});

// verify the connection
async function cekKoneksi(): Promise<void> {
  try {
    const info = await client.info();
    console.log(`Connected to Elasticsearch ${info.version.number}`);
  } catch (error) {
    console.error("Connection failed:", error);
    throw error;
  }
}

For local development, run Elasticsearch via Docker: docker run -d --name elasticsearch -p 9200:9200 -e "discovery.type=single-node" -e "xpack.security.enabled=false" elasticsearch:8.13.0

Mapping and Index #

Mapping defines how every field in a document is stored and indexed. Defining the mapping explicitly — instead of letting Elasticsearch guess the types — is a must in production.

Field Types Often Used #

// field types in mapping:
// text      → full-text search, analyzed by an analyzer
// keyword   → exact match, for filtering/sorting/aggregation
// integer, float, double → numbers
// boolean   → true/false
// date      → date and time
// object    → a simple nested object
// nested    → an array of objects with independent queries
// geo_point → lat/lon coordinates

Creating an Index with Mapping #

interface Produk {
  nama: string;
  deskripsi: string;
  kategori: string;
  harga: number;
  stok: number;
  tags: string[];
  aktif: boolean;
  createdAt: string; // ISO 8601
}

async function buatIndexProduk(): Promise<void> {
  const indexAda = await client.indices.exists({ index: "produk" });

  if (indexAda) {
    console.log("Index 'produk' already exists");
    return;
  }

  await client.indices.create({
    index: "produk",
    body: {
      settings: {
        number_of_shards: 1,    // enough for development/small scale
        number_of_replicas: 1,
        analysis: {
          analyzer: {
            // custom analyzer for the Indonesian language
            analyzer_indonesia: {
              type: "custom",
              tokenizer: "standard",
              filter: ["lowercase", "asciifolding"],
            },
          },
        },
      },
      mappings: {
        properties: {
          nama: {
            type: "text",
            analyzer: "analyzer_indonesia",
            fields: {
              keyword: { type: "keyword" }, // for exact match & sort
            },
          },
          deskripsi: {
            type: "text",
            analyzer: "analyzer_indonesia",
          },
          kategori: {
            type: "keyword", // exact match, not analyzed
          },
          harga: { type: "float" },
          stok: { type: "integer" },
          tags: { type: "keyword" },
          aktif: { type: "boolean" },
          createdAt: {
            type: "date",
            format: "strict_date_optional_time",
          },
        },
      },
    },
  });

  console.log("Index 'produk' created successfully");
}

Indexing Documents #

“Indexing” in Elasticsearch means storing a document into an index — not creating an index like in a database. Every stored document is immediately available for search.

Store a Single Document #

async function simpanProduk(produk: Produk): Promise<string> {
  const hasil = await client.index({
    index: "produk",
    document: produk,
    // the id can be set manually or left to be auto-generated
    // id: "produk-001",
  });

  return hasil._id; // the ID generated by Elasticsearch
}

// with a predefined ID — useful for syncing from another database
async function simpanProdukDenganId(id: string, produk: Produk): Promise<void> {
  await client.index({
    index: "produk",
    id,
    document: produk,
    // refresh: "wait_for" — wait until the document is searchable before returning
    // use only in tests, not production (expensive)
  });
}

Bulk Indexing — Storing Many Documents at Once #

For storing many documents, always use the bulk API. Sending one document per request is a very expensive anti-pattern — every request has network overhead and an index refresh.

// ANTI-PATTERN: a loop with one index per item
async function simpanProdukSatuSatu(produkList: Produk[]): Promise<void> {
  for (const produk of produkList) {
    await client.index({ index: "produk", document: produk }); // ✗ N requests for N documents
  }
}

// CORRECT: use the bulk API
async function bulkSimpanProduk(produkList: Array<{ id: string; data: Produk }>): Promise<{
  berhasil: number;
  gagal: number;
}> {
  const operations = produkList.flatMap(({ id, data }) => [
    { index: { _index: "produk", _id: id } },
    data,
  ]);

  const hasil = await client.bulk({
    operations,
    refresh: false, // don't wait for refresh — let Elasticsearch refresh periodically
  });

  const gagal = hasil.items.filter((item) => item.index?.error).length;
  const berhasil = hasil.items.length - gagal;

  if (hasil.errors) {
    const errorItems = hasil.items
      .filter((item) => item.index?.error)
      .slice(0, 5); // log at most the first 5 errors
    console.error("Some documents failed to index:", errorItems);
  }

  return { berhasil, gagal };
}

Updating and Deleting Documents #

// partial update — only the included fields change
async function updateProduk(id: string, perubahan: Partial<Produk>): Promise<void> {
  await client.update({
    index: "produk",
    id,
    doc: perubahan,
    doc_as_upsert: false, // false = error if the document doesn't exist
  });
}

// update using a script — for atomic operations
async function tambahStok(id: string, jumlah: number): Promise<void> {
  await client.update({
    index: "produk",
    id,
    script: {
      source: "ctx._source.stok += params.jumlah",
      params: { jumlah },
    },
  });
}

// delete a document
async function hapusProduk(id: string): Promise<boolean> {
  try {
    await client.delete({ index: "produk", id });
    return true;
  } catch (error: any) {
    if (error?.meta?.statusCode === 404) return false;
    throw error;
  }
}

// delete by query
async function hapusProdukTidakAktif(): Promise<number> {
  const hasil = await client.deleteByQuery({
    index: "produk",
    body: {
      query: {
        term: { aktif: false },
      },
    },
  });
  return hasil.deleted ?? 0;
}

Query DSL — Searching Documents #

The Query DSL (Domain Specific Language) is the main way to interact with Elasticsearch. Every query is represented as a JSON object.

flowchart TD
    A[Query DSL] --> B[Query Context]
    A --> C[Filter Context]
    B --> D["Calculates a relevance score\nMore relevant documents = higher score\nExamples: match, multi_match, fuzzy"]
    C --> E["Yes or No — no score\nFaster & cacheable\nExamples: term, range, exists"]

Understanding the difference between query context and filter context is critical for performance. Use filter context as much as possible for binary conditions (active/inactive, price ranges, etc.), and save query context for text searches that need relevance ranking.

Match Query — Full-Text Search #

interface HasilCari<T> {
  hits: Array<{
    _id: string;
    _score: number;
    _source: T;
  }>;
  total: number;
}

async function cariProduk(keyword: string): Promise<HasilCari<Produk>> {
  const hasil = await client.search<Produk>({
    index: "produk",
    body: {
      query: {
        match: {
          nama: {
            query: keyword,
            fuzziness: "AUTO", // automatic typo tolerance
          },
        },
      },
    },
  });

  return {
    hits: hasil.hits.hits.map((hit) => ({
      _id: hit._id!,
      _score: hit._score ?? 0,
      _source: hit._source as Produk,
    })),
    total:
      typeof hasil.hits.total === "number"
        ? hasil.hits.total
        : (hasil.hits.total?.value ?? 0),
  };
}

// multi_match — search across several fields at once
async function cariMultiField(keyword: string): Promise<HasilCari<Produk>> {
  const hasil = await client.search<Produk>({
    index: "produk",
    body: {
      query: {
        multi_match: {
          query: keyword,
          fields: [
            "nama^3",       // 3x weight — more relevant if in the name
            "deskripsi^1",  // 1x weight
            "tags^2",       // 2x weight
          ],
          type: "best_fields", // take the best score from all fields
          fuzziness: "AUTO",
        },
      },
    },
  });

  return {
    hits: hasil.hits.hits.map((hit) => ({
      _id: hit._id!,
      _score: hit._score ?? 0,
      _source: hit._source as Produk,
    })),
    total:
      typeof hasil.hits.total === "number"
        ? hasil.hits.total
        : (hasil.hits.total?.value ?? 0),
  };
}

Bool Query — Combining Conditions #

The bool query is the way to combine several queries. It’s the most commonly used query in production because there’s almost always more than one search condition.

// must      → must match, affects the score
// should    → good if matches, boosts the score
// must_not  → must not match, doesn't affect the score
// filter    → must match, doesn't affect the score (faster)

async function cariProdukLanjutan(params: {
  keyword?: string;
  kategori?: string;
  hargaMin?: number;
  hargaMaks?: number;
  tags?: string[];
  halaman?: number;
  perHalaman?: number;
}): Promise<HasilCari<Produk> & { halaman: number; totalHalaman: number }> {
  const { keyword, kategori, hargaMin, hargaMaks, tags, halaman = 1, perHalaman = 10 } = params;
  const from = (halaman - 1) * perHalaman;

  const must: any[] = [];
  const filter: any[] = [];

  // full-text search goes into must (affects relevance)
  if (keyword) {
    must.push({
      multi_match: {
        query: keyword,
        fields: ["nama^3", "deskripsi", "tags^2"],
        fuzziness: "AUTO",
      },
    });
  }

  // binary conditions go into filter (don't affect the score, cacheable)
  filter.push({ term: { aktif: true } });

  if (kategori) {
    filter.push({ term: { kategori } });
  }

  if (hargaMin !== undefined || hargaMaks !== undefined) {
    const range: any = {};
    if (hargaMin !== undefined) range.gte = hargaMin;
    if (hargaMaks !== undefined) range.lte = hargaMaks;
    filter.push({ range: { harga: range } });
  }

  if (tags && tags.length > 0) {
    filter.push({ terms: { tags } }); // documents containing any of the tags
  }

  const hasil = await client.search<Produk>({
    index: "produk",
    from,
    size: perHalaman,
    body: {
      query: {
        bool: {
          must: must.length > 0 ? must : [{ match_all: {} }],
          filter,
        },
      },
      sort: keyword
        ? [{ _score: "desc" }]                         // if there's a keyword, sort by relevance
        : [{ createdAt: "desc" }],                     // otherwise, sort by newest
    },
  });

  const total =
    typeof hasil.hits.total === "number"
      ? hasil.hits.total
      : (hasil.hits.total?.value ?? 0);

  return {
    hits: hasil.hits.hits.map((hit) => ({
      _id: hit._id!,
      _score: hit._score ?? 0,
      _source: hit._source as Produk,
    })),
    total,
    halaman,
    totalHalaman: Math.ceil(total / perHalaman),
  };
}

Additional Queries Often Used #

// term — exact match for keyword fields
{ term: { kategori: "elektronik" } }

// terms — exact match against any of several values
{ terms: { kategori: ["elektronik", "komputer"] } }

// range — filter by a range of values
{ range: { harga: { gte: 100000, lte: 5000000 } } }
{ range: { createdAt: { gte: "2024-01-01", lte: "2024-12-31" } } }

// exists — documents that have a certain field
{ exists: { field: "deskripsi" } }

// wildcard — pattern matching (slow, avoid in production)
{ wildcard: { "nama.keyword": "*gaming*" } }

// prefix — documents whose field starts with a certain prefix
{ prefix: { "nama.keyword": "laptop" } }

Aggregation #

Aggregation in Elasticsearch lets you calculate statistics, build histograms, and group data — similar to GROUP BY in SQL but far more expressive. Aggregation runs on top of query results, so you can combine search and analytics in a single request.

interface AggregasiProduk {
  perKategori: Array<{
    key: string;
    jumlah: number;
    rataRataHarga: number;
    minHarga: number;
    maxHarga: number;
  }>;
  distribusiHarga: Array<{
    key: number;
    jumlah: number;
  }>;
  totalProdukAktif: number;
}

async function statistikProduk(keyword?: string): Promise<AggregasiProduk> {
  const hasil = await client.search({
    index: "produk",
    size: 0, // we only need aggregations, not documents
    body: {
      query: keyword
        ? { match: { nama: keyword } }
        : { match_all: {} },

      aggs: {
        // terms aggregation — group by field value
        per_kategori: {
          terms: {
            field: "kategori",
            size: 20, // at most 20 categories
            order: { _count: "desc" },
          },
          // sub-aggregation — calculate statistics inside every bucket
          aggs: {
            rata_rata_harga: { avg: { field: "harga" } },
            min_harga: { min: { field: "harga" } },
            max_harga: { max: { field: "harga" } },
          },
        },

        // histogram — distribution over value ranges
        distribusi_harga: {
          histogram: {
            field: "harga",
            interval: 1000000, // every 1 million
            min_doc_count: 1,
          },
        },

        // filter aggregation — count a specific subset
        produk_aktif: {
          filter: { term: { aktif: true } },
        },
      },
    },
  });

  const aggs = hasil.aggregations as any;

  return {
    perKategori: (aggs.per_kategori.buckets as any[]).map((bucket) => ({
      key: bucket.key,
      jumlah: bucket.doc_count,
      rataRataHarga: Math.round(bucket.rata_rata_harga.value ?? 0),
      minHarga: bucket.min_harga.value ?? 0,
      maxHarga: bucket.max_harga.value ?? 0,
    })),

    distribusiHarga: (aggs.distribusi_harga.buckets as any[]).map((bucket) => ({
      key: bucket.key,
      jumlah: bucket.doc_count,
    })),

    totalProdukAktif: aggs.produk_aktif.doc_count,
  };
}

Date Histogram — Time-Based Analytics #

async function trendProdukBaruPerBulan(): Promise<
  Array<{ bulan: string; jumlah: number }>
> {
  const hasil = await client.search({
    index: "produk",
    size: 0,
    body: {
      aggs: {
        per_bulan: {
          date_histogram: {
            field: "createdAt",
            calendar_interval: "month",
            format: "yyyy-MM",
            order: { _key: "asc" },
          },
        },
      },
    },
  });

  const aggs = hasil.aggregations as any;
  return (aggs.per_bulan.buckets as any[]).map((bucket) => ({
    bulan: bucket.key_as_string,
    jumlah: bucket.doc_count,
  }));
}

Highlight and Suggest #

Highlight — Mark Matching Words #

Highlight returns text snippets with matching words marked — a feature very useful for displaying search results to users.

async function cariDenganHighlight(keyword: string) {
  const hasil = await client.search<Produk>({
    index: "produk",
    body: {
      query: {
        multi_match: {
          query: keyword,
          fields: ["nama", "deskripsi"],
        },
      },
      highlight: {
        pre_tags: ["<mark>"],   // opening highlight tag
        post_tags: ["</mark>"], // closing highlight tag
        fields: {
          nama: { number_of_fragments: 0 },        // show the whole nama field
          deskripsi: {
            number_of_fragments: 2,  // at most 2 fragments
            fragment_size: 150,       // each fragment's length (characters)
          },
        },
      },
    },
  });

  return hasil.hits.hits.map((hit) => ({
    id: hit._id,
    produk: hit._source,
    highlight: {
      nama: hit.highlight?.nama?.[0],
      deskripsi: hit.highlight?.deskripsi?.join(" ... "),
    },
  }));
}

Autocomplete Suggest #

// for autocomplete, use the completion suggester
// its mapping must be defined first:
// suggest: { type: "completion" }

async function autocomplete(prefix: string): Promise<string[]> {
  const hasil = await client.search({
    index: "produk",
    body: {
      suggest: {
        nama_suggest: {
          prefix,
          completion: {
            field: "suggest", // a field with type "completion" in the mapping
            size: 5,
            skip_duplicates: true,
          },
        },
      },
    },
    _source: false, // no need for the document source, only suggestions
  });

  const suggest = hasil.suggest as any;
  return (suggest?.nama_suggest?.[0]?.options ?? []).map(
    (opt: any) => opt.text as string
  );
}

Integration Pattern: Elasticsearch as a Search Layer #

Elasticsearch rarely stands alone as the primary database. The most common pattern uses a relational database or MongoDB as the source of truth, with Elasticsearch as a search layer that’s filled in synchronously.

sequenceDiagram
    participant Client
    participant API
    participant DB as Primary Database
    participant ES as Elasticsearch

    Client->>API: POST /produk (create)
    API->>DB: INSERT INTO produk
    DB-->>API: new document id
    API->>ES: index the document into ES
    ES-->>API: acknowledged
    API-->>Client: 201 Created

    Client->>API: GET /produk/search?q=laptop
    API->>ES: search query
    ES-->>API: search results + IDs
    API-->>Client: search results

// a service that keeps the DB and Elasticsearch in sync
class ProdukService {
  constructor(
    private readonly db: any, // primary database connection
    private readonly es: Client
  ) {}

  async buat(data: Omit<Produk, "createdAt">): Promise<string> {
    // 1. save to the primary database first
    const id = await this.db.insert("produk", {
      ...data,
      createdAt: new Date().toISOString(),
    });

    // 2. index into Elasticsearch
    // if ES fails, the document still exists in the DB — can be retried
    try {
      await this.es.index({
        index: "produk",
        id,
        document: { ...data, createdAt: new Date().toISOString() },
        refresh: false,
      });
    } catch (error) {
      console.error(`Failed to index product ${id} into Elasticsearch:`, error);
      // save to a retry queue (Redis, RabbitMQ, etc.)
    }

    return id;
  }

  async cari(params: { keyword?: string; kategori?: string }) {
    // search always goes through Elasticsearch
    return cariProdukLanjutan(params);
  }

  async ambilById(id: string): Promise<Produk | null> {
    // get by ID can go directly to the primary database (more consistent)
    return this.db.findById("produk", id);
  }
}

Don’t use Elasticsearch as the primary database. Elasticsearch doesn’t guarantee data consistency like an ACID database — a newly indexed document isn’t necessarily immediately searchable (depends on the refresh interval, 1 second by default). Always use Elasticsearch as a search layer on top of a more consistent primary database.

Error Handling #

import { errors } from "@elastic/elasticsearch";

async function cariAman(keyword: string): Promise<HasilCari<Produk>> {
  try {
    return await cariProduk(keyword);
  } catch (error) {
    if (error instanceof errors.ResponseError) {
      // an error from Elasticsearch (4xx or 5xx status code)
      const statusCode = error.meta.statusCode;

      if (statusCode === 404) {
        // index not found
        throw new Error(`Index 'produk' not found. Make sure it has been created.`);
      }

      if (statusCode === 400) {
        // invalid query
        const detail = error.meta.body?.error?.reason ?? "Query tidak valid";
        throw new Error(`Query error: ${detail}`);
      }

      if (statusCode === 429) {
        // too many requests — Elasticsearch is overloaded
        throw new Error("Elasticsearch sedang sibuk, coba lagi sebentar");
      }
    }

    if (error instanceof errors.ConnectionError) {
      throw new Error("Tidak bisa terhubung ke Elasticsearch");
    }

    if (error instanceof errors.TimeoutError) {
      throw new Error("Request ke Elasticsearch timeout");
    }

    throw error;
  }
}

// re-index — useful for resyncing if ES lags behind the DB
async function reindexSemua(
  ambilSemua: () => AsyncGenerator<Array<{ id: string; data: Produk }>>
): Promise<void> {
  let totalBerhasil = 0;
  let totalGagal = 0;

  for await (const batch of ambilSemua()) {
    const { berhasil, gagal } = await bulkSimpanProduk(batch);
    totalBerhasil += berhasil;
    totalGagal += gagal;
    console.log(`Progress: ${totalBerhasil} succeeded, ${totalGagal} failed`);
  }

  console.log(`Reindex done: ${totalBerhasil} documents`);
}

When to Use Elasticsearch #

Elasticsearch isn’t a solution for every problem. Understanding its limits is as important as understanding its capabilities.

Use Elasticsearch if:
  ✓ The application needs relevant full-text search (e-commerce, blogs, news portals)
  ✓ Need search with typo tolerance (fuzziness)
  ✓ Need highlighting of matching words in search results
  ✓ Need real-time analytics on continuously incoming data (logs, events)
  ✓ Autocomplete and suggestions on the search field
  ✓ Very large data scale needing horizontal scaling for search

Don't use Elasticsearch as a replacement if:
  ✗ Data requires strong ACID consistency — use PostgreSQL/MySQL
  ✗ Simple queries like "find by ID" or "filter by status" — overkill
  ✗ Data has complex relationships requiring joins — use a relational database
  ✗ Need atomic multi-document transactions — Elasticsearch doesn't support this
  ✗ The team lacks capacity to maintain a cluster — consider PostgreSQL full-text search

Summary #

Elasticsearch isn’t the primary database — use it as a search layer on top of a more consistent database (PostgreSQL, MongoDB). The primary database is the source of truth.
Define the mapping explicitly before storing data. Elasticsearch’s automatic mapping is often suboptimal — text vs keyword must be chosen deliberately according to query needs.
Query context vs filter context — use filter for binary conditions (active/inactive, ranges, exact matches) because it’s faster and cacheable; use must only for conditions affecting the relevance score.
The bool query is the foundation — almost every production query uses a combination of must, filter, should, and must_not inside a bool query.
The bulk API is a must for mass indexing — never loop index() one by one; send in batches using bulk() for far better performance.
Aggregation replaces SQL GROUP BY — take advantage of terms, histogram, date_histogram, and sub-aggregations for analytics reports directly from Elasticsearch.
Highlight and fuzziness are features that immediately improve UX — show matching words with highlights and typo tolerance with fuzziness: "AUTO".
Catch errors.ResponseError for server errors and errors.ConnectionError for network issues — both need different handling.

← Previous: MongoDB Next: Redis →