Migrating Products to Elasticsearch: Powering Faceted Search at Scale

This is Part 4 of the "Building a Scalable, Faceted Online Marketplace" series. Read the Introduction here.

Why Elasticsearch?

While MongoDB is great for flexible storage and rapid prototyping, true large-scale faceted search demands a search engine built for the job. Elasticsearch is the industry standard for e-commerce search, powering giants like Amazon and Walmart.

In this article, we'll walk through the process and tooling for migrating millions of products from MongoDB (or your data generator) into Elasticsearch, ready for lightning-fast, dynamic faceted queries.

The Migration Tool: `migrate_products_to_elasticsearch.js`

Streams product data from NDJSON, MongoDB, or another source
Transforms and maps product fields for optimal ES indexing
Handles batching, error retries, and progress reporting
Supports dynamic attributes and category-aware mappings

Key Features

Bulk Indexing: Uses Elasticsearch's bulk API for speed
Dynamic Mapping: Handles category-specific fields and nested attributes
Resumable: Can pick up where it left off after a failure

Example Usage

node tools/migrate_products_to_elasticsearch.js --file ./out/products.computers.00000.ndjson.gz --es http://localhost:9200 --index products

Real-World Tips

Always test your mapping on a small dataset before bulk import
Monitor ES cluster health and disk space during migration
Use gzip/NDJSON for efficient streaming

Next up: Faceted Search APIs: Dynamic, Category-Aware, and Scalable (on Elasticsearch)