OpenSearch in Production
OpenSearch is what happens when AWS needs a search engine it can ship as a managed service without worrying about license compatibility. When Elastic changed Elasticsearch to SSPL in January 2021, AWS forked 7.10.2 under Apache 2.0, renamed it OpenSearch, replaced Kibana with OpenSearch Dashboards, and has been maintaining it independently since. The fork is now over four years old, the project is governed by the OpenSearch Software Foundation under the Linux Foundation (not by AWS), and OpenSearch 3.0 shipped in April 2025 with Lucene 10 underneath and roughly 20% query performance improvements over 2.x.
For most search and log analytics workloads, OpenSearch is a drop-in Elasticsearch replacement with lower licensing risk and competitive feature depth. The areas where it falls behind — advanced vector quantization, tighter LLM inference integration, Kibana’s UX maturity — are real, and the last section covers when they actually matter. The rest of this post is about running OpenSearch well: sizing shards correctly, managing index lifecycle, building alerting that does not cause alert fatigue, using k-NN search, and operating a cluster on Kubernetes without getting surprised by behavior that should have been obvious in advance.
OpenSearch vs Elasticsearch in 2026
The licensing situation is no longer as clean as “OpenSearch is free, Elasticsearch is not.” Elastic added AGPL v3 as a third licensing option for Elasticsearch 8.15+ alongside SSPL and the Elastic License. AGPL is a copyleft license: if you modify Elasticsearch and distribute it or make it accessible over a network, you may be required to release your modifications under AGPL. For organizations embedding search in a product they distribute or operate as SaaS, the legal analysis is non-trivial. OpenSearch’s Apache 2.0 license has no such requirement. This is not a hypothetical concern — it is the reason organizations with in-house legal review consistently prefer OpenSearch for product integrations.
Practically, the feature comparison in 2026:
| Area | OpenSearch 3.x | Elasticsearch 8.x/9.x |
|---|---|---|
| Core search (BM25, aggregations) | Full parity | Full parity |
| k-NN / vector search | Production-ready, FAISS/NMSLIB/Lucene engines | More advanced quantization (BBQ, DiskBBQ in 9.x) |
| Neural/semantic search | Neural search plugin, ML Commons | ELSER, native LLM inference pipelines |
| License | Apache 2.0 (Linux Foundation) | AGPL/SSPL/Elastic License |
| Managed service | Amazon OpenSearch Service | Elastic Cloud (AWS/Azure/GCP) |
| Dashboard | OpenSearch Dashboards (Kibana 7.10 fork) | Kibana 8.x/9.x (more mature UX) |
| Security features (LDAP, SAML, FLS) | Free, included | Paid tier on Elastic Cloud |
| Index lifecycle | ISM (more flexible state machine) | ILM (simpler phase model) |
| Kubernetes operator | opensearch-k8s-operator (operator 3.0, Jan 2026) | ECK (Elastic Cloud on Kubernetes, more mature) |
Choose OpenSearch if: you are on AWS, you need Apache 2.0 for a product integration, you are primarily doing log analytics, or you are cost-sensitive (security features like LDAP, SAML, field-level security are free). Choose Elasticsearch if: AI-native semantic search is your primary workload and you need the latest vector quantization and LLM inference features, or you need true multi-cloud parity through a single managed service.
Cluster Architecture
An OpenSearch cluster is a set of nodes, each running one or more roles. The roles determine what the node does:
cluster_manager (formerly called master): Manages cluster-wide metadata — index creation, shard allocation, node membership. Does not store data, does not handle queries. Should receive no external traffic. For production clusters with more than six data nodes, dedicate three nodes exclusively to this role.
data: Stores index shards and handles search and indexing requests. Forms the bulk of any cluster.
ingest: Runs ingest pipelines (field transformation, geoip enrichment, script processors) before documents are indexed. Separating ingest from data nodes prevents CPU-intensive processing from competing with search.
coordinating-only (no roles configured): Acts as a smart load balancer — receives queries, fans them out to all relevant shards, merges and aggregates results. Useful only for very large clusters (20+ nodes) where aggregation overhead on data nodes becomes measurable.
For small clusters (under 10 nodes), let each node handle all roles. The separation overhead is not worth it at small scale. The configuration that fails consistently is two cluster_manager-eligible nodes — two nodes require both to be present for a quorum, so losing one node stops the cluster. Always use an odd number of cluster_manager-eligible nodes: 3 for most production clusters (quorum of 2), 5 for very large clusters (quorum of 3).
The segment lifecycle
Each shard is a Lucene index. Lucene stores documents in segments — immutable files on disk. When documents are indexed, they accumulate in an in-memory buffer, are flushed to a new segment on refresh, and then segments are periodically merged in the background. Searching requires reading from every segment in the shard; more segments means more work per query. The force merge operation compacts all segments into one, producing the fastest possible search for read-only data. This is a one-time cost worth paying for historical indices that will never receive new writes.
Index Lifecycle Management
ISM (Index State Management) is OpenSearch’s index lifecycle automation. You define a policy as a state machine: each state has actions and transitions, and indices move through states based on age, size, document count, or cron schedules.
A production ISM policy for time-series log indices:
|
|
The rollover action creates a new index when the current one reaches any of the configured thresholds. The write alias automatically moves to the new index; readers continue querying the old index until ISM moves it to warm. The force_merge in the warm state merges all segments to one, which is safe because warm indices receive no new writes.
Reducing replicas to 0 in the warm state cuts storage cost in half. For log analytics, a single lost copy of 30-day-old data is acceptable; for compliance data it may not be. Set replica count based on your actual durability requirements, not as a reflex.
Rollover alias pattern
The rollover action requires a write alias. Applications write to the alias; ISM manages which backing index receives writes:
|
|
All producers write to logs-write. On rollover, logs-000001 has is_write_index set to false and logs-000002 is created with is_write_index: true. Search queries against logs-* or logs-write span all backing indices automatically.
Composable index templates
Separate settings, mappings, and aliases into component templates for reuse:
|
|
dynamic: false prevents mapping explosion — a common OpenSearch cluster killer where a field with high cardinality (user-supplied JSON, arbitrary key-value logs) creates thousands of mapping entries, exhausting JVM heap. Set dynamic: false on indices where you do not control the document structure, and explicitly map only the fields you need.
doc_values: false on trace_id is correct if you never sort or aggregate on trace IDs. Doc values live on disk but are loaded into memory for sorting and aggregations. Disabling them on fields you only use for filtering saves memory and speeds up document writes.
Shard Sizing
The guideline is 10–50 GB per shard. The reason it is a range, not a number, is that the right answer depends on your query pattern and write throughput, and the only way to know is to measure.
What the guideline prevents:
Too many small shards: Each shard is a Lucene index with its own file handles, in-memory segment metadata, field data structures, and JVM overhead. A cluster with 10,000 shards averaging 500 MB each is spending most of its JVM heap on shard overhead rather than on actually useful caches. Bulk indexing throughput drops because every document write is coordinated across too many shards. The symptom is slow bulk indexing that does not improve when you add nodes.
Too few large shards: Search parallelism comes from spreading work across shards. A single 500 GB shard cannot be searched faster than one thread can scan 500 GB of Lucene data. The symptom is slow query latency that does not improve when you add data nodes, because the shard count is the hard limit on parallelism.
A reasonable sizing starting point:
primary_shard_count = ceil(compressed_data_size_GB / 30)
For a log stream that produces 1 TB/day and retains 7 days of hot data: 7 TB compressed (typically 4:1 compression on JSON logs → 1.75 TB on disk, but index overhead puts effective size at ~2.5 TB) → ceil(2500 / 30) = 84 primary shards. Round to a number that divides evenly across your data node count.
The critical constraint: primary shard count is fixed at index creation and cannot be changed without a full reindex. Set it in the index template before data flows in. If you use rollover, each new index can have a different shard count — you can correct a wrong decision on the next rollover boundary without touching historical data.
Force merge for read-only indices
Once an index transitions to warm (no new writes), force merging to one segment produces a meaningful query speedup:
|
|
This blocks shard activity during the merge, which can take minutes for large shards. Run it via ISM or during off-peak hours, not against active write indices. A force-merged index queries 10–50% faster than a multi-segment equivalent, and the improvement is most significant for range queries over time series data.
Target shards per node
Cluster health degrades when nodes have too many shards. A general limit is 20–50 shards per GB of JVM heap per node. On a node with 16 GB of JVM heap (32 GB container memory), that means a soft ceiling around 320–800 shards. Exceeding this causes GC pressure, slower shard allocation, and cluster instability during node failures.
Alerting
OpenSearch’s alerting plugin supports five monitor types. Three of them handle almost all production use cases:
Per-query monitors: Run a search query on a schedule, evaluate the results against a condition. Best for threshold alerts on aggregated metrics.
Per-bucket monitors: Aggregate data over a time window, alert when a bucket value crosses a threshold for a configurable number of consecutive periods. Reduces noise compared to per-query monitors by requiring the condition to be sustained.
Per-cluster-metrics monitors: Alert on internal cluster state — node JVM usage, shard counts, cluster health status. These do not require document indexing; they query the cluster’s internal stats API.
A per-query monitor for p99 latency:
|
|
A cluster health monitor that fires when any node exceeds 85% JVM usage:
|
|
Alert fatigue is the main operational risk. Avoid monitoring every metric with a tight threshold — the oncall becomes desensitized and real alerts get buried. Productive monitors:
- Cluster health RED or YELLOW for more than 5 minutes
- Any node JVM heap above 85%
- Unassigned shard count above 0 for more than 10 minutes
- p99 latency above SLO threshold
- Indexing lag (documents per second drops below expected minimum)
Everything else — disk usage warnings, individual query latency spikes, minor transient rebalancing — is better surfaced in a dashboard that engineers check on a regular cadence, not via pager.
k-NN Neural Search
OpenSearch’s k-NN plugin supports approximate nearest-neighbor search across dense vector fields. Three engines are available: NMSLIB (HNSW algorithm, CPU-friendly), FAISS (Facebook AI Similarity Search, supports IVF and product quantization for compression), and Lucene (pure Java, no native library dependency). For most deployments, FAISS with HNSW is the right starting point.
Index setup
|
|
ef_construction controls build quality: higher values build a better HNSW graph at the cost of slower indexing. 256 is a reasonable default for production. m controls graph connectivity: each node connects to m neighbors. Higher m improves recall but increases memory usage and build time. 16 is the standard starting point.
Space type cosinesimil measures the angle between vectors, appropriate for embedding models that produce unit-normalized outputs (most sentence transformers). Use l2 for models that are not normalized.
Querying
|
|
num_candidates controls the quality-latency tradeoff at search time: the algorithm evaluates this many candidates and returns the top k. Higher values find more accurate results at the cost of more computation. Start at 10× your k value and measure recall against an exact search baseline.
The filter field is important: filtering after the k-NN search (post-filter) is faster but can miss relevant results when the filter is selective. For high-selectivity filters, use pre-filtering by configuring it in the knn query parameters rather than as a separate filter clause.
Hybrid search
In practice, keyword and semantic search are complementary. BM25 is good at exact terminology matching; k-NN is good at conceptual similarity. Hybrid search combines both:
|
|
The normalization processor scales both scores to [0, 1] before combining. The weights (0.4 BM25, 0.6 k-NN) are tunable — test with your workload. Hybrid search typically outperforms either method alone on diverse user queries.
Memory and scaling
Vector storage is memory-bound. Each 768-dimensional float32 embedding uses 3 KB of storage, but the HNSW graph adds roughly 30% overhead. 1 million product embeddings require approximately 4 GB of RAM on the nodes serving k-NN queries. For larger corpora, FAISS with IVF + product quantization reduces memory by 10–30× at the cost of 5–15% recall — worth evaluating once you exceed tens of millions of vectors.
Running on Kubernetes
The OpenSearch Kubernetes operator (opensearch-project/opensearch-k8s-operator) manages OpenSearch clusters as Kubernetes custom resources. Version 3.0 (January 2026) added quorum-safe rolling restarts, multi-namespace support, and TLS hot-reload. It is production-ready.
Cluster definition
|
|
JVM heap sizing
The heap must be at least equal to requests.memory divided by 2, and must not exceed 32 GB. The 32 GB limit is not arbitrary — it is the threshold below which the JVM uses compressed ordinary object pointers (compressed OOPs), which reduces heap overhead by 30–40%. A 31 GB heap outperforms a 33 GB heap in most benchmarks because of this.
Always set -Xms equal to -Xmx. A heap that starts small and grows triggers repeated GC pauses as it expands. Setting both to the same value pre-allocates the entire heap at startup.
Set requests.memory equal to limits.memory. A container that starts with 4 GB available but can burst to 8 GB will have its JVM heap sized for 4 GB (at startup) but the OS will allow 8 GB of off-heap use. The result is OOMKilled when total memory (heap + off-heap) exceeds the limit. Equal requests and limits produces Guaranteed QoS and predictable behavior.
Storage for hot tier
Use local NVMe or high-IOPS network volumes for data nodes. OpenSearch’s write path is latency-sensitive — segment flushes and translog writes are synchronous, so storage latency directly affects indexing throughput. On AWS, gp3 with 3000+ IOPS and 125+ MB/s throughput is the minimum for hot data nodes. io2 with 10,000+ IOPS is appropriate for high-throughput log ingestion.
For cluster_manager nodes, a small gp3 volume suffices — they store only cluster metadata.
PodDisruptionBudget
The operator creates a PDB automatically when podDisruptionBudget.enable: true. Setting minAvailable: 2 for a 3-node data pool means Kubernetes cannot evict more than one pod at a time, which keeps the cluster above quorum during node drains. Without a PDB, a cluster upgrade or node maintenance can evict all three pods simultaneously, causing cluster unavailability.
Snapshots
The operator does not manage snapshots. Use a separate CronJob or an external tool to register a snapshot repository and trigger scheduled snapshots:
|
|
Run this daily via a CronJob. Verify restore regularly — a snapshot that cannot be restored is not a backup.
Amazon OpenSearch Service
For teams that want OpenSearch without the operational overhead of running it themselves, Amazon OpenSearch Service is the managed option. Key decisions:
UltraWarm: S3-backed warm storage that is approximately 90% cheaper per GB than hot storage. Query latency is 10–100x higher than hot, which is acceptable for logs older than 30 days that are queried infrequently. Enable it if you have more than a few terabytes of warm data — the cost difference is significant.
Multi-AZ with Standby: Three-AZ deployment with synchronous replication to a standby node set. Automatic failover completes in roughly one minute versus 10+ minutes for two-AZ configurations. Required for production workloads with availability SLOs.
Instance type recommendation: OR1 (OpenSearch Optimized) instances use local NVMe with synchronous S3 backup for durability. They deliver about 30% better price-performance than equivalent r7g/r8g instances for dense data workloads. Use Graviton4-based instances (R8g, C8g, M8g) added in October 2025 for 30% better compute performance over Graviton3 at lower cost.
Blue-green deployments: All configuration changes trigger a blue-green deployment — a parallel cluster is provisioned, data is synchronized, and traffic is cut over. This eliminates downtime for most changes but adds 15–60 minutes to every configuration update. For clusters over 30 nodes, the “capacity optimized” blue-green mode (March 2026) provisions capacity incrementally rather than all at once, reducing the risk of capacity-constrained deployments.
Performance Tuning Reference
During bulk load:
|
|
Disabling refresh prevents segments from being written to disk during the load, allowing the indexing buffer to accumulate more data before flush. Setting replicas to 0 halves the total writes. After the load:
|
|
Mapping best practices:
- Use
keywordnottextfor fields you aggregate or sort on - Set
dynamic: falsefor indices with uncontrolled field cardinality - Disable
doc_values: falseon keyword fields that are only used for filtering, never sorting or aggregations - Disable
_source: {enabled: false}only if you never need to retrieve original documents (saves 10–30% storage)
JVM tuning:
-Xms<N>g -Xmx<N>g
-XX:+UseG1GC
-XX:G1HeapRegionSize=32m
-XX:+ParallelRefProcEnabled
G1GC is OpenSearch’s default and works well. Set G1HeapRegionSize to 32m for heaps above 16 GB to reduce the number of regions and lower GC overhead. ParallelRefProcEnabled speeds up reference processing during GC cycles.
Heap monitoring thresholds:
| Heap usage | Action |
|---|---|
| < 70% | Normal |
| 70–85% | Monitor closely; reduce field data cache usage if text aggregations are active |
| > 85% | Immediate action — add nodes or reduce shard count |
| > 90% | Cluster is at risk of OOM; shed load |
Circuit breakers in OpenSearch will start rejecting requests above configurable heap thresholds to prevent OOM. The defaults are conservative. Do not raise circuit breaker limits as a workaround for a cluster that is genuinely under-resourced.
Honest Operational Assessment
Running OpenSearch in production requires understanding a set of failure modes that are not obvious from the documentation: shard allocation failures during node restarts, GC pauses that coincide with bulk indexing bursts, mapping explosions from dynamic fields, and split-brain conditions during network partitions in multi-AZ deployments. None of these are unique to OpenSearch — they are Lucene and distributed systems problems that Elasticsearch operators have been dealing with for a decade.
The tools for managing them are good: the operator handles Kubernetes lifecycle, ISM handles index lifecycle, the alerting plugin handles notification, and the cluster health API exposes the information you need. The learning curve is real but not steep. A team that understands the shard sizing principles, keeps JVM heap below 85%, uses ISM to manage rollover and cleanup, and runs regular restore tests on their snapshots will operate OpenSearch without significant incidents.
The operational complexity is roughly equivalent to self-hosted Elasticsearch. The main practical advantage of choosing OpenSearch is not operational simplicity — it is license clarity, and for teams on AWS, tight integration with the managed service and lower cost on equivalent workloads.
Comments