CLOUD SERVICE · AWS

Amazon OpenSearch Service

Amazon OpenSearch Service is a scalable and fully managed search and analytics service powered by the open source Elasticsearch and OpenSearch engines, designed for real-time application monitoring, log analytics, and search functionality.

Cloud Services Hub →

Icon source: AWS

What is Amazon OpenSearch Service

Read the extensive description

Amazon OpenSearch Service, formerly known as Amazon Elasticsearch Service, is a fully managed service that makes it easy for developers to deploy, secure, operate, and scale OpenSearch, an open-source, distributed search and analytics suite.

This service offers a seamless way to perform interactive log analytics, real-time application monitoring, website search, and more, on massive volumes of data with high velocity and variety, typically referred to as "big data".

The roots of Amazon OpenSearch Service trace back to Elasticsearch, but following the licensing changes in Elasticsearch 7.11, Amazon Web Services opted to fork the project, leading to the birth of OpenSearch. This move ensured that users could continue leveraging the robust capabilities of the platform under an open-source license, providing a pathway for both existing Elasticsearch users and new adopters to benefit from the innovations and improvements in search and analytics without licensing restrictions.

OpenSearch Service simplifies the complex tasks of managing search engine operations. It automates tedious tasks such as hardware provisioning, setup, software patching, failure recovery, backups, and monitoring. This allows developers and businesses to focus on creating innovative applications and derive meaningful insights from their data, without worrying about the underlying infrastructure.

Managed clusters can use Multi-AZ with Standby, a deployment option that maintains availability and consistent performance during infrastructure failures, including a single Availability Zone failure.

Amazon OpenSearch Service offers a variety of features that cater to diverse needs. It supports OpenSearch Dashboards, which is a suite of visualization tools to monitor, analyze, and explore data stored in your OpenSearch domains visually. This is particularly useful for businesses that need to analyze large datasets to make informed decisions quickly.

Moreover, the service provides a robust security offering. Users can control access to their OpenSearch domains using AWS Identity and Access Management (IAM) for fine-grained access control. Additionally, it supports data encryption both in transit and at rest, ensuring that sensitive information is protected from unauthorized access.

Another critical aspect of Amazon OpenSearch Service is its scalability. Users can scale their OpenSearch clusters up or down, based on their requirements, without incurring downtime. This feature is crucial for handling variable workloads, helping businesses maintain optimal performance while managing costs effectively.

Amazon OpenSearch Serverless offers NextGen and Classic collections. Classic also offers a dev-test deployment without redundant standby nodes.

Furthermore, Amazon OpenSearch Service is deeply integrated with the AWS ecosystem, offering out-of-the-box integrations with a wide range of AWS services such as Amazon Kinesis, AWS Lambda, and Amazon CloudWatch. This facilitates efficient data ingestion, processing, and analytics workflows, empowering users to build comprehensive, real-time analytics solutions on AWS. Overall, Amazon OpenSearch Service stands as a powerful, scalable, and fully managed service that offers a

Key Amazon OpenSearch Service Features

Amazon OpenSearch Service offers powerful, scalable, and easy-to-manage search and analytics capabilities with real-time application monitoring and log analytics features.

Managed Service

Amazon OpenSearch Service is a fully managed service that makes it easy for you to deploy, secure, and run OpenSearch cost effectively at scale. It takes care of the heavy lifting of managing your clusters.

Easy to Scale

OpenSearch Serverless automatically scales capacity to match demand without downtime.

Secure

Supports role-based access control (RBAC) for fine-grained permissions and index-level encryption for tenant data isolation.

Highly Available

Designed for high availability, Amazon OpenSearch Service automatically replaces failed nodes in your cluster, reducing the impact on your applications and minimizing downtime.

Monitoring and Logging

Integrated with Amazon CloudWatch and AWS CloudTrail, providing deep insights into queries and metrics for your OpenSearch clusters, and logging for audit, compliance, and operational history.

Compatible with OpenSearch and Elasticsearch

Supports both OpenSearch and legacy Elasticsearch APIs, giving users flexibility in their choice of search and analytics engines without having to manage the underlying infrastructure.

Machine Learning Features

Includes built-in machine learning tools to help enhance search and analytics capabilities, such as anomaly detection and adaptive relevance for more accurate and personalized search results.

Data Durability

Utilizes automated snapshots for data backups, ensuring your data is durable and can be recovered in the case of a failure.

Rich Set of Plugins

Offers a wide range of plugins for different use cases, including security, alerting, monitoring, and more, allowing for a customized search and analytics solution.

Global Reach

Available in multiple AWS regions around the world, making it easy to deploy applications close to your users for reduced latency and improved performance.

Amazon OpenSearch Service Use Cases

Amazon OpenSearch Service is utilized for real-time application monitoring, log and event analysis, website search enhancements, and data visualization tasks across diverse industries.

Real-Time Analytics

Amazon OpenSearch Service enables businesses to perform real-time analysis of their data, such as website clickstreams, financial transactions, and social media signals. Companies can use this to monitor performance metrics, detect anomalies, and make data-driven decisions instantaneously.

Log Analytics

It's common to use Amazon OpenSearch Service for centralizing and analyzing logs from various sources like web servers, applications, and IoT devices. This allows IT and DevOps teams to monitor system health, troubleshoot issues quickly, and understand operational trends over time.

Search-Enabled Applications

Developers can leverage Amazon OpenSearch Service to add rich search features to their applications, websites, and services. This includes full-text search with relevance ranking, faceted search for filtering results, and auto-completion features to enhance user experiences.

Security Information and Event Management (SIEM)

Organizations use Amazon OpenSearch Service as part of their SIEM solution to aggregate and analyze security event data in real time. This assists in threat detection, compliance monitoring, and maintaining secure operations across their digital environments.

Machine Learning for Predictive Analytics

With built-in machine learning capabilities, Amazon OpenSearch Service can be used for predictive analytics, helping businesses forecast trends, understand customer behavior, and make proactive decisions based on predictive models generated from their data sets.

Amazon OpenSearch Service pricing models

Amazon OpenSearch Service pricing is based on the compute instance, storage used, data transfer, and optional features such as UltraWarm and Cold Storage.

Amazon OpenSearch Storage

Charged based on the amount of data you store in your OpenSearch domains. Pricing varies by storage type (e.g., SSD vs. HDD) and region.

Automatic Semantic Enrichment

Semantic encoding generation during ingestion is charged in OCU-hours with per-minute granularity. No Semantic Search OCUs are charged when unused, and there are no additional Semantic Search OCU charges for searches or storage.

Data Transfer

Incoming data transfer is free. Outgoing data transfer is charged based on the total amount of data transferred out of the OpenSearch Service across all APIs, and pricing varies by region.

Database Savings Plans

Offers savings for both Managed Clusters and Serverless in exchange for a usage commitment measured in dollars per hour over a one-year term.

Direct Query

Compute used to query connected data sources or maintain indexes is charged in OCU-hours with per-minute granularity. No OCUs are consumed when no queries or indexing jobs are active.

Extended Support

Engine versions in Extended Support incur a surcharge in addition to standard instance costs. Depending on the version and support period, this is a flat fee per Normalized Instance Hour or, for an additional extension beyond the originally published support end date, an amount equal to the instance cost; storage costs are unaffected by that extension surcharge.

On-Demand Instances

Charges are based on the type and number of OpenSearch instances (nodes) used, the hours those instances are running, and the region in which the instances are located.There is no long-term commitment required.

OpenSearch Ingestion

Compute used to ingest, transform, and route data in ingestion pipelines is charged in OCU-hours with per-minute granularity. Pipeline minimum and maximum OCUs can be configured, and paused pipelines consume no OCUs.

Reserved Instances

Offers a discount on the hourly charge for an instance, in exchange for a commitment to use a specific instance type in a single region for a one or three year term.

S3 Vectors

Vector uploads are charged by logical gigabytes uploaded, and storage is based on the logical size of vector data, metadata, and keys. Queries incur both a per-API charge and a data-processing charge based on index vector count and average vector size, including keys and filterable metadata.

Serverless Classic Collections

Indexing and search compute is billed in OCU-hours with per-second granularity, and managed storage is billed in gigabyte-months. The first collection has a minimum of 2 OCUs with redundancy, or 1 OCU for dev-test without redundant standby nodes; minimum capacity varies with data size and collection type.

Serverless NextGen Collections

Compute for indexing and search is charged in OCU-hours at the collection-group level with per-second granularity, and hot storage is charged separately in gigabyte-months. There is no minimum OCU requirement; indexing and search OCUs scale to zero after 10 minutes of inactivity.

Snapshot Storage

Automated cluster snapshot storage is free, and snapshots are retained for 14 days. Manual snapshots are stored in Amazon S3 and incur standard Amazon S3 usage charges.

Vector Auto-Optimize Jobs

Charged at a fixed rate per job for hyperparameter optimization that evaluates index configurations and produces optimization recommendations.

Vector Index GPU-Acceleration

Accelerated vector indexing is charged for consumed Vector Acceleration OCU time, billed hourly with per-second granularity. Managed auto-scaling and warm pooling do not add billed time.

Services Amazon OpenSearch Service integrates with

AWS CloudTrail

AWS CloudTrail logs can be indexed in Amazon OpenSearch Service for monitoring API activity and detecting unusual behavior.

Open AWS CloudTrail →

Amazon CloudWatch

Amazon CloudWatch can be used to monitor and log OpenSearch Service metrics and set alarms.

Open Amazon CloudWatch →

AWS Lambda

AWS Lambda can be used to preprocess data before ingesting it into Amazon OpenSearch Service, and can also trigger actions based on OpenSearch data.

Open AWS Lambda →

Amazon S3

Amazon OpenSearch Service can index data stored in Amazon S3 buckets and be used to store logs and backups of your OpenSearch Service data.

Open Amazon S3 →

Amazon Data Firehose

Amazon Kinesis Data Firehose can be used to load streaming data directly into Amazon OpenSearch Service for near real-time analytics.

Open Amazon Data Firehose →

Amazon Kinesis Data Streams

Amazon Kinesis Data Streams can be integrated with OpenSearch Service to capture, process, and analyze real-time streaming data.

Open Amazon Kinesis Data Streams →