opensearchexporter

OpenSearch Exporter

Status
Stability	alpha: traces, logs
Distributions	contrib
Issues
Code coverage
Code Owners	@ps48 | Seeking more code owners!
Emeritus	@Aneurysm9, @MitchellGale, @MaxKsyunz, @YANG-DB

OpenSearch exporter supports sending OpenTelemetry signals as documents to OpenSearch.

The documents are sent using observability catalog schema.

Configuration options

Indexing Options

The Observability indices would follow the recommended pattern for immutable data stream ingestion using the data_stream concepts. Index pattern will follow the next naming template ss4o_{type}-{dataset}-{namespace}

• dataset (default=default) a user-provided label to classify source of telemetry. It is used to construct the name of the destination index or data stream.
• namespace (default=namespace) a user-provided label to group telemetry. It is used to construct the name of the destination index or data stream.

Dynamic Indexing

The OpenSearch exporter supports dynamic index names for both logs and traces using placeholders. You can use any attribute or context key as a placeholder to construct index names dynamically per record.

Caution: In practice, OpenSearch clusters can become unstable or even break down when index counts reach a very high level. Introducing attributes with high cardinality resulting in many separate indices can significantly impact the stability of your target cluster.

Configuration Options:

• logs_index - Custom index name pattern for logs
• logs_index_fallback - Fallback value when placeholder is missing (default: unknown)
• logs_index_time_format - Time suffix format for logs
• traces_index - Custom index name pattern for traces
• traces_index_fallback - Fallback value when placeholder is missing (default: unknown)
• traces_index_time_format - Time suffix format for traces

Placeholder Syntax:

• Placeholder: %{key}
  • Example: otel-logs-%{service.name} or otel-traces-%{service.name}-%{env}
  • Multiple placeholders are supported per index name
  • The value is looked up from item attributes (log/span), scope attributes, and resource attributes (in that precedence order)
  • If the key is missing, the fallback value is used
  • Generated index names must adhere to OpenSearch index naming restrictions
  • A substituted attribute value that is empty, starts with ., or contains .. is skipped as if the key were missing, so the next attribute in the precedence order (then the fallback) is used. This keeps a value from resolving to a system index (e.g. .kibana) or a different index via a .. segment. Other values are substituted as-is; a value OpenSearch rejects (uppercase, spaces, forbidden characters) surfaces as an indexing error rather than being silently rerouted. The configured fallback value is trusted and is not validated.

Time Suffix Format:

Both logs and traces support time-formatted suffixes using *_time_format options. The time suffix only supports UTC.

• Valid tokens (case-sensitive):
  • yyyy (4-digit year), yy (2-digit year)
  • MM (2-digit month), dd (2-digit day)
  • HH (2-digit hour, 24h), mm (2-digit minute), ss (2-digit second)
• Allowed separators: -, ., _, +
• Examples: yyyy.MM.dd → 2024.06.07, yyyy-MM → 2024-06, yyMMdd → 240607

Default Behavior:

If custom index names are not set, the exporter uses default patterns:

• Logs: ss4o_logs-{dataset}-{namespace}
• Traces: ss4o_traces-{dataset}-{namespace}

Example Configuration

exporters:
  opensearch:
    http:
      endpoint: http://opensearch.example.com:9200
    # Logs configuration
    logs_index: "otel-logs-%{service.name}-%{env}"
    logs_index_fallback: "default"
    logs_index_time_format: "yyyy.MM.dd"
    # Traces configuration
    traces_index: "otel-traces-%{service.name}-%{deployment.environment}"
    traces_index_fallback: "unknown"
    traces_index_time_format: "yyyy.MM.dd"
    sending_queue:
      batch:

This configuration will create:

• Log indexes like: otel-logs-myservice-prod-2024.06.07
• Trace indexes like: otel-traces-myservice-production-2024.06.07

If any placeholder key is missing, the fallback value is used e.g.:

• otel-logs-myservice-default-2024.06.07).
• otel-traces-unknown-production-2024.06.07

OpenSearch document mapping

The mapping mode can be controlled via the scope attribute opensearch.mapping.mode.

The OpenSearch exporter supports several document schemas and preprocessing behaviors, which may be configured through the following settings:

• mapping:
  • mode (default=ss4o): Configures the field mappings. Supported modes are:
    • ss4o: Exports logs in the Simple Schema for Observability standard.
    • ecs: Maps fields defined in the OpenTelemetry Semantic Conventions to the Elastic Common Schema
    • flatten_attributes: Uses the ECS mapping but flattens all resource and log attributes in the record to the top-level.
    • bodymap: uses the "body" of a log record as the exact content of the OpenSearch document, without any transformation. This mapping mode is intended for use cases where the client wishes to have complete control over the OpenSearch document structure.
    • otel-v1: exports logs and traces using the Data Prepper OTel v1 schema, compatible with OpenSearch Observability dashboards.
  • manage_index_template: (optional, default=false) When true, creates composable index templates on startup. Only valid with otel-v1 mode.
  • timestamp_field: (optional) Field to store the timestamp in. If not set, uses the default @timestamp.
  • unix_timestamp: (optional) Whether to store the timestamp in epoch milliseconds.
  • dedup: (optional) removes fields from the document, that have duplicate keys. The filtering only keeps the last value for a key.
  • dedot: (optional) convert dotted keys into nested JSON objects.

SS4O mapping mode

The default Simple Schema for Observability mapping mode.

In ss4o mapping mode, the OpenSearch exporter stores documents using the SS4O schema, which is designed for observability data in OpenSearch. Documents use standardized field names and structure to facilitate integration with OpenSearch dashboards and tools.

Signal	Supported
Logs	✅
Traces	✅

ECS mapping mode

Warning

The ECS mapping mode is currently undergoing changes, and its behaviour is unstable.

In ecs mapping mode, the OpenSearch exporter maps fields from OpenTelemetry Semantic Conventions to Elastic Common Schema (ECS) where possible. This mode may be used for compatibility with dashboards and tools that expect ECS.

Signal	ecs
Logs	✅
Traces	🚫

Flatten attributes mapping mode

Warning

The Flatten attributes mapping mode is currently undergoing changes, and its behaviour is unstable.

In flatten_attributes mapping mode, the OpenSearch exporter uses the ECS mapping but flattens all resource and log attributes in the record to the top-level of the document.

Signal	flatten_attributes
Logs	✅
Traces	🚫

Bodymap mapping mode

In bodymap mapping mode, the OpenSearch exporter supports only logs and uses the "body" of a log record as the exact content of the OpenSearch document, without any transformation. This mapping mode is intended for use cases where the client wishes to have complete control over the OpenSearch document structure.

The bodymap mapping mode only supports log records where the body is of type Map. If the log body is not a Map, encoding will fail with an error. This ensures that only structured map data can be used as the document content in bodymap mode.

Signal	bodymap
Logs	✅
Traces	🚫

OTel v1 mapping mode

In otel-v1 mapping mode, the OpenSearch exporter produces documents compatible with Data Prepper's OTel v1 index schemas. This enables interoperability with OpenSearch Observability dashboards that consume Data Prepper indices.

Default index names:

• Traces: otel-v1-apm-span
• Logs: otel-v1-logs

These defaults can be overridden using traces_index and logs_index configuration options.

The manage_index_template option controls whether the exporter automatically creates composable index templates on startup. This ensures correct field mappings (e.g., date_nanos timestamps, dynamic attribute typing) are in place before documents are indexed.

Template creation is best-effort: if the cluster is temporarily unreachable or rejects the request, the exporter logs a warning and continues — the failure is not propagated through Start(). The affected index will then fall back to OpenSearch's dynamic mapping (timestamp fields inferred as date rather than date_nanos) until the template is installed. If templates with the same names already exist (for example, user-customized variants), they are left in place rather than overwritten.

Signal	otel-v1
Logs	✅
Traces	✅

Schema references:

• otel-v1-apm-span index template
• logs-otel-v1 index template

Note

The exporter emits nanosecond-precision timestamps in the document body, but OpenSearch's dynamic mapping will infer date (millisecond precision) unless a matching index template is installed before indexing begins. Set manage_index_template: true to have the exporter create the templates on startup, or install them out-of-band — see the schema references above.

Example Configuration

exporters:
  opensearch:
    http:
      endpoint: http://opensearch.example.com:9200
    mapping:
      mode: "otel-v1"
      manage_index_template: true
    sending_queue:
      batch:

HTTP Connection Options

OpenSearch export supports standard HTTP client settings.

• http.endpoint (required) <url>:<port> of OpenSearch node to send data to.

TLS settings

Supports standard TLS settings as part of HTTP settings. See TLS Configuration/Client Settings.

Retry Options

• retry_on_failure: See retry_on_failure

Sending Queue Options

• sending_queue: See sending_queue

Timeout Options

• timeout : See timeout

Bulk Indexer Options

• bulk_action (optional): the action for ingesting data. Only create and index are allowed here.
• pipeline (optional): the ID of an ingest pipeline to apply when indexing documents. When set, all documents sent via the bulk API will be processed by the specified pipeline before being indexed. The ingest pipeline must exist in the cluster and there must be at least one node with the ingest node role assigned.

Example

extensions:
  basicauth/client:
    client_auth:
      username: username
      password: password

exporters:
  opensearch/trace:
    http:
      endpoint: https://opensearch.example.com:9200
      auth:
        authenticator: basicauth/client
    sending_queue:
      batch:
# ······
service:
  pipelines:
    traces:
      receivers: [otlp]
      exporters: [opensearch/trace]