Bug Report: Microsoft Sentinel Logstash Output Plugin "dcr_id must be provided and cannot be empty"

[AndrewPascoCS]

Bug Report: Microsoft Sentinel Logstash Output Plugin

Describe the bug

The Microsoft Sentinel Log Analytics Logstash output plugin fails to initialize with an unclear error message when the dcr_id parameter is not provided in the output configuration. The error occurs during pipeline initialization and causes Logstash to shut down, but the error message does not clearly indicate that the dcr_id parameter is mandatory or how to resolve the issue.

Error Message:

java.lang.IllegalArgumentException: dcr_id must be provided and cannot be empty

This error is thrown from the plugin factory during CompiledPipeline initialization and prevents the entire Logstash instance from starting.

To Reproduce

Steps to reproduce the behavior:

1. Create a Docker image using Logstash 9.2.5 (or compatible version)
2. Install the Microsoft Sentinel Log Analytics Logstash output plugin via:

   RUN bin/logstash-plugin install microsoft-sentinel-log-analytics-logstash-output-plugin
   

3. Create a Logstash pipeline configuration file that uses the Microsoft Sentinel output plugin without specifying the dcr_id parameter:

    output {
    # For debugging purposes, uncomment the following line to see all processed messages
    #stdout { codec => rubydebug }
    microsoft-sentinel-log-analytics-logstash-output-plugin {
        client_app_Id => "27xxxx33-fxxc-xxd2-80xx-6xx3f407d2ee"
        client_app_secret => "${Beats_DCR_Secret}"
        tenant_id => "1xxxxxx8-exxb-44xx-xxxd-b57xxxxxxxx8"
        data_collection_endpoint => "https://gc-syslog-prod-monitoring-dce-9xxx.australiaeast-1.ingest.monitor.azure.com"
        dcr_immutable_id => "dcr-xxxxxxxxxx304b87b7fa1a4570a1701c"
        dcr_stream_name => "Custom-SyslogStream"
        }
    } #closing DCR output plugin for Sentinel
   

4. Start the Logstash container with the configuration
5. Observe the error in the logs

Expected behavior

The plugin should either:

1. Provide a clear error message that explicitly states: "The dcr_id parameter is required and must be configured in the output block" at startup with guidance on how to configure it
2. Fail gracefully with descriptive logging that identifies the missing mandatory parameter before attempting to initialize the pipeline
3. Validate configuration before the Java compilation phase and provide user-friendly error messages in the Logstash logs

Currently, the error is buried deep in the Java stack trace and does not clearly indicate what parameter is missing or how to fix it.

Actual behavior

Logstash fails with the following error stack trace:

[2026-04-02T04:36:17,326][INFO ][org.logstashplugins.MicrosoftSentinelLogAnalyticsLogstashOutputPlugin] Starting Microsoft Sentinel output plugin
[2026-04-02T04:36:17,328][INFO ][org.logstashplugins.LogAnalyticsEventsHandler.LAEventsHandlerConfiguration] Azure client credentials not fully provided, falling back to default Azure credential chain
{[2026-04-02T04:36:17,329][ERROR][logstash.agent ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:5514-syslog-raw, :exception=>"Java::JavaLang::IllegalStateException", :message=>"Unable to configure plugins: java.lang.IllegalArgumentException: dcr_id must be provided and cannot be empty", :backtrace=>[...truncated stack trace...]}
[2026-04-02T04:36:17,340][INFO ][logstash.runner ] Logstash shut down.
[2026-04-02T04:36:17,347][FATAL][org.logstash.Logstash ] Logstash stopped processing because of an error: (SystemExit) exit

The issue is that:

1. The error message is cryptic and embedded in a Java exception
2. There is no guidance on what the dcr_id parameter is or where to find its value
3. The error occurs during plugin instantiation rather than during configuration validation
4. For users unfamiliar with Java exceptions, the error is difficult to troubleshoot

Screenshots

Not applicable (console error output provided above)

Desktop Information

Item	Value
OS	Linux (in Docker container)
Container Runtime	Podman
Logstash Version	9.2.5
JDK Version	OpenJDK 64-Bit Server VM 21.0.9+10-LTS
JRuby Version	jruby 9.4.13.0 (3.1.4) 2025-06-10 9938a3461f
Plugin	microsoft-sentinel-log-analytics-logstash-output-plugin (latest version)

Additional context

Environment Details

Dockerfile Configuration:

FROM public.ecr.aws/docker/library/logstash:9.2.5
RUN bin/logstash-plugin install microsoft-sentinel-log-analytics-logstash-output-plugin
COPY Logstash/msplugin/pipeline/* /usr/share/logstash/pipeline/
COPY Logstash/msplugin/config/* /usr/share/logstash/config/
RUN rm /usr/share/logstash/config/logstash-sample.conf
CMD ["logstash"]

Container Launch Command:

podman run -d \
  -p 5514:5514/tcp \
  -p 5514:5514/udp \
  -e ADXSecret=$env:ADXSecret \
  -e Beats_DCR_Secret=$env:Beats_Security_Logs_DCR_secret \
  -e LS_JAVA_OPTS=$env:LS_JAVA_OPTS \
  -e ADXTenanntId=$env:ADXTenanntId \
  -e ADXAppId=$env:ADXAppId \
  --name Sentinel-DCR-Test \
  localhost/dcrbug

Root Cause Analysis

The plugin requires the dcr_id parameter to be explicitly configured in the output block. The parameter validation exception is thrown during the Java plugin instantiation phase rather than during Logstash's YAML parsing and validation, making it difficult for users to identify and resolve the issue.

Suggested Resolution

1. Add explicit configuration validation in the plugin initializer that checks for required parameters before calling parent constructors
2. Provide a clear, user-friendly error message that identifies the missing dcr_id parameter
3. Update documentation to clearly indicate that dcr_id is a mandatory parameter
4. Consider making the error message accessible at the INFO or WARN log level in addition to ERROR

[lsoumille]

Microsoft released a new version of this plugin. The code and documentation in this repository have not been updated.

You should set the plugin version to 1.1.4 to make it work with your configuration https://rubygems.org/gems/microsoft-sentinel-log-analytics-logstash-output-plugin/versions/1.1.4