Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

When using Java Execution Engine, using quoted plugin directives crashes Logstash #11222

Closed
yaauie opened this issue Oct 15, 2019 · 2 comments · Fixed by #11688
Closed

When using Java Execution Engine, using quoted plugin directives crashes Logstash #11222

yaauie opened this issue Oct 15, 2019 · 2 comments · Fixed by #11688

Comments

@yaauie
Copy link
Member

yaauie commented Oct 15, 2019

Pipeline Plugin directives historically could be either quoted or bareword, meaning that the following were equivalent:

filter {
  kv {
    "source" => "message"
  }
}
filter {
  kv {
    source => "message"
  }
}

But under the Java Execution Engine, quoted directives cause an Java::JavaLang::IllegalStateException during startup that crashes the process.

╭─{ yaauie@castrovel:~/src/elastic/logstash (✘ 7.x) }
╰─● bin/logstash -e 'input { stdin {} } filter { kv { "source" => "message" } } output { stdout {} }'
[snip]
[2019-10-15T19:58:25,603][ERROR][logstash.filters.kv      ] Unknown setting '"source"' for kv
[2019-10-15T19:58:25,618][ERROR][logstash.agent           ] Failed to execute action {:action=>LogStash::PipelineAction::Create/pipeline_id:main, :exception=>"Java::JavaLang::IllegalStateException", :message=>"Unable to configure plugins: (ConfigurationError) Something is wrong with your configuration.", :backtrace=>["org.logstash.config.ir.CompiledPipeline.<init>(CompiledPipeline.java:100)", "org.logstash.execution.JavaBasePipelineExt.initialize(JavaBasePipelineExt.java:60)", "org.logstash.execution.JavaBasePipelineExt$INVOKER$i$1$0$initialize.call(JavaBasePipelineExt$INVOKER$i$1$0$initialize.gen)", "org.jruby.internal.runtime.methods.JavaMethod$JavaMethodN.call(JavaMethod.java:837)", "org.jruby.ir.runtime.IRRuntimeHelpers.instanceSuper(IRRuntimeHelpers.java:1156)", "org.jruby.ir.runtime.IRRuntimeHelpers.instanceSuperSplatArgs(IRRuntimeHelpers.java:1143)", "org.jruby.ir.targets.InstanceSuperInvokeSite.invoke(InstanceSuperInvokeSite.java:39)", "Users.yaauie.src.elastic.logstash.logstash_minus_core.lib.logstash.java_pipeline.RUBY$method$initialize$0(/Users/yaauie/src/elastic/logstash/logstash-core/lib/logstash/java_pipeline.rb:26)", "org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:91)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:90)", "org.jruby.runtime.callsite.CachingCallSite.cacheAndCall(CachingCallSite.java:332)", "org.jruby.runtime.callsite.CachingCallSite.call(CachingCallSite.java:86)", "org.jruby.RubyClass.newInstance(RubyClass.java:915)", "org.jruby.RubyClass$INVOKER$i$newInstance.call(RubyClass$INVOKER$i$newInstance.gen)", "org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:183)", "Users.yaauie.src.elastic.logstash.logstash_minus_core.lib.logstash.pipeline_action.create.RUBY$method$execute$0(/Users/yaauie/src/elastic/logstash/logstash-core/lib/logstash/pipeline_action/create.rb:36)", "Users.yaauie.src.elastic.logstash.logstash_minus_core.lib.logstash.pipeline_action.create.RUBY$method$execute$0$__VARARGS__(/Users/yaauie/src/elastic/logstash/logstash-core/lib/logstash/pipeline_action/create.rb)", "org.jruby.internal.runtime.methods.CompiledIRMethod.call(CompiledIRMethod.java:91)", "org.jruby.internal.runtime.methods.MixedModeIRMethod.call(MixedModeIRMethod.java:90)", "org.jruby.ir.targets.InvokeSite.invoke(InvokeSite.java:183)", "Users.yaauie.src.elastic.logstash.logstash_minus_core.lib.logstash.agent.RUBY$block$converge_state$2(/Users/yaauie/src/elastic/logstash/logstash-core/lib/logstash/agent.rb:326)", "org.jruby.runtime.CompiledIRBlockBody.callDirect(CompiledIRBlockBody.java:136)", "org.jruby.runtime.IRBlockBody.call(IRBlockBody.java:77)", "org.jruby.runtime.Block.call(Block.java:129)", "org.jruby.RubyProc.call(RubyProc.java:295)", "org.jruby.RubyProc.call(RubyProc.java:274)", "org.jruby.RubyProc.call(RubyProc.java:270)", "org.jruby.internal.runtime.RubyRunnable.run(RubyRunnable.java:105)", "java.lang.Thread.run(Thread.java:748)"]}
warning: thread "Converge PipelineAction::Create<main>" terminated with exception (report_on_exception is true):
LogStash::Error: Don't know how to handle `Java::JavaLang::IllegalStateException` for `PipelineAction::Create<main>`
          create at org/logstash/execution/ConvergeResultExt.java:109
             add at org/logstash/execution/ConvergeResultExt.java:37
  converge_state at /Users/yaauie/src/elastic/logstash/logstash-core/lib/logstash/agent.rb:339
[2019-10-15T19:58:25,628][ERROR][logstash.agent           ] An exception happened when converging configuration {:exception=>LogStash::Error, :message=>"Don't know how to handle `Java::JavaLang::IllegalStateException` for `PipelineAction::Create<main>`", :backtrace=>["org/logstash/execution/ConvergeResultExt.java:109:in `create'", "org/logstash/execution/ConvergeResultExt.java:37:in `add'", "/Users/yaauie/src/elastic/logstash/logstash-core/lib/logstash/agent.rb:339:in `block in converge_state'"]}
[2019-10-15T19:58:25,657][FATAL][logstash.runner          ] An unexpected error occurred! {:error=>#<LogStash::Error: Don't know how to handle `Java::JavaLang::IllegalStateException` for `PipelineAction::Create<main>`>, :backtrace=>["org/logstash/execution/ConvergeResultExt.java:109:in `create'", "org/logstash/execution/ConvergeResultExt.java:37:in `add'", "/Users/yaauie/src/elastic/logstash/logstash-core/lib/logstash/agent.rb:339:in `block in converge_state'"]}
[2019-10-15T19:58:25,724][ERROR][org.logstash.Logstash    ] java.lang.IllegalStateException: Logstash stopped processing because of an error: (SystemExit) exit
bin/logstash -e   68.04s user 2.17s system 405% cpu 17.312 total
[error: 1 (17.000s)]

Disabling Java Execution and falling back to the Ruby Execution Engine, we encounter no such failure:

╭─{ yaauie@castrovel:~/src/elastic/logstash (✘ 7.x) }
╰─● bin/logstash --java-execution=false -e 'input { stdin {} } filter { kv { "source" => "message" } } output { stdout {} }'
[snip]
[2019-10-15T19:57:48,814][INFO ][logstash.pipeline        ] Starting pipeline {:pipeline_id=>"main", "pipeline.workers"=>8, "pipeline.batch.size"=>125, "pipeline.batch.delay"=>50, "pipeline.sources"=>["config string"]}
The stdin plugin is now waiting for input:
[2019-10-15T19:57:48,969][INFO ][logstash.pipeline        ] Pipeline started successfully {:pipeline_id=>"main", :thread=>"#<Thread:0xd4c7e3d sleep>"}
[2019-10-15T19:57:49,005][INFO ][logstash.agent           ] Pipelines running {:count=>1, :running_pipelines=>[:main], :non_running_pipelines=>[]}
[2019-10-15T19:57:49,327][INFO ][logstash.agent           ] Successfully started Logstash API endpoint {:port=>9600}
^C[2019-10-15T19:57:58,895][WARN ][logstash.runner          ] SIGINT received. Shutting down.
[2019-10-15T19:57:59,042][INFO ][logstash.pipeline        ][main] Pipeline has terminated {:pipeline_id=>"main", :thread=>"#<Thread:0xd4c7e3d run>"}
[2019-10-15T19:57:59,103][INFO ][logstash.runner          ] Logstash shut down.
bin/logstash --java-execution=false -e   78.73s user 2.75s system 266% cpu 30.519 total
[success (31.000s)]

@colinsurprenant
Copy link
Contributor

PR in #11688

@colinsurprenant
Copy link
Contributor

Fix will be available in upcoming v7.6.2.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

3 participants