elasticsearch_keystore resource fails due to legacy instance code

[johnkazmerzak]

Affected Puppet, Ruby, OS and module versions/distributions

• Puppet: 6.26.0
• Ruby: 2.5.9p229
• Distribution: CentOS 7
• Module version: 8.0.1

How to reproduce (e.g Puppet code you use)

elasticsearch_keystore { $hostname:
  settings => {
    'bootstrap.password' => $bootstrap_password,
    'xpack.security.transport.ssl.keystore.secure_password' => $keystore_truststore_secure_password,
    'xpack.security.transport.ssl.truststore.secure_password' => $keystore_truststore_secure_password,
  },
}

What are you seeing

Could not evaluate: Execution of '/usr/share/elasticsearch/bin/elasticsearch-keystore create' returned 1:

What behaviour did you expect instead

Successful execution of elasticsearch_keystore resource

Output log

Notice: /Stage[main]/Sstsrdnalin_profiles::Elastic/Elasticsearch_keystore[redacted]/ensure: created
Debug: Executing with uid=elasticsearch gid=elasticsearch: '/usr/share/elasticsearch/bin/elasticsearch-keystore create'
Error: /Stage[main]/Sstsrdnalin_profiles::Elastic/Elasticsearch_keystore[dnatestlog03]: Could not evaluate: Execution of '/usr/share/elasticsearch/bin/elasticsearch-keystore create' returned 1: 
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/execution.rb:297:in `execute'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/provider.rb:106:in `execute'
/opt/puppetlabs/puppet/cache/lib/puppet/provider/elasticsearch_keystore/ruby.rb:49:in `run_keystore'
/opt/puppetlabs/puppet/cache/lib/puppet/provider/elasticsearch_keystore/ruby.rb:98:in `flush'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/type.rb:1045:in `flush'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction/resource_harness.rb:25:in `evaluate'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:267:in `apply'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:287:in `eval_resource'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:191:in `call'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:191:in `block (2 levels) in evaluate'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:546:in `block in thinmark'
/opt/puppetlabs/puppet/lib/ruby/2.5.0/benchmark.rb:308:in `realtime'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:545:in `thinmark'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:191:in `block in evaluate'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/graph/relationship_graph.rb:122:in `traverse'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction.rb:178:in `evaluate'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:240:in `block (2 levels) in apply'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:546:in `block in thinmark'
/opt/puppetlabs/puppet/lib/ruby/2.5.0/benchmark.rb:308:in `realtime'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:545:in `thinmark'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:239:in `block in apply'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/log.rb:165:in `with_destination'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/transaction/report.rb:152:in `as_logging_destination'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/resource/catalog.rb:238:in `apply'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:196:in `block (2 levels) in apply_catalog'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:546:in `block in thinmark'
/opt/puppetlabs/puppet/lib/ruby/2.5.0/benchmark.rb:308:in `realtime'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:545:in `thinmark'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:195:in `block in apply_catalog'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:233:in `block in benchmark'
/opt/puppetlabs/puppet/lib/ruby/2.5.0/benchmark.rb:308:in `realtime'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:232:in `benchmark'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:194:in `apply_catalog'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:412:in `run_internal'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:251:in `block in run'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/context.rb:62:in `override'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet.rb:310:in `override'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/configurer.rb:221:in `run'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/agent.rb:60:in `block (5 levels) in run'
/opt/puppetlabs/puppet/lib/ruby/2.5.0/timeout.rb:93:in `block in timeout'
/opt/puppetlabs/puppet/lib/ruby/2.5.0/timeout.rb:103:in `timeout'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/agent.rb:59:in `block (4 levels) in run'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/agent/locker.rb:21:in `lock'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/agent.rb:53:in `block (3 levels) in run'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/agent.rb:136:in `with_client'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/agent.rb:50:in `block (2 levels) in run'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/agent.rb:101:in `run_in_fork'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/agent.rb:49:in `block in run'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application.rb:179:in `controlled_run'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/agent.rb:47:in `run'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application/agent.rb:437:in `onetime'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application/agent.rb:397:in `block in run_command'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/context.rb:62:in `override'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet.rb:310:in `override'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application/agent.rb:393:in `run_command'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application.rb:382:in `block in run'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util.rb:735:in `exit_on_fail'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/application.rb:382:in `run'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/command_line.rb:143:in `run'
/opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/util/command_line.rb:77:in `execute'
/opt/puppetlabs/puppet/bin/puppet:5:in `<main>'

Any additional information you'd like to impart

The resource fails because of legacy instance configuration in this definition,

puppet-elasticsearch/lib/puppet/provider/elasticsearch_keystore/ruby.rb

Line 30 in 7ef7983

def self.run_keystore(args, instance, configdir = '/etc/elasticsearch', stdin = nil)

[psychonaut]

this actually made secrets management unusable with new module

[psychonaut]

I've created a hackish, drop-in replacement for elasticsearch_keystore provider: https://github.com/psychonaut/puppet-elasticsearch/blob/keystore/lib/puppet/provider/elasticsearch_keystore/ruby.rb - it works but to be honest, I don't like this approach: since in new module there's only one keystore there's no point of creating resource for keystore. Moreover, even in the old instance version, it didn't check the value of the secret, it only checked if a secret with this name exists.

The new approach should be built around secrets, not keystores. So instead of

elasticsearch_keystore { 'whatever': # it's no longer relevant what you put here since there's only one keystore
  ..
  settings => {secret1_key => secret1_value, secret2_key => secret2_value},
}

it should be

elasticsearch_secret { 'secret_key':
  ensure => absent|present
  value  => secret_value,
}

[victorvarias]

Here is another "dirty" solution similar to the one provided by @psychonaut but also solves #983
victorvarias@1b02c89