Create resources not doing deep merging in hiera

[devopsprosiva]

Hi,

I use hiera to classify nodes and push plugins and checks to the sensu clients. However if I specify common checks in my global.yaml they are not being applied to the client. Only the checks defined in the client specific hiera files are being applied. As a result, I've to define the same checks in all the client hiera files. Is there anyway to enable deep merging so I can define all the common checks and plugins in the global.yaml file?

Thanks in advance.

Siva

[poolski]

@devopsprosiva how is your Hiera configured in terms of merge behaviour? Can you check /etc/hiera.yaml?

Specifically, look for a line that should say: :merge_behavior: deeper.

Also, it's worth checking this for better documentation: https://docs.puppetlabs.com/hiera/1/lookup_types.html#deep-merging-in-hiera--120

[devopsprosiva]

@poolski I've enabled deeper merge in hiera.

Thanks for sharing the link. Quoting from that link, A typical use case for hashes in Hiera is building a data structure which gets passed to the create_resources function., how can this be enabled for create_resources?

Here is my hiera.yaml.

---
:backends:
  - yaml
:hierarchy:
  - "nodes/%{::fqdn}"
  - "environment/%{::environment}"
  - "roles/%{::role}"
  - global

:yaml:
# datadir is empty here, so hiera uses its defaults:
# - /etc/puppetlabs/code/hieradata on *nix
# - %CommonAppData%\PuppetLabs\code\hieradata on Windows
# When specifying a datadir, make sure the directory exists.
  :datadir: "/etc/puppetlabs/code/hieradata/%{environment}"

:merge_behavior: deeper

[poolski]

Here's my hiera.yaml I use a JSON backend for storing the data as I don't like yaml much, but the principle is the same.

---
:backends:
  - json
:json:
  :datadir: /etc/puppet/hiera
:hierarchy:
  - nodes/%{::clientcert}
  - clients/%{::clientprefix}/nodes/%{::hostname}
  - clients/%{::clientprefix}/%{::roletype}/%{::envtype}
  - clients/%{::clientprefix}/%{::roletype}
  - roles/%{::roletype}
  - envtype/%{::envtype}
  - environments/%{::environment}
  - "%{::domain}"
  - base
:merge_behavior: deeper

As for my checks and how they're defined:

/etc/puppet/hiera/base.json - represents the last item in the hierarchy above.

{
  "sensu::checks": {
    "os-disks": {
      "command": "check-disk.rb -w 90 -c 95",
      "handlers": [
        "default"
      ],
      "high_flap_threshold": "20",
      "interval": 300,
      "low_flap_threshold": "5",
      "refresh": 1209600,
      "standalone": true
    },
    "os-load": {
      "command": "check-load.rb --per-core",
      "handlers": [
        "default"
      ],
      "high_flap_threshold": "20",
      "interval": 30,
      "low_flap_threshold": "5",
      "refresh": 1209600,
      "standalone": true
    },
    "os-memory": {
      "command": "check-ram.rb",
      "handlers": [
        "default"
      ],
      "high_flap_threshold": "20",
      "interval": 60,
      "low_flap_threshold": "5",
      "refresh": 1209600,
      "standalone": true
    }
  }
}

/etc/puppet/hiera/nodes/mynode.mynet.net.json - relates to the first entry in the hierarchy - the nodes folder.

{
  "sensu::checks": {
    "node-specific-check": {
      "command": "check-something.rb",
      "handle": false,
      "handlers": [
        "IM"
      ],
      "interval": 30,
      "refresh": 3600,
      "standalone": false,
      "subscribers": [
        "specialsnowflakes"
      ]
    }
}

Does that help a little?

*tl;dr: you need to make sure that the sensu::checks hash is present in both your "common" hiera data and your node-specific data. Hiera will look at both of these hashes and intelligently merge them

[devopsprosiva]

@poolski Thanks for the detailed example. I'll try this and let you know.

I created an issue #381 few days ago. Can you take a look and let me know if there is any workaround? I really appreciate your help

[devopsprosiva]

@poolski I tried having configuration similar to yours, but it's still not working.

• Here is my global.yaml

---
# Node classification
classes:
  - ntp
  - roles::sensu::client::core

# Sensu client settings
sensu::client: true
sensu::sensu_plugin_provider: 'sensu_gem'
sensu::sensu_plugin_version: 'present'
sensu::use_embedded_ruby: true
sensu::rabbitmq_host: 'sensuserver'
sensu::rabbitmq_port: '5672'
sensu::rabbitmq_user: 'sensu'
sensu::rabbitmq_vhost: 'sensu'
sensu::rabbitmq_password: 'sensu'
sensu::purge_config: true

sensu::checks:
  'check_cpu':
    command: '/etc/sensu/plugins/check-cpu.rb -w 75 -c 90'
    handlers: 'mail'

• This is my role specific yaml.

# Add sensu plugins
sensu::plugins:
  - "puppet:///modules/profiles/sensu/plugins/http/check-http.rb"
  - "puppet:///modules/profiles/sensu/plugins/system/check-cpu.rb"
  - "puppet:///modules/profiles/sensu/plugins/system/check-ram.rb"
  - "puppet:///modules/profiles/sensu/plugins/processes/check-procs.rb"

# Add sensu checks
sensu::checks:
  'check_ram':
    command: '/etc/sensu/plugins/check-ram.rb -c 5'
    handlers: 'mail'
  'check_tomcat':
    command: '/etc/sensu/plugins/check-procs.rb -p tomcat -C 1'
    handlers: 'mail'

• When I tested initially without adding the role specific yaml, the puppet agent used global.yaml and installed the check_cpu check.
• But when I added the role specific yaml, the puppet agent removed the check_cpu check and added check_ram and check_tomcat checks.

[jlambert121]

I'm going to go ahead and close this issue as there hasn't been any traffic on it. If there are issues with this module, please re-open this issue or create a new one.