ext_pillar data independent of salt/pillar environment

[BenjaminSchiborr]

Description of Issue/Question

We have a setup where we store very static customer information in a git pillar. This information should be available to all salt/pillar environments. This is especially important during development when we use dynamic pillar/salt environments (e.g. equal to ticket numbers).

Question: In the setup below, how can we make sure that the information from the master branch of git repository "cc:/v1/repos/customer_pillar" is available for all salt/pillar environments?

Setup

Example snippet from a master config:

ext_pillar:
  - git:
    - __env__ cc:/v1/repos/salt:
      - root: pillar
    - master cc:/v1/repos/customer_pillar


ext_pillar_first: True
pillarenv_from_saltenv: True
top_file_merging_strategy: same
pillar_merge_lists: True
pillar_source_merging_strategy: recurse

Example top.sls in customer_pillar repo:

{{saltenv}}:
    '*':
        - customer.my_customer

Steps to Reproduce Issue

1. Reference your pillar repository in the master config
2. On the master branch add the top.sls and a file "my_customer.sls" in a customer folder
3. Add some random pillar data
4. Call "salt '*' pillar.get foo pillarenv=base" <-- See that it works
5. Call "salt '*' pillar.get foo pillarenv=IDONTEXIST" <-- See that it does not work!
6. How can I get the call to work for any pillarenv?

Versions Report

Salt Version:
           Salt: 2017.7.5

Dependency Versions:
           cffi: Not Installed
       cherrypy: unknown
       dateutil: 2.4.2
      docker-py: Not Installed
          gitdb: 0.6.4
      gitpython: 1.0.1
          ioflo: Not Installed
         Jinja2: 2.8
        libgit2: Not Installed
        libnacl: Not Installed
       M2Crypto: Not Installed
           Mako: Not Installed
   msgpack-pure: Not Installed
 msgpack-python: 0.4.6
   mysql-python: Not Installed
      pycparser: Not Installed
       pycrypto: 2.6.1
   pycryptodome: Not Installed
         pygit2: Not Installed
         Python: 2.7.12 (default, Dec  4 2017, 14:50:18)
   python-gnupg: Not Installed
         PyYAML: 3.11
          PyZMQ: 15.2.0
           RAET: Not Installed
          smmap: 0.9.0
        timelib: Not Installed
        Tornado: 4.2.1
            ZMQ: 4.1.4

System Versions:
           dist: Ubuntu 16.04 xenial
         locale: ANSI_X3.4-1968
        machine: x86_64
        release: 4.4.0-79-generic
         system: Linux
        version: Ubuntu 16.04 xenial

[Ch3LL]

I'm not aware of an environment in pillars that will run every time if using a different pillar environment. ping @saltstack/team-core are you aware of anything? or just this could be a feature request?

[gtmanfred]

I am not aware of anything in this but @terminalmage would be the one to know for certain.

[scallister]

There is a new feature in 2018.3.0 called all_saltenvs, unfortunately it only seems to be available for states and not for pillars. I'm also in need of a way to have pillars available in every salt environment.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.

[scallister]

Commenting to indicate this isn't stale and is still a needed feature 👍.

[stale]

Thank you for updating this issue. It is no longer marked as stale.

[sathieu]

@scallister all_saltenvs is also available for git_pilar. You'll need #50768 and #51661 (ex-#51597).

Also look at fallback parameter (#51971).

I think that all of those are in 2019.2.1+ (not sure).

[scallister]

@sathieu thank you, I'll take a look at those! :)

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

If this issue is closed prematurely, please leave a comment and we will gladly reopen the issue.

[sagetherage]

yes, this does look like a Feature Request updating and will attempt to get the discussion going.

[stale]

Thank you for updating this issue. It is no longer marked as stale.

[garethgreenaway]

@scallister Following up on this one, based on the comment above from @sathieu mentioning various PRs going into 2019.2.1, has the feature you've requested been implemented into Salt? Thanks!

[scallister]

@garethgreenaway those look like it should fix the issue. I've since switched employers and don't readily have a Salt setup, particularly one using this feature so I can't test it easily.

From my perspective I think we're good to close this out for now. Thanks for following up ❤️.

cc'ing @BenjaminSchiborr in case this still needs to stay open 👍.