Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix for 54941 pillar_refresh regression #54942

Merged
merged 3 commits into from
Oct 11, 2019
Merged

Conversation

dwoz
Copy link
Contributor

@dwoz dwoz commented Oct 9, 2019

What does this PR do?

Only refresh pillar data when BaseMinion.gen_modules is called for the first time. Subsequent pillar refreshes are done when the pillar_refresh event if fired.

What issues does this PR fix or reference?

#54941

Tests written?

Yes

Commits signed with GPG?

Yes

@dwoz dwoz requested a review from a team as a code owner October 9, 2019 23:19
@ghost ghost requested a review from Ch3LL October 9, 2019 23:19
@dwoz dwoz changed the title [WIP] Fix for 54941 Fix for 54941 pillar_refresh regression Oct 10, 2019
Copy link
Contributor

@UtahDave UtahDave left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's one more scenario I think I'd love to see tested. I'm not sure exactly where you would add this, but it would be great if in one of these spots you add this workflow:

  1. Add a key/value pair to the pillar sls file
  2. Execute a test.ping
  3. Test that the pillar data has NOT been updated with the new key/value pair yet.
  4. Run a saltutil.refresh_pillar
  5. Then verify that the pillar data has been updated.

The reason for this is it would be great to make sure that a full pillar refresh isn't happening during a regular salt command.

Is this scenario already covered by one of these tests you already have?

@dwoz
Copy link
Contributor Author

dwoz commented Oct 10, 2019

There's one more scenario I think I'd love to see tested. I'm not sure exactly where you would add this, but it would be great if in one of these spots you add this workflow:

  1. Add a key/value pair to the pillar sls file
  2. Execute a test.ping
  3. Test that the pillar data has NOT been updated with the new key/value pair yet.
  4. Run a saltutil.refresh_pillar
  5. Then verify that the pillar data has been updated.

The reason for this is it would be great to make sure that a full pillar refresh isn't happening during a regular salt command.

Is this scenario already covered by one of these tests you already have?

I thought about adding this exact test because it most closely mirrors the way we found the regression. The only reason I didn't is that we're essentially using pillar.get, pillar.item, and pillar.raw calls in place of test.ping.

@UtahDave
Copy link
Contributor

There's one more scenario I think I'd love to see tested. I'm not sure exactly where you would add this, but it would be great if in one of these spots you add this workflow:

  1. Add a key/value pair to the pillar sls file
  2. Execute a test.ping
  3. Test that the pillar data has NOT been updated with the new key/value pair yet.
  4. Run a saltutil.refresh_pillar
  5. Then verify that the pillar data has been updated.

The reason for this is it would be great to make sure that a full pillar refresh isn't happening during a regular salt command.
Is this scenario already covered by one of these tests you already have?

I thought about adding this exact test because it most closely mirrors the way we found the regression. The only reason I didn't is that we're essentially using pillar.get, pillar.item, and pillar.raw calls in place of test.ping.

OK, valid point. You are correct. Those calls would trigger the bad behavior, if present.

@dwoz dwoz merged commit 2f817bc into saltstack:2019.2.1 Oct 11, 2019
DmitryKuzmenko added a commit to DSRCorporation/salt that referenced this pull request May 10, 2020
Allow to refresh pillar synchronously
This PR adds the optional ability to run saltutil.refresh_pillar Allow
to refresh pillar synchronously, saying saltutil.refresh_pillar
async=False. I tried to follow the explanatory comment from @gtmanfred
and I believe that this works, still I am not sure about the mentioned
use of the _gather_pillar() method. I am happy to rework the PR if
something is missing.

[DKuzmenko] I've slightly reworked it to resolve conflicts with
functionality added to master by DWoz in saltstack#54942.

Co-authored-by: Dmitry Kuzmenko <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants