-
Notifications
You must be signed in to change notification settings - Fork 5.5k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[BUG] salt-ssh support for extra_filerefs in Saltfile is broken #60003
Comments
@edgan Can you provide more information to reproduce this is since I have been able to replicate the issue using Salt 3002.2 with Centos 7 for master and minion. I may have a misconfiguration causing similar response.
With top.sls
and /srv/pillar/base.sls
with roster
with _grains/map.jinja
and Saltfile
|
I have attached the bare minimum version of my setup. This fails in the same way with 3003 and works with 3002.6. Note I do have to run No pillars |
FWIW, we've run into the same issue and have had to stay on salt 3002 as a result. I've tried to find some other workarounds but for now we'll just avoid upgrading to 3003 until there's a fix or someone figures out a good workaround. |
I believe I have very similar issue. My minimal test case:
After debugging it a bit, I've found it is about cached FileClient instance. I've added the following debug/workaround patch: --- /usr/lib/python3.9/site-packages/salt/utils/jinja.py.orig 2021-05-07 12:08:45.897000000 +0200
+++ /usr/lib/python3.9/site-packages/salt/utils/jinja.py 2021-05-07 12:08:53.543000000 +0200
@@ -99,6 +99,11 @@ class SaltCacheLoader(BaseLoader):
if self._file_client is None:
attr = "_cached_pillar_client" if self.pillar_rend else "_cached_client"
cached_client = getattr(self, attr, None)
+ if cached_client is not None and cached_client.opts['cachedir'] != self.opts['cachedir']:
+ log.error('Cached FileClient instance has mismatching cachedir: %s != %s',
+ cached_client.opts['cachedir'], self.opts['cachedir'])
+ cached_client = None
+
if cached_client is None:
cached_client = salt.fileclient.get_file_client(
self.opts, self.pillar_rend
And it works, and shows what's the issue at the same time:
That said, I have no idea why this happens (especially - why cachedir changes at some point, still within master context) and what is the proper fix. Any help would be appreciated. salt-ssh --versions-report
|
Unbreak salt using dirty hack, until upstream applies the proper fix. This is executed in a DispVM, so the change is not persistent - when upstream fix will be there, it's enough to simply drop this line. Workaround for saltstack/salt#60003 Fixes QubesOS/qubes-issues#6580
Unbreak salt using dirty hack, until upstream applies the proper fix. This is executed in a DispVM, so the change is not persistent - when upstream fix will be there, it's enough to simply drop this line. Workaround for saltstack/salt#60003 Fixes QubesOS/qubes-issues#6580 (cherry picked from commit 5d3b029)
I think I've bumped into this today. Applying the fix above broke imports. Is it going to get fixed soon? |
I added the following snippet at the beginning of the
There are two observations:
--- /tmp/trace3002-6.txt 2021-07-10 11:48:11.000000000 +0700
+++ /tmp/trace3003-1.txt 2021-07-10 11:48:14.000000000 +0700
@@ -1,4 +1,4 @@
-ARGS ('', 'salt://filerefs/map.jinja')
+ARGS ('', '')
File "/Users/user/.virtualenvs/salt3003/bin/salt-ssh", line 7, in <module>
exec(compile(f.read(), __file__, 'exec'))
File "/Users/user/repos/saltstack-repo/scripts/salt-ssh", line 9, in <module>
@@ -33,6 +33,14 @@
stdout, retcode = self.run_wfunc()
File "/Users/user/repos/saltstack-repo/salt/client/ssh/__init__.py", line XXXX, in run_wfunc
result = self.wfuncs[self.fun](*self.args, **self.kwargs)
+ File "/Users/user/repos/saltstack-repo/salt/loader.py", line 1241, in __call__
+ return self.loader.run(run_func, *args, **kwargs)
+ File "/Users/user/repos/saltstack-repo/salt/loader.py", line 2274, in run
+ return self._last_context.run(self._run_as, _func_or_method, *args, **kwargs)
+ File "/Users/user/.virtualenvs/salt3003/lib/python3.6/site-packages/contextvars/__init__.py", line 38, in run
+ return callable(*args, **kwargs)
+ File "/Users/user/repos/saltstack-repo/salt/loader.py", line 2289, in _run_as
+ return _func_or_method(*args, **kwargs)
File "/Users/user/repos/saltstack-repo/salt/client/ssh/wrapper/state.py", line XXX, in apply_
return sls(mods, **kwargs)
File "/Users/user/repos/saltstack-repo/salt/client/ssh/wrapper/state.py", line XXX, in sls I also bisected the Git history down to the following two pull requests: The following quick and dirty patch for 3003.1 fixes just the diff --git a/salt/client/ssh/__init__.py b/salt/client/ssh/__init__.py
index b6bf987117..952a943a92 100644
--- a/salt/client/ssh/__init__.py
+++ b/salt/client/ssh/__init__.py
@@ -1196,6 +1196,9 @@ def run_wfunc(self):
opts["grains"][grain] = self.target["grains"][grain]
opts["pillar"] = data.get("pillar")
+ extra_filerefs = opts.get("__master_opts__", {}).get("extra_filerefs")
+ if extra_filerefs:
+ opts["extra_filerefs"] = extra_filerefs
wrapper = salt.client.ssh.wrapper.FunctionWrapper(
opts,
self.id, This is definitely not the right fix, but I hope it can unblock your workflows. A proper one requires thorough understanging of Loader Contexts and how options are passed between those. The only thing I can do is ping @dwoz to help with that. I also have another patch to avoid specifying an |
And I strongly suggest making a proper fix not just in Silicon, but also backporting it to Aluminium (3003.2). This is a significant regression and it doesn't make sense to leave 3003.x series broken while it is still supported. |
FWI: This is the patch I use since v3002.1: diff --git a/salt/client/ssh/wrapper/state.py b/salt/client/ssh/wrapper/state.py
index 1a13af5eda..db8d2ab0c5 100644
--- a/salt/client/ssh/wrapper/state.py
+++ b/salt/client/ssh/wrapper/state.py
@@ -5,6 +5,7 @@ Create ssh executor system
import logging
import os
import time
+from copy import deepcopy
import salt.client.ssh.shell
import salt.client.ssh.state
@@ -662,15 +663,20 @@ def highstate(test=None, **kwargs):
st_kwargs = __salt__.kwargs
__opts__["grains"] = __grains__.value()
opts = salt.utils.state.get_sls_opts(__opts__, **kwargs)
+ highstate_opts = deepcopy(opts)
+ highstate_opts["extra_filerefs"] = opts["__master_opts__"].get("extra_filerefs", "")
+ highstate_opts["cachedir"] = opts["__master_opts__"].get("cachedir", "")
st_ = salt.client.ssh.state.SSHHighState(
- opts, __pillar__.value(), __salt__.value(), __context__["fileclient"]
+ highstate_opts, __pillar__.value(), __salt__.value(), __context__["fileclient"]
)
st_.push_active()
chunks = st_.compile_low_chunks()
file_refs = salt.client.ssh.state.lowstate_file_refs(
chunks,
_merge_extra_filerefs(
- kwargs.get("extra_filerefs", ""), opts.get("extra_filerefs", "")
+ kwargs.get("extra_filerefs", ""),
+ opts.get("extra_filerefs", ""),
+ opts.get("__master_opts__", {}).get("extra_filerefs", "")
),
)
# Check for errors
--
2.25.1 Had to modify it a bit for v3003.1. Tested it. Works. [edit]I don't use If someone verifies that this patch fixes the issues, I'd be happy to open a PR.[/edit] |
Salt 3005 adds check if |
salt-ssh constructs file client with cachedir set to /var/tmp/... already (as it would be on the minion side). Make sure it isn't used when looking for files on the master (with cachedir /var/cache/salt or similar), as it wouldn't find them. related to saltstack#60003
This makes it detect cases where remote/local paths gets confused (like in saltstack#60003 or saltstack#62043). Without using a container, remote paths works locally too, so tests do not detect an issue. Rename test_ssh_setup.py to test_ssh_setup_and_state.py to match its new content. pytest.mark.skip_on_windows is dropped as test_ssh_setup.py already has necessary mark (dockerd present).
Here is the fix if anyone wants to test: #63184 |
I think maybe this already works now? I just tried on 3005.1. |
Its definitely broken in the example the original submitter provided. It does work in certain scenarios though, so I'm guessing your use case fits that. |
Ah, yeah, I'm not doing import/include in top, but in other files, so I guess it works there now, at least. |
Description
I use lots of map.jinjas with salt-ssh, as part of many formulas in my states git repo. Since the files are only imported I need to use extra_filerefs to include them all. I do this in my Saltfile. As of 3003 this doesn't work anymore. If I downgrade to 3002.6, it works.
I thought it might relate to this issue below, but reverting the patch didn't seem to fix it for me. Though I do question the conversion of the type of extra_filerefs from a list to a str. The only reasonable use of extra_filerefs is as a list. I have over 70 map.jinjas, and I need them all.
https://github.com/saltstack/salt/pull/59664/files
Setup
./Saltfile:
salt-formula/top.sls:
Steps to Reproduce the behavior
Expected behavior
It to include the map.jinja in the tgz, and not throw an error about not finding it.
Versions Report
The text was updated successfully, but these errors were encountered: