[BUG] [ERROR ] Module/package collision

[TheBigBear]

Description
After upgrading the Linux master to the latest version 3007.0 we get an error on every invocation of salt, saying:

[ERROR   ] Module/package collision: '/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/__pycache__/vault.cpython-310.pyc' and '/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/vault' twice. 

Setup
salt master is setup on a AlmaLinux 8.9

Please be as specific as possible and give set-up details.

• on-prem machine
• VM (Virtualbox, KVM, etc. please specify)
• VM running on a cloud service, please be explicit and add details
• VM salt master running on microsoft azure
• container (Kubernetes, Docker, containerd, etc. please specify)
• or a combination, please be explicit
• jails if it is FreeBSD
• classic packaging
• onedir packaging
• used bootstrap to install

Steps to Reproduce the behavior
salt myminion1 test.ping

[ERROR   ] Module/package collision: '/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/__pycache__/vault.cpython-310.pyc' and '/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/vault'
[ERROR   ] Module/package collision: '/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/__pycache__/vault.cpython-310.pyc' and '/opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/vault'
myminion1
    True

Expected behavior
salt myminion1 test.ping

myminion1:
    True

It should not output the two lines of [ERROR ]

Screenshots
If applicable, add screenshots to help explain your problem.

Versions Report

salt --versions-report

(Provided by running salt --versions-report. Please also mention any differences in master/minion versions.)

salt --versions-report
Salt Version:
          Salt: 3007.0

Python Version:
        Python: 3.10.13 (main, Feb 19 2024, 03:31:20) [GCC 11.2.0]

Dependency Versions:
          cffi: 1.16.0
      cherrypy: unknown
      dateutil: 2.8.2
     docker-py: Not Installed
         gitdb: Not Installed
     gitpython: Not Installed
        Jinja2: 3.1.3
       libgit2: Not Installed
  looseversion: 1.3.0
      M2Crypto: Not Installed
          Mako: Not Installed
       msgpack: 1.0.7
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     packaging: 23.1
     pycparser: 2.21
      pycrypto: Not Installed
  pycryptodome: 3.19.1
        pygit2: Not Installed
  python-gnupg: 0.5.2
        PyYAML: 6.0.1
         PyZMQ: 25.1.2
        relenv: 0.15.1
         smmap: Not Installed
       timelib: 0.3.0
       Tornado: 6.3.3
           ZMQ: 4.3.4

Salt Package Information:
  Package Type: onedir

System Versions:
          dist: almalinux 8.9 Midnight Oncilla
        locale: utf-8
       machine: x86_64
       release: 4.18.0-513.18.1.el8_9.x86_64
        system: Linux
       version: AlmaLinux 8.9 Midnight Oncilla

Additional context
Add any other context about the problem here.

[TheBigBear]

I got rid of this error using this cmd line, but I am not sure I should have to resort to this sort of heavy handed python cache management across all linux minions?
salt -G 'kernel:Linux' cmd.run 'rm -f /opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/__pycache__/*' -l info

[Vengeance1776]

I am also seeing this issue.

[jennywatuncch]

I am seeing this issue also

[z900collector]

Same/similar to #66177
#66177

[themowski]

We are also seeing this exact issue after a 3006.7 to 3007 upgrade on AlmaLinux 8.

[Vengeance1776]

I got rid of this error using this cmd line, but I am not sure I should have to resort to this sort of heavy handed python cache management across all linux minions? salt -G 'kernel:Linux' cmd.run 'rm -f /opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/__pycache__/*' -l info

Thanks, TheBigBear, this was helpful for me.

[ssoto2]

Also seeing this error on Rocky for my master.

salt --versions-report
Salt Version:
          Salt: 3007.0
 
Python Version:
        Python: 3.10.13 (main, Feb 19 2024, 03:31:20) [GCC 11.2.0]
 
Dependency Versions:
          cffi: 1.16.0
      cherrypy: unknown
      dateutil: 2.8.2
     docker-py: Not Installed
         gitdb: 4.0.10
     gitpython: 3.1.32
        Jinja2: 3.1.3
       libgit2: Not Installed
  looseversion: 1.3.0
      M2Crypto: Not Installed
          Mako: Not Installed
       msgpack: 1.0.7
  msgpack-pure: Not Installed
  mysql-python: Not Installed
     packaging: 23.1
     pycparser: 2.21
      pycrypto: Not Installed
  pycryptodome: 3.19.1
        pygit2: Not Installed
  python-gnupg: 0.5.2
        PyYAML: 6.0.1
         PyZMQ: 25.1.2
        relenv: 0.15.1
         smmap: 5.0.0
       timelib: 0.3.0
       Tornado: 6.3.3
           ZMQ: 4.3.4
 
Salt Package Information:
  Package Type: onedir
 
System Versions:
          dist: rocky 8.9 Green Obsidian
        locale: utf-8
       machine: x86_64
       release: 4.18.0-513.24.1.el8_9.x86_64
        system: Linux
       version: Rocky Linux 8.9 Green Obsidian

[matthewsht]

We as well - note that this is on a minion.

[root@mon-zabbix-prod pycache]# salt-call --versions-report
Salt Version:
Salt: 3007.0

Python Version:
Python: 3.10.13 (main, Feb 19 2024, 03:31:20) [GCC 11.2.0]

Dependency Versions:
cffi: 1.16.0
cherrypy: 18.8.0
dateutil: 2.8.2
docker-py: Not Installed
gitdb: Not Installed
gitpython: Not Installed
Jinja2: 3.1.3
libgit2: Not Installed
looseversion: 1.3.0
M2Crypto: Not Installed
Mako: Not Installed
msgpack: 1.0.7
msgpack-pure: Not Installed
mysql-python: Not Installed
packaging: 23.1
pycparser: 2.21
pycrypto: Not Installed
pycryptodome: 3.19.1
pygit2: Not Installed
python-gnupg: 0.5.2
PyYAML: 6.0.1
PyZMQ: 25.1.2
relenv: 0.15.1
smmap: Not Installed
timelib: 0.3.0
Tornado: 6.3.3
ZMQ: 4.3.4

Salt Package Information:
Package Type: onedir

System Versions:
dist: centos 8 n/a
locale: utf-8
machine: x86_64
release: 4.18.0-552.1.1.el8.x86_64
system: Linux
version: CentOS Stream 8 n/a

[xion824]

Still having the same error on RHEL 8, RHEL 9, Rocky 9 with 3007.1. Removing files under /opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/__pycache__/ does not solve the issue (used the command salt -G 'kernel:Linux' cmd.run 'rm -f /opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/__pycache__/*' -l info). Anyone find an actual fix for this?

[TheBigBear]

Still having the same error on RHEL 8, RHEL 9, Rocky 9 with 3007.1. Removing files under /opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/pycache/* does not solve the issue. Anyone find an actual fix for this?

the path is different from what your post says:  '__pycache__' not '**pycache**' that is two underscores the word pycache foloowed by another two underscores.

salt -G 'kernel:Linux' cmd.run 'rm -f /opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/__pycache__/*' -l info

[xion824]

Still having the same error on RHEL 8, RHEL 9, Rocky 9 with 3007.1. Removing files under /opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/pycache/* does not solve the issue. Anyone find an actual fix for this?

the path is different from what your post says:  '__pycache__' not '**pycache**' that is two underscores the word pycache foloowed by another two underscores.

salt -G 'kernel:Linux' cmd.run 'rm -f /opt/saltstack/salt/lib/python3.10/site-packages/salt/utils/__pycache__/*' -l info

Fixed my post. I had the correct text, but forgot to put it in code. That ended up removing the double undercores.

[TheBigBear]

@xion824 ah i see. Are your site-packages maybe under a different path? Look for the salt/utils/__pychache__/ set of subdirs . Are you removing them under the master or the minions, or both? ( under the minions the python site-packages path might be different or your python version might be different ?

[xion824]

@TheBigBear Same location. So I reverted back to 3006.4 and retried updating to 3007.0, then ran the command to clear the __pycache__ files. Error went away. Until I updated to 3007.1, and the error came back again. Hoping this won't be an every update kind of problem.