py spyder-notebook with nix spyder: Found duplicated packages in closure for dependency 'setuptools-scm':

[InLaw]

let
  ref = "refs/tags/3.1.1" ; processMode = "mksh"; versionPy = "38"; testPkgPy = ""; testPkgNix = "";
  mach-nix = import (builtins.fetchGit {
    url = "https://github.com/DavHau/mach-nix/";
    ref = "${ref}" ; 
  }) {  
      #pkgs= import <nixpkgs>  {config= { allowUnfree = true; }; };
      python = "python${versionPy}"; 
};
  nixPkgs = import mach-nix.nixpkgs.path {} ; 
  py = nixPkgs."python${versionPy}Packages" ;
in let
  envPy = mach-nix.mkPython rec {
    requirements =  ''
        ipython 
        jupyterlab
        pandas

        spyder-notebook # Found duplicated packages in closure for dependency 'setuptools-scm':
        #spyder-terminal # 
        #spyder-unittest 

      ''; 

      packagesExtra = []
          ++ [
            # nixPkgs.spyder
          ]
      ; 

  }; 
in 
 # env = 
  nixPkgs.buildEnv rec { 
    name = "bEnvJL";    
    paths =
        [ (envPy) ]  
         ++ [
            nixPkgs.spyder
          ]
      ;
  }

---

Found duplicated packages in closure for dependency 'setuptools-scm':
  setuptools-scm 4.1.2 (/nix/store/mzxxiwvr1a65h3q5i2b2khbn7kv60762-python3.8-setuptools_scm-4.1.2/lib/python3.8/site-packages)
  setuptools-scm 4.1.2 (/nix/store/xkdxgqh405wzy6ggp0brpkgkjpz3ll4m-python3.8-setuptools-scm-4.1.2/lib/python3.8/site-packages)

Package duplicates found in closure, see above. Usually this happens if two packages depend on different version of the same dependency.
builder for '/nix/store/wwgsdlqhip493prn0qjp5mw7gia3jq9x-python3.8-black-20.8b1.drv' failed with exit code 1
cannot build derivation '/nix/store/6cplm4922yx7fv56dy354rbh0df91m6l-python3-3.8.6-env.drv': 1 dependencies couldn't be built
cannot build derivation '/nix/store/f9rn4vm2km2iql96acpab76aiv24323p-bEnvJL.drv': 1 dependencies couldn't be built
error: build of '/nix/store/f9rn4vm2km2iql96acpab76aiv24323p-bEnvJL.drv' failed

[bjornfor]

I hit the same thing today, by simply pulling in black~=20.8b1.

Reproducer:

mach-nix on master [$?] 
$ rm result && nix-build -A mach-nix && ./result/bin/mach-nix env -r <(echo black~=20.8b1) env && nix-shell env --run true
[...]
Finished executing pythonRemoveTestsDir
pythonCatchConflictsPhase
Found duplicated packages in closure for dependency 'setuptools-scm':
  setuptools-scm 4.1.2 (/nix/store/ngk96p2yjwf6l5221vh4665hk2d357qm-python3.7-setuptools_scm-4.1.2/lib/python3.7/site-packages)
  setuptools-scm 4.1.2 (/nix/store/ih5vp9l9jspwf6bskiy1yxspdssjb610-python3.7-setuptools-scm-4.1.2/lib/python3.7/site-packages)

Package duplicates found in closure, see above. Usually this happens if two packages depend on different version of the same dependency.
builder for '/nix/store/7vawjy753d44449rrbpgych3d7y741wm-python3.7-black-20.8b1.drv' failed with exit code 1

It works with black though, but that doesn't help in my case -- I need the latest.

[DavHau]

This is resolved by 6473903

[bjornfor]

It still fails for me:

$ cd mach-nix
$ git pull
...

$ git rev-parse HEAD                                                                                                                                                                                                                          
6473903da7e17b11f7c41f5729713bf46f84e6c2

$ rm -f result && nix-build -A mach-nix && ./result/bin/mach-nix env -r <(echo black~=20.8b1) env && nix-shell env --run true
...
pythonCatchConflictsPhase
Found duplicated packages in closure for dependency 'setuptools-scm':
  setuptools-scm 4.1.2 (/nix/store/ngk96p2yjwf6l5221vh4665hk2d357qm-python3.7-setuptools_scm-4.1.2/lib/python3.7/site-packages)
  setuptools-scm 4.1.2 (/nix/store/ih5vp9l9jspwf6bskiy1yxspdssjb610-python3.7-setuptools-scm-4.1.2/lib/python3.7/site-packages)

Package duplicates found in closure, see above. Usually this happens if two packages depend on different version of the same dependency.
builder for '/nix/store/7vawjy753d44449rrbpgych3d7y741wm-python3.7-black-20.8b1.drv' failed with exit code 1

[DavHau]

@bjornfor The problem there is, whenever you use the mach-nix cmdline tool to initialize the env, it always defaults to the latest stable version, even if you use the cmdline tool from the master branch. Should we change this behaviour?

[bjornfor]

The problem there is, whenever you use the mach-nix cmdline tool to initialize the env, it always defaults to the latest stable version, even if you use the cmdline tool from the master branch. Should we change this behaviour?

Yes please! How else would I get your fix into my mach-nix CLI?

Btw, am I missing something here, what's the benefit of the current behaviour? (Why is it even a question? 😄 )

[DavHau]

Btw, am I missing something here, what's the benefit of the current behaviour? (Why is it even a question? smile )

The reason is mostly, that I'm not sure how to implement that.
The problem is, that mach-nix needs to write some version reference of itself inside the lock.toml file inside the env. Currently the mach-nix tool just reads its own version via the ./mach-nix/VERSION file, queries the github api for the revision of that tag and uses that.
If mach-nix was always executed via flakes, we could use the revision from there, but I'm not sure if we want to force all users to use flakes.
With the current mechanism I could always change back the ./mach-nix/VERSION to master after doing a release. But then, if you are on any other branch, it will still initialize with the master branch of mach-nix.
Any ideas a welcome ;)

[bjornfor]

Maybe you can get the revision from builtins.fetchGit here, and then write the .rev attribute to a file in the output?

If that works, it'd have at least one downside: it'd break if I built from a local commit that doesn't exit upstream.

Alternatively, make releases very often 😄

[DavHau]

Maybe you can get the revision from builtins.fetchGit here, and then write the .rev attribute to a file in the output?

How do I know which revision to fetch via fetchGit?
How would you get the revision when installed from a tarball?

[DavHau]

I changed the VERSION file to point to master for now. see afa94cb

[bjornfor]

How do I know which revision to fetch via fetchGit?

builtins.fetchGit ./. returns the revision, you don't need to pass it.

How would you get the revision when installed from a tarball?

Good question. I guess you don't, and would have to generate tarballs yourself :-/

I changed the VERSION file to point to master for now.

Thanks.