extra-packages make cabal install fail

[andreabedini]

Describe the bug

It looks like the mere existence of a package in extra-packages make cabal install fail.

To Reproduce

mkdir temp-cabal-install && cd temp-cabal-install
cabal init -n -m
echo -e 'packages: .\nextra-packages: lens' > cabal.project
cabal v2-install temp-cabal-install --dry-run

The output is

Wrote tarball sdist to
/home/andrea/temp-cabal-install/dist-newstyle/sdist/temp-cabal-install-0.1.0.0.tar.gz
Got NamedPackage lens
CallStack (from HasCallStack):
  error, called at src/Distribution/Client/CmdInstall.hs:524:33 in cabal-install-3.10.1.0-inplace:Distribution.Client.CmdInstall

Notes:

• The same happens using the more explicit target temp-cabal-install:exe:temp-cabal-install
• Adding -v3 shows that this happens after planning. Indeed dist-newstyle/cache/plan.json is created and it seems correct (it includes both lens and temp-cabal-install)
• cabal v2-build has no issues
• There seem to be some re-planning when alternating between cabal v2-build temp-cabal-install --dry-run and the failing cabal v2-install temp-cabal-install --dry-run. You can see the replanning using -v2 or -v3 but dist-newstyle/cache/plan.json does not change (according to sha256sum).

Expected behavior

I'd expect cabal to be able to install temp-cabal-install:exe:temp-cabal-install just like without the line extra-packages: lens in cabal.project (which of course works).

System information

• x86_64 GNU/Linux
• ghc-9.2.6
• cabal-3.6.2.0, cabal-3.8.1.0 and cabal-3.10.1.0 seem all affected.

Additional context

In the cardano-node repo we moved from using source-repository-packages to external repository for some dependencies not present on Hackage. So in IntersectMBO/cardano-node#4921 the following lines were removed from cabal.project

source-repository-package
  type: git
  location: https://github.com/input-output-hk/ouroboros-network
  tag: 679c7da2079a5e9972a1c502b6a4d6af3eb76945
  --sha256: 138mqd5cv0b13giwjvlz3pr6l1cwgpn38n0q3m11mrjwwmmxl0mw
  subdir:
    monoidal-synchronisation
    network-mux
    ouroboros-consensus
    ouroboros-consensus-byron
    ouroboros-consensus-cardano
    ouroboros-consensus-protocol
    ouroboros-consensus-shelley
    ouroboros-network
    ouroboros-network-framework
    ouroboros-network-testing
    ouroboros-consensus-cardano-tools

This had the unintended effect of removing the executables ouroboros-consensus-cardano-tools:exe:db-analyser and ouroboros-consensus-cardano-tools:exe:db-synthesizer from the plan, since ouroboros-consensus-cardano-tools had no other reasons to be brought into the plan other than the, now removed, source-repository-package stanza.

Our nix setup relied on those executables being built as part of the cabal project so I added extra-packages: ouroboros-consensus-cardano-tools to have them back into the plan. This fixed the nix setup, but broke cabal install.

The package cardano-cli is a local package (in packages:) and now cabal install cardano-cli fails as described above.

We will be able to find a workaround but I am a bit surprised by this failure. I haven't yet looked at the code.

[Mikolaj]

Does cabal 3.8.1.0 fail similarly?

[andreabedini]

Yes, cabal-3.6.2.0, cabal-3.8.1.0 and cabal-3.10.1.0 seem all affected.

[Mikolaj]

Thanks. That seems to be a part of the known problem that cabal install is almost not tested at all and apparently not fully implemented.

If we had a minimal reproducer for this problem included in our test suite, that would be a great start to fixing it.