Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

yaml-0.8.31 causes issues on GHC 7.10 and below #165

Closed
ndmitchell opened this issue Jun 29, 2018 · 36 comments
Closed

yaml-0.8.31 causes issues on GHC 7.10 and below #165

ndmitchell opened this issue Jun 29, 2018 · 36 comments

Comments

@ndmitchell
Copy link

I'm seeing this on Travis for HLint: https://travis-ci.org/ndmitchell/hlint/builds/397826785

And the Hackage matrix is showing black squares: https://matrix.hackage.haskell.org/package/yaml

CC @snoyberg. Looking at the code, it has a lower bound on base of GHC 8, so not sure what the problem is...

@jgm
Copy link

jgm commented Jun 29, 2018

I'm having the same issue with my pandoc CI builds on ghc 7.10.3 + cabal 2.0.

@jgm
Copy link

jgm commented Jun 29, 2018

--- yaml-0.8.30/yaml.cabal	2018-06-29 19:58:56.000000000 +0200
+++ yaml-0.8.31/yaml.cabal	2018-06-29 19:59:56.000000000 +0200
@@ -1,5 +1,5 @@
 name:            yaml
-version:         0.8.30
+version:         0.8.31
 license:         BSD3
 license-file:    LICENSE
 author:          Michael Snoyman <[email protected]>, Anton Ageev <[email protected]>,Kirill Simonov
@@ -46,7 +46,9 @@
 
 library
     other-extensions: LambdaCase
-    build-depends:   base >= 4 && < 5
+    if impl(ghc < 8.0.2)
+      buildable: False
+    build-depends:   base >= 4.9.1 && < 5
                    , transformers >= 0.1
                    , bytestring >= 0.9.1.4
                    , conduit >= 1.2.8 && < 1.4

Looks like the main thing that changed was raising the minimum bound for base to 4.9.1 and the restriction to ghc >= 8.0.2. But as @ndmitchell says, it's hard to see why cabal's solver didn't just reject the package.

@snoyberg is it impossible to continue to support ghc 7.10.3? We'll have to drop support for it in pandoc solely because of this package. [EDIT: Well, I suppose I can just put an upper bound on the package for older ghcs.]

@ndmitchell
Copy link
Author

@jgm as long as Cabal solver is smart enough to pick an older version (which it isn't for no obvious reason), you can continue supporting 7.10.3 even if it results in using an older yaml.

@hvr
Copy link
Contributor

hvr commented Jun 29, 2018

@ndmitchell the reason matrix.hho has black squares is that the construct if impl(ghc < 8.0.2) buildable: False on a library component is something the cabal solver doesn't like much and so doesn't backtrack;

The issues with buildable: False have known for some time (and how buildable: False and build-depends were supposed to interact with each other semantically wasn't too obvious initially either), and until recently there was no urgency to act on it; but when http-client started using this construct, a preliminary fix was created (see haskell/cabal#5337) and it's going to be available in the next cabal-2.2 point release; (but the fix is already available in my PPA's cabal-install-2.2 packages);

@jgm btw, if you just need a standards compliant YAML 1.2 parser that avoids FFI and supports GHCs back till GHC 7.4, HsYAML is currently officially the most compliant YAML 1.2 parser available :-)

@ndmitchell
Copy link
Author

@hvr perfectly happy with the yaml package, I just want it to install :)

Is the advice that @snoyberg should avoid the impl(ghc) constraint because that breaks older Cabal users? Saying that people on 5 year old compilers will be fine if only they upgraded their Cabal is really just ditching support for them :(.

@phadej
Copy link
Member

phadej commented Jun 29, 2018

@ndmitchell Old Cabal (the lib) is fine. People only need to update cabal-install (the tool), which is not a bad idea.

@jgm
Copy link

jgm commented Jun 29, 2018

@hvr thanks for the pointer about HsYaml; I'd like to use a pure Haskell library if possible. I'll try it out.

@ndmitchell
Copy link
Author

@phadej saying you can use GHC out the box if you upgrade all the things around it to me doesn't work - the people who are on old GHC are also on old Cabal and old cabal-install.

@gbaz
Copy link
Contributor

gbaz commented Jun 29, 2018

@ndmitchell true enough, but upgrading from an older ghc is hard -- you may have a codebase which is not compatible library changes, other couplings, etc. On the other hand, updating cabal-install is just replacing a binary, and should otherwise be just fine.

(also: another, certainly nonideal, workaround, would be for the cabal files of their projects, which are likely proprietary, to simply just exclude the offending version of yaml)

@snoyberg
Copy link

snoyberg commented Jul 1, 2018

Before I make a release to Hackage, can someone confirm that this release will properly work around the bug in cabal-install? snoyberg/yaml@2735b09

snoyberg added a commit to snoyberg/http-client that referenced this issue Jul 1, 2018
@snoyberg
Copy link

snoyberg commented Jul 1, 2018

And here's a workaround for http-client: snoyberg/http-client@1390d6e.

@jgm
Copy link

jgm commented Jul 1, 2018 via email

@ndmitchell
Copy link
Author

@snoyberg unfortunately my CI is set up thus that it's almost impossible to test before release is made... If you do make it thought I can give quick feedback after that.

@snoyberg
Copy link

snoyberg commented Jul 1, 2018

If you add a cabal 2.0 + ghc 7.10.3 combination to your travis CI testing, that should give you the answer.

Afaict it won't. The bug here is in the solver choosing the wrong version of the yaml package, but building a local package won't trigger that code path. I think I'll just move ahead with the release, do some Hackage revisions, and hope for the best.

@snoyberg
Copy link

snoyberg commented Jul 1, 2018

I've published a new version of yaml, and added the only revision I could that should fix this: totally breaking yaml-0.8.31 (http://hackage.haskell.org/package/yaml-0.8.31/revisions/). Can you both give it a try and let me know if it works? If so, I'll release http-client as well.

@hvr
Copy link
Contributor

hvr commented Jul 1, 2018

Before I make a release to Hackage, can someone confirm that this release will properly work around the bug in cabal-install? snoyberg/yaml@2735b09

Btw, for more or less obvious reasons that won't work due to how constraint solving works

PS: And neither the revision at http://hackage.haskell.org/package/yaml-0.8.31/revisions/ will work

PS2: It's better for you to not do anything to avoid making things worse; I'm looking into addressing this from a more direct and effective angle

PS3: If something's unclear just ask here; there's no need to go complaining on Twitter; the short gist is that backtracking doesn't kick in for the same reason we're having this problem in the first place: once buildable: False is in effect, the build-depends-constraints are ignored, so it's pointless to try to use those to steer the solver away from the affected versions.

@gbaz
Copy link
Contributor

gbaz commented Jul 1, 2018

Let me expand a bit, because I had to work it through myself. If there were no version with the buildable flag set to false, but instead of that, an unsatisfiable constraint were used, things would be fine. However, the existence of such a version means that the solver, finding the current version not satisfiable, would backtrack to the bad version, and then encounter that buildable flag, and then go kaput. My immediate thought was "can't we just unprefer the bad version" but it turns out that's not as strong as we'd like, since unprefering is not a hard blacklist, but just makes the solver try the other paths first. So if you have something that pushes the solver past the working paths, then it still follows the failing path instead of recognizing it as a "fully" not working path.

So we will try to work out a fix and sort this out, but the above approach, which does seem reasonable at first, isn't going to suffice.

@ndmitchell
Copy link
Author

Would using base versions, skipping any ghc-version test in the Cabal file itself (as this seems to lead to a pot of despair), and then having the Hackage UI display GHC version bounds based on base version bounds be the best approach?

@gbaz
Copy link
Contributor

gbaz commented Jul 1, 2018

I don' t know what the best long-term approach is. We do have an older ticket suggesting that certainly hackage displaying the base <-> ghc mapping would be a good idea, regardless: haskell/hackage-server#559

@ndmitchell
Copy link
Author

@snoyberg I confirm empirically that it's still bust for me.

hvr added a commit to haskell/hackage-server that referenced this issue Jul 1, 2018
This makes it possible to resolve issues such as
haskell-infra/hackage-trustees#165 in a simple, direct, and effective
way by using the well-proven Hackage meta-data revision facility.
@hvr
Copy link
Contributor

hvr commented Jul 1, 2018

Alright, the situation with yaml and http-client should have been addressed now (EDIT: I knew this was resolved, because I empirically verified this with affected cabal versions; and contrary to the unfoudned accusations this did not break http-client). Yet again, Hackage meta-data revisions saved the day! :-)

As a side-remark, in general I'd advise against using the if impl(ghc < x.y) construct as among other reasons, many people don't realise its subtle non-obvious semantics, it gives worse solver messages, you can't correct mistakes in the impl-condition via Hackage meta-data revisions currently, and it also violates the "test for features, not for versions" mantra, the argument of reinstallable base doesn't hold for lower bounds anyway... and so on. I noticed a blogpost trying to make the case for it but it's a flawed argument and it's really bad advice.

@hvr hvr closed this as completed Jul 1, 2018
@snoyberg
Copy link

snoyberg commented Jul 2, 2018

This was an invalid resolution to the situation with http-client: you have broken the most recent version of the package, which people are using, and without a new version to replace it. I do not agree to these kinds of changes to my packages being made, especially without even the courtesy of an explanation being made in this discussion as to what changes you made to my package.

I'll clean this one up myself by uploading a new version of http-client. But the fact that the only version of a package of mine adding features people are using was deprecated via a revision without any explanation that that was your plan is pretty unsettling.

@gbaz
Copy link
Contributor

gbaz commented Jul 2, 2018

I think this was just a miscommunication. Since you had uploaded a new yaml and attempted to make the prior one unbuildable, I think it was expected the same process would be taken with http-client. I agree that for clarity's sake, it would have been better to handle yaml, and then leave it to you to handle http-client following the same template.

@snoyberg
Copy link

snoyberg commented Jul 2, 2018

I was quite explicit about my intentions and my actions:

I've published a new version of yaml, and added the only revision I could that should fix this: totally breaking yaml-0.8.31 (http://hackage.haskell.org/package/yaml-0.8.31/revisions/). Can you both give it a try and let me know if it works? If so, I'll release http-client as well.

The issue here was that a Hackage Trustee—@hvr—felt it appropriate to tell me to stop maintaining my own packages and performed surgery on them without any explanation of the process being taken. This is precisely why the revision mechanism is so disliked by so many. This is an upsetting combination of unacceptable communication style and the disregard for breakage caused to users by actions. I highly recommend the Hackage Trustee team and haskell.org committee address these concerns, which have been regularly raised and continue to be a major issue.

@gbaz
Copy link
Contributor

gbaz commented Jul 2, 2018

Yes you were explicit, which is why this was a miscommunication and misunderstanding. I am glad we had this chance to resolve it to everyone's satisfaction.

@hvr
Copy link
Contributor

hvr commented Jul 2, 2018

@snoyberg

This was an invalid resolution to the situation with http-client: you have broken the most recent version of the package

I have not; please get your facts straight before jumping to false conclusions; next time, please ask if there's something you don't understand

@gbaz
Copy link
Contributor

gbaz commented Jul 2, 2018

To expand (and here it seems like more misunderstanding and miscommunication): I don't think the revision made broke the then-most-recent version (0.5.13). Looking at http://hackage.haskell.org/package/http-client-0.5.13/http-client.cabal it just seems like it swapped out the not-working buildable thing in the build-depends for the correctly-working base < 0 thing, which should not have broken the package for people.

@ElvishJerricco
Copy link

@snoyberg Can you elaborate on how the package was broken? @hvr's revision built fine for me with both cabal and stack.

@snoyberg
Copy link

snoyberg commented Jul 2, 2018

It looks like I misread the edit, that's my mistake. All of my other statements stand: this was handled in an unacceptable way, and needs to be addressed by improved processes. Gershom's comment "to everyone's satisfaction" is certainly not true, there is no satisfaction on my side from the behavior of the official haskell.org representatives and Hackage/Cabal maintainers present here. Unfortunately the only people to appeal to appear to be the people misbehaving.

@snoyberg
Copy link

snoyberg commented Jul 2, 2018

Also, for the record, the reason I got this wrong is that the revision display on Hackage strongly implies that base < 0 was added regardless of flags. I now know when investigating actions taken by Herbert:

  • I cannot hope to have him indicate what his planned actions are, even when discussing one of my own packages
  • I cannot trust the information displayed on the Hackage page, but instead need to look at the raw cabal file

screen shot 2018-07-02 at 10 39 12

@ElvishJerricco
Copy link

Considering this issue is resolved, I move that the discussion be taken elsewhere.

@snoyberg
Copy link

snoyberg commented Jul 2, 2018

I've opened up an issue for the incorrect revisions display: haskell/hackage-server#770.

@hvr
Copy link
Contributor

hvr commented Jul 2, 2018

@snoyberg

there is no satisfaction on my side from the behavior of the official haskell.org representatives and Hackage/Cabal maintainers present here. Unfortunately the only people to appeal to appear to be the people misbehaving.

I'm sorry that you feel that way but as history has shown it's difficult to meet your demands and when things aren't done 100% as requested you seem to resort to personal attacks and ad-hominems (like e.g. two years ago just as one of several examples). I stayed up late last night to fix http-client and yaml and all I get in return is more of the same. I for one refuse to interact under these operating conditions. Have a nice day.

@domenkozar
Copy link

I don't want to heat up this discussion more than needed, but unsupported-ghc-version is not ideal for cabal2nix since it tries to eagerly find out dependencies and it ends up in the inputs.

@snoyberg would using base < 0 instead of non-existing package still satisfy your requirements?

@ElvishJerricco
Copy link

@domenkozar Yeesh, didn't think of that. Definitely not ideal. As a workaround, can we add unsupported-ghc-version = null; to one of the xyz-configuration.nix files?

@snoyberg
Copy link

snoyberg commented Jul 9, 2018

@domenkozar I'm willing to change the cabal file, but I think you're going to have bigger problems than this. The "include a package name as a dependency and the package doesn't exist" approach was cribbed from base:

-- sanity-check to ensure exactly one flag is set
if !((flag(integer-gmp) && !flag(integer-simple)) || (!flag(integer-gmp) && flag(integer-simple)))
    build-depends: invalid-cabal-flag-settings<0

I'd rather move this discussion away from this issue though. If you want to follow up, please do so on the yaml repo instead.

netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this issue Jan 11, 2020
0.11.2.0
* Reduces some of the code duplication between the encode and
  encodePretty functions
* The output of encodePretty has been improved:
  * Multiline strings now use Literal style instead of SingleQuoted
  * Special keys are now quoted in mappings #179
* Support for complex keys in mappings: #182
  * Adds complexMapping function to Data.Yaml.Builder
  * Decode functions now return a NonStringKey error when attempting
    to decode a mapping with a complex key as it is not possible to
    decode these to an Aeson Value
* Adds missing ToYaml instances

0.11.1.2
* Compiles with GHC 8.8.1 (MonadFail split)

0.11.1.1
* Use the appropriate Scientific rendering function to avoid a memory
  overflow when rendering. The previously used function from aeson
  would not use scientific notation, and could use large amounts of
  memory for values such as 1e9999999999999.

0.11.1.0
* Better error messages in the Data.Yaml.Config module #168
* Add LoadSettingsException exception and remove error printing from
  loadYamlSettings #172

0.11.0.0
* Split out the libyaml and Text.Libyaml code into its own
  package. #145

0.10.4.0
* Add decodeMarked and decodeFileMarked functions to Text.Libyaml, and
  extend native bindings to extract mark information. #157

0.10.3.0
* Add support for anchors and aliases to Data.Yaml.Builder #155
* Fix test suite for 32 bit machines #158

0.10.2.0
* Add EncodeOptions and FormatOptions to control the style of the
  encoded YAML. #153
* Default to using literal style for multiline strings #152

0.10.1.1
* Correctly declare libyaml dependency on system-libyaml flag #151

0.10.1
* Avoid incurring a semigroups dependency on recent GHCs.
* Fix a space leak that was introduced with 0.10.0 #147

0.10.0
* Add decodeFileWithWarnings which returns warnings for duplicate
  fields

0.9.0
* Expose style and tags on mappings and sequences in Text.Libyaml #141

0.8.32

* Escape keys as necessary #137
* Support hexadecimal and octal number values #135
* More resilient isNumeric (should reduce cases of unneeded quoting)
* hpackify
* src subdir

0.8.31.1
* Add a workaround for a cabal bug haskell-infra/hackage-trustees#165

0.8.31
* Add decodeThrow and decodeFileThrow convenience functions.
* Upgrade libyaml versions
* Deprecate decode and decodeEither
netbsd-srcmastr pushed a commit to NetBSD/pkgsrc that referenced this issue Jan 14, 2020
0.11.2.0
* Reduces some of the code duplication between the encode and
  encodePretty functions
* The output of encodePretty has been improved:
  * Multiline strings now use Literal style instead of SingleQuoted
  * Special keys are now quoted in mappings #179
* Support for complex keys in mappings: #182
  * Adds complexMapping function to Data.Yaml.Builder
  * Decode functions now return a NonStringKey error when attempting
    to decode a mapping with a complex key as it is not possible to
    decode these to an Aeson Value
* Adds missing ToYaml instances

0.11.1.2
* Compiles with GHC 8.8.1 (MonadFail split)

0.11.1.1
* Use the appropriate Scientific rendering function to avoid a memory
  overflow when rendering. The previously used function from aeson
  would not use scientific notation, and could use large amounts of
  memory for values such as 1e9999999999999.

0.11.1.0
* Better error messages in the Data.Yaml.Config module #168
* Add LoadSettingsException exception and remove error printing from
  loadYamlSettings #172

0.11.0.0
* Split out the libyaml and Text.Libyaml code into its own
  package. #145

0.10.4.0
* Add decodeMarked and decodeFileMarked functions to Text.Libyaml, and
  extend native bindings to extract mark information. #157

0.10.3.0
* Add support for anchors and aliases to Data.Yaml.Builder #155
* Fix test suite for 32 bit machines #158

0.10.2.0
* Add EncodeOptions and FormatOptions to control the style of the
  encoded YAML. #153
* Default to using literal style for multiline strings #152

0.10.1.1
* Correctly declare libyaml dependency on system-libyaml flag #151

0.10.1
* Avoid incurring a semigroups dependency on recent GHCs.
* Fix a space leak that was introduced with 0.10.0 #147

0.10.0
* Add decodeFileWithWarnings which returns warnings for duplicate
  fields

0.9.0
* Expose style and tags on mappings and sequences in Text.Libyaml #141

0.8.32

* Escape keys as necessary #137
* Support hexadecimal and octal number values #135
* More resilient isNumeric (should reduce cases of unneeded quoting)
* hpackify
* src subdir

0.8.31.1
* Add a workaround for a cabal bug haskell-infra/hackage-trustees#165

0.8.31
* Add decodeThrow and decodeFileThrow convenience functions.
* Upgrade libyaml versions
* Deprecate decode and decodeEither
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

8 participants