Content Decisions for the new Cabal User Guide

[emilypi]

This repository intends to create a new Quickstart and User guide for the Cabal build tool. The Cabal team acknowledges that the current state of documents, located at cabal.readthedocs.io, serves more as a reference, and less as a guide for getting started or gathering intuition about how to effectively use the tool on projects. We would like to change this.

Tools

Rust has had a very successful, community-driven book built on mdbook, which allows users to contribute markdown to a codebase which may be compiled into a nice, legible and locally hostable book format. I propose we adopt this style, and contribute the missing content for this book in that format.

Content

There are two major areas of content that I'd like to cover:

1. Quickstart. How does a user come up to speed in as quickly a manner as possible, such that they can get hacking with minimal fuss. The sample out line for this content would look something like the following:

• Installation
• Creating a project from scratch
• Working with an existing project, considering existing tooling like stack, or otherwise.
• Hello world! Building and running a small project to output. This section would probably list a small subset of useful commands.

2. User guide. Given a sample project, with a contrived object, how would the user go about:

• Creating the project from scratch. Discussing naming conventions, the anatomy of a simple .cabal and its components/vocabulary/syntax at a high level
• Modifying a project: adding modules exposed or otherwise, adding dependencies, warnings, etc etc. at a high level, and discussing good practice for naming conventions and managing bounds.
• Building a project: this section could discuss the basic commands used to build a project once a selection of modules has been described, and mention cabal.project and local cabal.project configuration commands, syntax, and multiproject management. Here, we could add some content regarding remote source repositories like git.
• Testing a project
• Benchmarking a project
• Profiling a project
• Working with extra-build-tools and the more exotic parts of the .cabal ecosystem
• How to package a project and work with Hackage to release

Goals

The goals of this project, in my opinion, should achieve the following:

1. We want a quick reference to cite to beginners for getting up and running with a project that prints "Hello, World!" in as few commands as possible.
2. We want a user-guide that can serve as a hand-holding exercise for beginners and more casual haskellers, who can follow along with a relatively simple, contrived project that slowly builds up some complexity into something publishable by the end.
3. We want to delineate best practices for maintaining a cabal file, so that there is less confusion
4. We want to show people how to publish and release.
5. We would like people to walk away with a general understanding of how to do the most fundamental aspects of project maintenance, ranging from module additions, to testing, benchmarking, and profiling, as well as an understanding of how to work in general with dependencies, bounds, and remote source repos.
6. We should aim to be as friendly and high level as possible, while also achieving these goals.

I'm interested to hear from @JonathanLorimer, @Mikolaj, @fgaz, @gbaz, and anyone else who has time to hash out an outline for this book. I'd love to have a public record here of our discussions for posterity.

[JonathanLorimer]

Here are my initial thoughts:

• Should the current user guide ( cabal.readthedocs.io) be edited such that it is exclusively a reference material? I think this would be useful.
• I really like mdbook, so I am onboard with that
• I think the content section is a great guide for first efforts. @soupi provided some good content for inspiration Kowainik Blog and School of FP Blog

I would love to have an involved role in this, and will spend some time over the weekend putting together my thoughts and suggestions.

[soupi]

I'm very happy to see this initiative!

Do you think setting up CI for a cabal project is something that should be covered in this guide as well?

Another thing I'd be really happy to see is information on how to compile a Haskell project that can be deployed/run on other computers, maybe using static compilation and musl, ghc-musl, AppImage, or something else.

Here are a couple more articles that could be use as inspiration (they use stack though):

• Structuring your first Haskell project with Stack
• Practical Haskell programs from scratch - a quick and easy guide - how to use stack without a package format, maybe that could be done with cabal as well? Worth considering adding such content to the guide

[emilypi]

@JonathanLorimer To your first point, absolutely. I think the more we can make a distinction between what documentation is useful for the better. The fact that the purpose of the docs on readthedocs is ambiguous is a big part of the problem. When we pipe new users to it, it's functionally useless for autodidactism! Re: Kowainik and School of FP, great finds. I'm happy to lean on them for inspiration.

@soupi Absolutely. CI is fundamental to building a project and effectively testing it as it scales. Regarding static builds, I leave that to folks who have more experience with it.

[soupi]

Let me elaborate why I think static builds is an important addition: a common problem that I have is that I run ubuntu 20.04 and I want my Haskell program to run on a remote linux box that has an older version of ubuntu. This linux box is often too weak to be able to successfully compile my programs, so I need to compile it on my computer and run the executable on the remote computer. This often fails because of glibc version mismatch. This issue will also pop-up when trying to provide an executable project for download of course.

Over the years I've learned tricks to deal with it using static compilation. First with static-haskell-nix, and later with ghc-musl and even later with AppImage and hsinstall. But there may be other options I'm not aware of.

I don't know if this is something that many people run into, but I ran into it fairly early and I have a feeling there are more people that have run into it. So I would be really happy if this tutorial could address this issue, whether using static builds or not.

[emilypi]

@soupi I'm happy for you to write a blurb about it, but consider scope: if we have to introduce several new tools to work it in, it may be out of scope for this being a Cabal user guide, whereas it may be more appropriate for future Haskell Book endeavors that will come down the line. I leave it up to you and other people more familiar with it to hash it out. I will not block the contributions if people write it, and it's justified! 😄

[AlistairB]

I think a small note on stack vs cabal would be good somewhere in the guide. As this is a very common question from beginners. Ie. something like:

• Stack is an alternative build tool. Both stack and cabal work great!
• Main difference is stack uses stackage snapshots.
• Stack depends on cabal under the hood, so learning cabal is always useful.
• Stack is a bit more user friendly, but cabal is catching up!
• Both tools will work fine, whatever you choose.

[gbaz]

Disagree. This is a cabal user guide. If someone is reading it, that would be because they've already made a choice. That material is suitable elsewhere, but this guide is explicitly for a particular tool and it is important to keep its focus narrow and laser-like to be effective.

[JonathanLorimer]

I know the Haskell School initiative by the Haskell Foundation has a blog section, separate to their "lessons" where they are going to go into greater depth on particular subjects. If mdbook supports it, I think a blog post about cabal vs stack and static builds would be both considerate and helpful! But I agree that we should elide excessive content, so as not to overwhelm those just looking to "get started" with cabal.

[AlistairB]

I would argue that understanding alternatives to cabal is good for cabal users. It's fine if the comparison is a bit biased towards cabal. As an example I found req comparing itself to alternatives very useful when selecting a http request library (this comparison has been removed in master though for some reason).

More generally I think it is good for Haskell users. Perhaps this guide isn't the place? But I think it should be documented somewhere as it is key information that users currently need to get explained (in a biased way) in various Haskell forums. The standard answer to the question for beginners is 'just use stack'. A more nuanced comparison to link to would probably help cabal get users IMO.

[emilypi]

@AlistairB this could be an interesting addendum, and I'm imagining a table of comparisons between things like Stack, Bazel, Hadrian, and so on. However, I do think this user guide should focus on the tool in and of itself before considering its pros and cons with other tools. It does impose a burden, wherein we have to hope that at least a few contributors are familiar with both tools in order to maintain the documentation if Stack ever changes our comparison table, and I would hope that we could come up with something Cabal-centric first, before imposing that burden.

[Mikolaj]

Great ideas. I think it's fine to have an optional fluff introduction that discusses the context of cabal, such as GHC, ghcup and other ways to get cabal, Hackage, haddock, stack/LTS, CI, portability/static compilation, etc. and links to separate articles or appendixes or outside references about them. Just make sure it's in italics and otherwise obviously skippable for a person that googles for how-to-do-x and that the most common workflow howtos are at the start, perhaps in TL;DR version, again linking to a section describing the same in detail. Ideally the context fluff and the TL;DR versions would be immutable as tools change (e.g., the CI landscape changes often), while the in-detail articles would play the catch-up game.

One more note to put in the fluff: the text tries to be up to date wrt newest published cabal version (if you agree that's the right policy; let's link to download for that version) and doesn't branch out into how it was done previously (with a possible exception of "since 3.4" or "deprecated" or similar annotations, but I think even those fit more a command reference, not a user guide).

Lastly, I'm personally a fan of copy-paste programming and especially copy-paste learning, so I would not avoid front-loading the reader with a large .cabal file (to be copy-pasted from the guide, or even better, generated easily from cabal init) and then, slowly, as needed, guiding the user through modifying the file, explaining bits and pieces on the way. Just comfort the user that it's fine not to understand everything from the start. This is in contrast to a long series of larger and larger toy .cabal files, or fragments thereof. IMHO it's easier to modify a section of the file than to create a new section (panic! where? at the start? in the middle of another section? use colon? comma? is whitespace meaningful?).

[JonathanLorimer]

I took a first run at coming up with a content structure.This was informed by @emilypi 's original post as well as the existing cabal user guide, kowainik's blog post, and school of fp's blog post. I created a summary of the topics covered by all three here.

With regards to emily's original post, I figure the New To Cabal section will function as the quick start guide, and the Leveling Up section will be the user guide, and Getting Fancy can dive into more advanced and less common use cases. I tried to come up with fun names, that are less charged than Beginner Intermediate and Advanced. I also feel like the names I chose work semantically, I consider myself not-a-beginner but I could see myself using the New to Cabal section if I haven't bootstrapped a project in a while. All that said, I am not attached to these names and they could just as well be labelled Quick Start and User Guide.

I think that a single example project can live in this repo, and we can use different directories to demonstrate the progression of the project as the cabal setup is expanded and improved. I was thinking that these different stages could correspond to different sections of the guide (i.e. a directory called 0_Init that just has a minimal cabal setup, then 1_BasicPackage that has basic package properties, then 2_Library that includes a library and some source code, etc etc)

Content

Preamble

• Explanation of different pieces of cabal (cabal-install, Cabal, and .cabal files)
• What is a Module and a Package?
• Hackage (package ids, versioning, and global namespace)

New To Cabal

• Installing Cabal + GHC
• Initializing a Cabal repo
  • Manually
  • cabal init
  • cabal init --interactive
• Basic Package Properties
• First Cabal Library
• First Cabal Executable
• Cabal Repl
• Combining Library and Executable
• Adding Dependencies
  • Hackage
  • Tarball
  • Local
• Running a Project

Leveling Up

• Refactoring and Re-use
  • Common Stanzas
• First Cabal Test
• First Cabal Benchmark
• Build Products and Caching
• Generating Documentation with Haddock
• Uploading Package to Hackage

Getting Fancy

• Setting Up a Project
• Flags and Conditionals
• Foreign Libraries
• Accessing Data Files
• Hpack
• Custom Setup
• Backpack
• Nix Interop

Assumptions

• cabal v3.4
• no command prefixes
• no discussion of cabal sandbox

Ideas For Appendices

• Stack vs Cabal
• Static Builds

[yaxu]

For me this starts in the wrong place. The outlines look like a cabal package developer guide, rather than a cabal user guide. Before I want to initialise and develop a new package, I want to install an existing one.

For me programming and using a library are not separate activities. I spend a lot of time working in ghci, as an 'end-user programmer'. To do this I want to install some libraries and use them together, to explore some idea. That said, some of those libraries I might be developing, and will want to re-install many times in the process of doing that.

I can do cabal install --lib to install or upgrade a library in this case, but I find that if I take this approach, quite soon everything breaks. From my perspective things break because cabal is buggy, but others tell me that it's because I'm not supposed to use cabal in this way. I wonder if there is a philosophical difference here..

My big worry is that while the v1- commands work well for me, the v2- commands in general are not designed for my approach to using haskell. I'd be very happy if I could finally work out how to use the v2- commands to do what I want, though.

addendum
Lets have a look at the webpage for pip: https://pip.pypa.io/en/stable/

I click on 'quickstart' and get instructions for how to install and use pip to install some libraries. Once I do that I can jump into ipython and start exploring them. Great!

I'd find it a bit distressing if step one was "create a python project from scratch". Even if I click on "Python packaging user guide", the first tutorial is how to install packages, not how to make them.

[gbaz]

@yaxu I wish we could write a good users guide for the workflow you describe. But as you well know, until some additional work is done on cabal proper, that v2 workflow will not be nearly as seamless as you want!

I think we should try to provide two quickstarts though, one with the package/application oriented workflow, and one with an exploratory/hacking workflow. The latter can explain making use of cabal repl --build-depends for experimenting, as well as the use of cabal.project workflows for exploratory coding in a defined environment rather than app/lib development. When cabal env functionality is folded into cabal-install (which I understand is now on the table) we can update that section to describe the newer/cleaner workflows now possible.

[rd--]

Hello

I'm in the same position as @yaxu, i.e. my work is almost all done in the interpreter (ghci).

And I have the exact same problem, I can't work out how to get v2- to install a set of loosely inter-connected libraries and have them appear in ghci (and ghc --make).

So far I've just kept using v1- but I know at some point that won't work anymore.

It'd be super helpful to have a transition guide for this kind of work.

For instance, at the moment the way I update an existing library is to run "cabal v1-build && cabal v1-copy && cabal v1-register"

v2- doesn't seem to have either copy or register? It's not clear what to do instead.

Best,
Rohan

Ps. I recently tried to setup a system using v1- with ghc-8.10.5 and cabal-3.4.0 and it failed, with the error mentioned at:

haskell/cabal#7244

I'm a bit worried v1- will stop working before I can work out how to use v2- and I'll be stuck.

[JonathanLorimer]

@rd-- it seems like a Cabal User Guide that focuses on the v2 commands (just regular commands in cabal-3.4.0) would be beneficial for you? Correct me if I am wrong. I think a migration guide from v1 to v2 is more of a reference thing, but might make a nice appendix entry, I will add it to the list.

@yaxu I am very interested in your pedagogical philosophy. I am definitely biased here, I learn much better when I can start from first principles and understand something end to end. However, I definitely respect the exploratory approach; get up to speed quickly and start playing around. I have a big ask of you yaxu, could you take the content template I have laid out above and re-arrange it / add and subtract things so that it fits your ideal curriculum? I think that would be the most productive way forward. I think that there might be something clever we can do with mdbook to take the same chapters and produce 2 books laid out in different orders.

I know you say it is distressing to be confronted immediately with "starting a project from scratch", but you must admit that some people are looking for this kind of instruction. Also, in my opinion, installing a dependency is usually done relative to a project that is in need of a library, so I think that my original content template is at the very least defensible.

The key takeaway I am getting from this (yaxu, gbaz, and rd--'s commentary) is that, perhaps, the Cabal Repl section should appear right after Installing Cabal + GHC

[rd--]

Hello Jonathon

Thanks, perhaps, although the transition guide I was thinking of was quite specifically how to do what I think @gbaz calls above "exploratory coding in a defined environment", where the environment in the one that you get when you run ghci.

Does this make sense? It seem that perhaps at the moment this is not possible though?

I'm not sure if cabal repl is the right approach, the documentation says it loads everything as interpreted byte-code? Whereas ghci currently loads compiled code for installed libraries?

Best,
Rohan

[Mikolaj]

@JonathanLorimer: I'm afraid you are suddenly burdened with

1. deciding whether the "exploratory programming in a defined environment" use case is important enough to be appendix matter or main guide matter
2. whether to wait with your work until the external tool cabal-env is included in cabal (#6481, GHC fixes, etc.) or document v1 commands instead or document the external tool in its provisional state or leave a TODO
3. how cabal repl and ghci differ and whether the former should be tweaked

These are the most difficult kind of questions for cabal maintainers, because they involve prioritizing some users at the expense of others, so there are no good answers, only bad and worse, according to one of many possible metrics. Also, no matter how brilliant the prioritizing, lots of raw effort, timely user feedback, luck with upstream fixes, patience and good will on all sides, most of all the side that is losing due to the decisions, is needed to move forward.

FYI, the status quo we have inherited and we work with seems to be

1. "exploratory programming in a defined environment" is currently appendix matter (otherwise v1 commands would not be marked as deprecated before a replacement was ready)
2. accordingly, the current and future ways to handle it are best discussed in their tickets, with a TODO (and perhaps link to tickets) in the user guide, until the new ways are ready and accepted and fine-tuned by users
3. ???

@emilypi, @fgaz, other esteemed contributors, please correct me if I'm off the mark. Dear users of the "exploratory programming in a defined environment" workflow, thank you for making your voice heard and we also need your contributions in the respective tickets (mostly #6481, I guess).

[yaxu]

The key takeaway I am getting from this (yaxu, gbaz, and rd--'s commentary) is that, perhaps, the Cabal Repl section should appear right after Installing Cabal + GHC

Yes definitely. I think probably all package management software apart from cabal present things in this way. The reason for this is that almost all users will want to install a library before they make their own package.

If you disagree, please show me a Haskell textbook that puts 'how to create a haskell package' ahead of 'how to write haskell code'.

For example real world haskell first tells you how to install ghc and run ghci:
http://book.realworldhaskell.org/read/

There is something about cabal in an appendix, but only how to install something.

I feel like I'm being treated as an outsider when this is a totally standard way of learning and working, which is seemingly not supported by cabal any more.

I find this baffling, but in case my tone is misread - I'm not angered by this, I'm fascinated. I'd really like to get to the bottom of this mismatch.

@rd-- it seems like a Cabal User Guide that focuses on the v2 commands (just regular commands in cabal-3.4.0) would be beneficial for you? Correct me if I am wrong. I think a migration guide from v1 to v2 is more of a reference thing, but might make a nice appendix entry, I will add it to the list.

From what others write in this thread it seems that such a migration guide is not possible as the v2- commands are missing key features. At the same time v1- commands are unsupported and are undergoing bitrot.

@yaxu I am very interested in your pedagogical philosophy. I am definitely biased here, I learn much better when I can start from first principles and understand something end to end.

Me too! But I don't think learning the intricacies of packaging is part of the first principles of programming.

However, I definitely respect the exploratory approach; get up to speed quickly and start playing around. I have a big ask of you yaxu, could you take the content template I have laid out above and re-arrange it / add and subtract things so that it fits your ideal curriculum? I think that would be the most productive way forward. I think that there might be something clever we can do with mdbook to take the same chapters and produce 2 books laid out in different orders.

I think the above outlines are fine as an introduction to making cabal packages. I just think there needs to be a separate document presented for new users that is about using them.

I know you say it is distressing to be confronted immediately with "starting a project from scratch", but you must admit that some people are looking for this kind of instruction.

Yes, eventually, if they want to package up some code they've made. But I think they will want to install an existing library firs.t

Also, in my opinion, installing a dependency is usually done relative to a project that is in need of a library, so I think that my original content template is at the very least defensible.

Yes if you are already working on a project such content would be useful.

[yaxu]

I do by the way find this issue really serious. Newcomers to Haskell will pick up a copy of books like 'real world haskell' and find that the approach to programming that is introduced is not supported any more. I think haskell will be losing a lot of users over this, maybe over half? The profound difficulty in installing and using the tidal library is certainly losing end users in that kind of number.

[yaxu]

Some more examples..

Basics quickly cover how to install and use ruby gems. Step two is how to create your own package:
https://guides.rubygems.org/rubygems-basics/

However with stack, after a couple of minutes I still didn't find any information about how to install a library from stackage:
https://docs.haskellstack.org/en/stable/README/

Like with cabal, the 'quick start' documentation goes straight into how to create a new 'project'. So despite the cabal and stack folks behaving like different religions, it seems there is something about the recent Haskell mindset that runs against wanting to install a library outside of a particular project or package? I wonder where this comes from, and why it wasn't there when the v1- commands were designed. People keep mentioning this 'nix-like' thing, I'm not sure what that is, but maybe it's the source of this mindset?

[Mikolaj]

What you say, @yaxu, suggests the following questions to me:

To what extent Cabal User Guide should become or contain or link to Haskell User Guide or Haskell Interactive Environment User Guide?

What are the places readers are likely to be redirected to Cabal User Guide from and how should we cater to each class? E.g., one class I didn't think of is the people that googled cabal after trying to use cabal as instructed in old Haskell books and failing. There are, of course, many more obvious classes, very different from the dog-fooding class (cabal developers themselves, looking up a friendly workflow they never used before).

[yaxu]

Hi @Mikolaj,

I guess the point of your question is to passively imply that people who want to use cabal to install something aren't using it in some sense, but I'm not sure. Could you explain yourself a bit more please?

Jumping back in time a bit: https://cabal.readthedocs.io/en/2.4/

It starts with quite a friendly explanation of what cabal is, what it compares with (eggs, gems, cpan), and how to install packages with it including from hackage.

What was wrong with this? As far as I can see, starting with anything more advanced than that would alienate all new users, and introductory guides normally begin by considering with new users, rather than denying that they exist.

Again, I maintain a cabal package myself, so am also part of the dog-fooding class I suppose.

[JonathanLorimer]

Also one thing that hadn't occurred to me are the different definitions of "using" software here. When I think of using a package, I think of using a library and calling it from my code (my code is generally a repl, a project, or a cabal package itself). I did not think of the case where people are using cabal like npm, gem, or pip, I think the main difference here is that those are all interpreted languages, and not as many people want to compile from source. Perhaps the pre-compiled binary (pandoc and xmonad are distributed this way) has made me ignorant to this kind of workflow.

I also think that the questions highlighted by @Mikolaj are good ones.

Perhaps these action items are satisfactory?

• Move Cabal Repl section earlier (as mentioned here)
• Add an appendix entry for transitioning from v1 to v2+
• Discuss the differences between ghci and cabal repl in the Cabal Repl section
• Add a final section to the Preamble about using cabal to install and compile software for immediate use
• Have a very clear explanation that the guide deals with cabal 3.4.0+ for any readers arriving at the guide from old materials

[yaxu]

Also one thing that hadn't occurred to me are the different definitions of "using" software here. When I think of using a package, I think of using a library and calling it from my code (my code is generally a repl, a project, or a cabal package itself). I did not think of the case where people are using cabal like npm, gem, or pip, I think the main difference here is that those are all interpreted languages, and not as many people want to compile from source. Perhaps the pre-compiled binary (pandoc and xmonad are distributed this way) has made me ignorant to this kind of workflow.

Aha yes that makes sense! Of course Haskell is both interpreted and compiled. So could it be that between v2 and v3 of cabal people using it as an interpreted language somehow got totally forgotten? Despite that generally being how new users learn to program, and how many use it daily as part of an interactive workflow.

[Mikolaj]

...
install a library outside of a particular project or package
...
What was wrong with this?

Cabal hell was wrong with this. Daily reports from desperate users that wipe out their ~/.cabal [edit: and ~/.ghc] repeatedly and still can't compile, with different error each time, due to subtly transitively conflicting dependencies. Nix, stack, some OSes, most distributions of commercial software deal with it by either curating a fixed set of libraries or bundling all dependency libraries with the executable in hand and trying hard to avoid seeing any library installed outside a sandbox or project lest it introduces conflicts.

We naively thought that Nix style is the way to go, including repl use, because the topelevel global namespace is just one more project, right? Years later we'll have it implemented RSN (#6481 and GHC fixes).

[JonathanLorimer]

@yaxu

Yes definitely. I think probably all package management software apart from cabal present things in this way. The reason for this is that almost all users will want to install a library before they make their own package.

If you disagree, please show me a Haskell textbook that puts 'how to create a haskell package' ahead of 'how to write haskell code'.

I think the discrepancy here is that one must use cabal to use other developers' libraries, so I think there is an aspect of cabal that is commensurate with "how to write haskell code", but is still short of "how to create a haskell package". In this case, users still need to know how to make cabal aware of their code (as a library or executable) so that cabal can provide the external code and the user can compile everything. Even though compilation is generally seen as a "packaging" step, it is also a development aid in haskell.

I hope that explanation demonstrates that we really aren't trying to throw beginners off the deep end. From my perspective, and the beginners that I have helped teach haskell, starting off with a project is a totally standard way to get up and running. I also think this is pretty standard in the world of javascript, ruby and python. Many javascript guides begin with npm init (which scaffolds a js project / package), even before using npm install.

I feel like I'm being treated as an outsider when this is a totally standard way of learning and working, which is seemingly not supported by cabal any more.

I am sorry to hear that you feel this way. FWIW different opinions are really important for these kind of guides and your input has made me aware of a workflow I hadn't even thought of (I am a relatively new cabal user btw). I also hope you don't feel like we are ganging up on you. I think we are just a) eager to discuss this topic and b) keen to find some kind of consensus so development on this user guide can begin!

[yaxu]

@JonathanLorimer Well something that a lot of beginners/learners (and we are all learners) share with a lot of professional end-user programmers is that the outcome they want isn't a program as such. A programming language can be a user interface to interacting with the world rather than a system for building an application for later use. Tidal is used to make live music as per the videos here https://tidalcycles.org/ . Scientists often use REPLs in a similar way to explore a dataset using statistics and visualisation.

[gbaz]

I think @rd-- 's use case is fully solved by cabal repl --build-depends=. Regarding the concern expressed:

I'm not sure if cabal repl is the right approach, the documentation says it loads everything as interpreted byte-code? Whereas ghci currently loads compiled code for installed libraries?

This seems like a flaw in the documentation. The library dependencies will be compiled code. Only the current project (which is null in the standalone case) is loaded interpreted.

So simply moving cabal repl earlier and having a distinguished quick-start section for exploratory programming will suffice. I've found many people who are comfortable with new-build on irc still are surprised by/unaware of the utility of the repl workflow, and appreciate learning it, so I think it certainly has been undersold in the past.

The problem is that @yaxu 's workflow is more complicated, since that involves experimenting with a combination of local and hackage packages, i.e. sort of weaving together a combination environment, and also experimenting with updating local portions of that environment incrementally. This is precisely the sort of thing the cabal-env tool is supposed to help with. For those that don't know, yaxu works on https://hackage.haskell.org/package/tidal and so the end goal is often not a library or compiled executable, but instead a completely live-coded interactive musical performance. I encourage you to check out videos of these performances -- they're extremely cool.

That said, @yaxu, I'm still not sure why the repl --build-depends approach is not suitable for your end users if not your own development process. I know you've felt frustrated on this for some time, but feel free to ping me on irc and maybe if you want to bloc out an hour of time we could do a call and try to work through some stuff together. I really enjoy your music and appreciate your projects, and want to help you thrive.

[yaxu]

Thanks @gbaz, looking at cabal repl --build-depends=tidal, it looks like that could be a nice command for Tidal editor plugins to run instead of ghci, aside from being a bit slower to start with a couple of confusing messages about fake packages - it would be nice if those could be suppressed. Anyway I've probably spammed this issue enough and will take up your kind offer if I have any troubles with this, thanks!

[JonathanLorimer]

This is my revised content outline, given the discussion above

Content

Preamble

• Explanation of different pieces of cabal (cabal-install, Cabal, cabal.project, and .cabal files)
• What is a Module and a Package?
• Hackage (package ids, versioning, and global namespace)
• How to install haskell software from hackage

New To Cabal

• Installing Cabal + GHC
• Cabal Repl
  • Be sure to highlight --build-depends
• Initializing a Cabal repo
  • Manually
  • cabal init
  • cabal init --interactive
• Basic Package Properties
• First Cabal Library
• First Cabal Executable
• Combining Library and Executable
• Adding Dependencies
  • Hackage
  • Tarball
  • Local
• Running a Project

Leveling Up

• Refactoring and Re-use
  • Common Stanzas
• First Cabal Test
• First Cabal Benchmark
• Build Products and Caching
• Generating Documentation with Haddock
• Uploading Package to Hackage

Getting Fancy

• Setting Up a Project
• Adding Doctests
• Adding Local Hoogle
• Flags and Conditionals
• Profiling an Existing Project
• Foreign Libraries
  • Make sure to mention cpp, hsc2hs, and c2hs
• Accessing Data Files
• Hpack
• Custom Setup
• Backpack
• Nix Interop

Assumptions

• cabal v3.4
• no command prefixes
• no discussion of cabal sandbox

Ideas For Appendices

• Stack vs Cabal
• Static Builds
• Cabal in CI
• Transitioning from v1 to v2+

[rd--]

Thanks @Mikolaj and @gbaz for pointing me to cabal-env. It seems like it may be exactly the thing I was looking for! If that is, in one sense, the v2- replacement for v1-install then that might be a nice thing to add to the guide?

Thanks again,
Rohan

[gbaz]

Outline looks good. A few notes: "Explanation of different pieces of cabal (cabal-install, Cabal, and .cabal files)" -- mention cabal.project too even though you'll only get to them in "Getting Fancy". Under "Foreign Libraries" probably worth mentioning explicitly cpp and hsc2hs and c2hs and how they work with cabal (though not teaching how to use them as such). Also perhaps worth mentioning doctests in the fancy section.

Also worth teaching: "If I have a project and now want to do a profiling buiild, how do I do that".

[JonathanLorimer]

great feedback, edited to reflect those suggestions.

[fendor]

Great outline, definitely the right steps! It is already quite detailed, which I think is awesome!

I am a bit afraid of assuming version 3.4, e.g. ubuntu has at most version 3.0 (https://packages.ubuntu.com/search?suite=default&section=all&arch=any&keywords=cabal&searchon=names).
But I think it is fine since we probably recommend installing the latest version (ghcup) in the first section?

Additionally, I think it might be nice to have a guide (maybe appendix?) on how to start your own local hoogle instance for querying your project, hope that isn't out of scope, but I think it is an important part of a cabal project.

In the current Outline, I don't see where CI would be mentioned, just want to remind that this would be cool, too. Maybe even a docker section for gitlab? In that sense, deployment could be a section in the guide (expanding on static builds topic).

[JonathanLorimer]

@fendor I think CI is a rather large topic. I am not against it as an appendix, but it seems like there are lots of variables (cabal with nix, cabal with docker, github actions, travis ci, appveyor etc.). I will add it to the content outline as a stretch goal. I think its a great topic and would serve a lot of people well, but I am a bit uncertain how to structure that section.

[Mikolaj]

For me, it would be useful to even learn from the guide that various tasks can be integrated/automated/helped by cabal, with links or even without. I can google the details but the inspiration is invaluable on its own. E.g., various CIs offer ages old GHC setups and I might naively try to use these instead of searching for up to date scripts maintained by users.

[JonathanLorimer]

@Mikolaj thats very interesting. Perhaps an example that uses almost exclusively cabal for running tasks in ci would be useful. Then users with more complicated setups (thinking of nix here) can take the bits that apply!

[Mikolaj]

A good example is the old and trusty https://github.com/haskell-CI/haskell-ci for Travis with its interactive tool that generates a Travis script that uses cabal for the heavy lifting. I'm not up to date with others (AppVeyor, github, etc.) or perhaps there is (going to be) a universal one somewhere?

[rd--]

Hello again,

Looking further into this I'm still not sure cabal repl is always a solution.

One reason is that it's very slow, which is a problem if you have small haskell programs that you need to run in an interactive context.

ghci isn't terribly fast, but it's "responsive".

$ time echo "1 + 2" | ghci
GHCi, version 8.8.4: https://www.haskell.org/ghc/  :? for help
Loaded GHCi configuration from /home/rohan/.ghci
Prelude> 3
...
real    0m0.207s
...
$ time echo "1 + 2" | cabal repl
Resolving dependencies...
...
GHCi, version 8.8.4: https://www.haskell.org/ghc/  :? for help
Loaded GHCi configuration from /home/rohan/.ghci
Loaded GHCi configuration from /tmp/cabal-repl.-103415/setcwd.ghci
Prelude> 3
...
real    0m3.075s
...
$

One-fifth of a second is acceptable latency for "Select Menu Item -> Get Output" and on, whereas three seconds isn't really.

So I think a section on "How to install libraries so they can be loaded by ghci (and located by ghc --make)" would still be very useful!

Also (and I very, very much appreciate all the excellent things cabal does, it's wonderful!) I do think that for certain people in certain situations it's still nice to be able to think about Cabal as a program that's used to install Haskell libraries.

So it'd be nice to have a section in the manual that tells people how to use it that way.

Best,
Rohan

[Mikolaj]

One reason is that it's very slow, which is a problem if you have small haskell programs that you need to run in an interactive context.

Is there a cabal issue about that? I'd like to understand why cabal repl is so much slower and why people use it by invoking it repeatedly, instead of having a long running, say, GHC API process that consumes expressions.

[rd--]

I don't know if there's an issue. I did look now and couldn't see one, but there's 1414 open issues (eek!) so I might've missed it...

I don't actually use cabal repl yet, but it was suggested above as a replacement for ghci so I was looking into it.

And mostly I do work in Emacs, with a long running connection to ghci.

But not always, so it's nice to have scripts that do useful things and run fast.

It does seems like (hopefully) some form of cabal-env may be the way forwards.

But maybe this does mean, in relation to the outline @JonathanLorimer sent above, that it might be nice to have a distinct section in "New To Cabal" for this, instead of just having "Cabal Repl".

(Or perhaps making that section cover both cases and calling it something else? It's a little complicated because it's not only about the "interpreter", it's also related to whether you can use ghc --make to build a very simple program, or if you need to use cabal there as well).

Best,
Rohan

[yaxu]

@rd-- Yes the startup delay is enough of a reason to stick with the v1- commands for me.

Perhaps there should be a transition section of the user guide that covers aspects not currently supported by v2-, that are covered by v1-, and explains how utilities like ghc-pkg relate to the different versions?

[fendor]

@Mikolaj Some meta issue is here: haskell/cabal#6977

[Mikolaj]

@fendor: thank you. The issue suggests the slowness comes from filesystem usage and lays out a path to working exclusively in-memory in such situations. I bumped priority of the issue, given that we'd love to get rid of v1-. Contributions are very welcome.

Edit: however, /tmp on Linux boxes is usually a virtual file system, that is, in-memory. So I'm not 100% sure of the diagnosis. May be the fs-like bookkeeping and lots of memory allocations for the files is enough of a drag (and OS syscalls and inter-process communications at various stages).

Edit2: @fgaz explains more in the issue, in particular, caching is needed.

[Mikolaj]

@JonathanLorimer and dear users, from our brainstorming on IRC and haskell/cabal#6977, it seems, if cabal repl is too slow for a use case, cabal env is the best future bet (the prototype for now, the official thing once it's merged into cabal). Feedback would be very much appreciated (haskell/cabal#6481).

[JonathanLorimer]

I am going to add a chapter for Cabal Env that will remain incomplete until the feature is officially merged. I am happy to add a section in the Cabal Repl that addresses slowness on startup and points to ghci.

I think that the cabal user guide is not necessarily the correct place for deep treatment of cabal-alternatives as they pertain to specific workflow needs (startup speeds etc.), but rather it should focus on the kinds of workflows that cabal can offer or how cabal could fit into ones workflow. Additionally I am wary to direct users to v1 commands in this guide, since they are being deprecated.

[rd--]

@Mikolaj : I've left feedback (very positive!) for cabal-env at haskell/cabal#6481 Thanks again for pointing me to it.

@JonathanLorimer : A chapter on cabal env sounds perfect. I think so long as there is a link that people can be directed to for instructions on "How to install a library so ghc/ghci can load it" then that will be perfect!

I do think it's important that the manual not make it seem as if using cabal to install libraries for ghc/ghci is a particularly eccentric thing to want to do though, or that it's "for beginners" who will eventually move on to more serious things.

I think it's just a different way of working, c.f. https://discourse.haskell.org/t/state-of-the-cabal-q1-q2-2021/2548/36 (and https://mail.haskell.org/pipermail/haskell-cafe/2021-March/133676.html which that quotes).

[JonathanLorimer]

@rd--

I do think it's important that the manual not make it seem as if using cabal to install libraries for ghc/ghci is a particularly eccentric thing to want to do though, or that it's "for beginners" who will eventually move on to more serious things.

I think it's just a different way of working, c.f. https://discourse.haskell.org/t/state-of-the-cabal-q1-q2-2021/2548/36 (and https://mail.haskell.org/pipermail/haskell-cafe/2021-March/133676.html which that quotes).

100% agree on this.

[Mikolaj]

@rd--: I'm happy we are finding common ground.

To reiterate: the v1- workflow leads to cabal hell (a kind of DLL hell; users getting used to it and then as their project grows suddenly getting impossible to resolve conflicts regularly). This is why we strive to convert as many users as possible to v2- and (with cabal env) minimize the impossible to replace remnants of v1- workflow and be very careful not to advertise them as an alternative to v2- for compiling a project (a very common workflow) or for any other workflow that doesn't need it. Thank you for cooperating with us in this long term and, so far, quite bumpy mission. I hope it's clear your (and other similar) use case is not frowned upon in any way --- however, as long as it's prone to cabal hell, we're going to be suspicious and difficult. :)

[Mikolaj]

BTW, it seems this will be (quite late) addressed by the new guide: haskell/cabal#6104 (Documentation: v1 to v2 migration guide). I guess we should close that ticket once the guide has a preliminary release.

Edit: I've also found a whole cabal env project page here: https://github.com/haskell/cabal/projects/10 and I've opened one about replacing v1- in hackage codebase (not needed for the guide, I think, but a part of the context of removing v1-) https://github.com/haskell/cabal/projects/12

[JonathanLorimer]

I am closing this issue because of #3 , If anyone wants new content to be included, or existing content to be altered, please submit a separate issue! Thank you to everyone who provided feedback!