Build GHC with cabal-install and a Makefile #3

hsyl20 · 2024-11-27T12:10:25Z

Note that we need a patched cabal: https://github.com/hsyl20/cabal/tree/hsyl20/per-file-extra-source-options

Build ghc-stage1
Generate valid non-cross stage1 settings (reusing stage0's settings for the most part)
Build boot libraries with stage1: ghc-prim, ghc-internal, base
Build ghc-stage2
For every host/arch we want to build a compiler for:
- generate valid stage2 settings for the target (using ghc-toolchain?)
- use ghc-stage2 with these settings to build root libraries
- use ghc-stage2 with these settings to build iserv
- (optional) use ghc-stage2 with these settings to cross-build a GHC
...

hsyl20 · 2024-11-27T12:26:12Z

Current status: it builds some ghc program in _build/stage0/bin/ghc. It seems to be linked with the wrong ghc-boot because it reports a GHC version of 9.8.2 (my bootstrap GHC).

hsyl20 · 2024-11-27T12:29:42Z

Ah, I need to pass some environment variable to ghc-boot's Setup.hs

angerman

Thank you @hsyl20 this looks like a great start!

angerman · 2024-11-28T04:44:42Z

Makefile

@@ -0,0 +1,67 @@
+HADRIAN_SETTINGS_STAGE0 := $(shell ghc --info | runghc GenSettings.hs ghc-boot)


Why is this called HADRIAN_SETTINGS

That's how the environment variable is called in ghc-boot's Setup.hs ¯_(ツ)_/¯

angerman · 2024-11-28T04:45:27Z

Makefile

+	## Substituting variables
+	cp _build/stage0/src/ghc-bin/ghc-bin.cabal{.in,}
+	cp _build/stage0/src/libraries/ghc/ghc.cabal{.in,}
+	cp _build/stage0/src/libraries/ghc/GHC/CmmToLlvm/Version/Bounds.hs{.in,}
+	cp _build/stage0/src/libraries/ghc-boot/ghc-boot.cabal{.in,}
+	cp _build/stage0/src/libraries/ghc-boot-th/ghc-boot-th.cabal{.in,}
+	cp _build/stage0/src/libraries/ghc-heap/ghc-heap.cabal{.in,}
+	cp _build/stage0/src/libraries/ghci/ghci.cabal{.in,}
+	cp _build/stage0/src/utils/ghc-pkg/ghc-pkg.cabal{.in,}
+
+	sed -i 's/@ProjectVersion@/9.13/' _build/stage0/src/ghc-bin/ghc-bin.cabal
+	sed -i 's/@ProjectVersionMunged@/9.13/' _build/stage0/src/ghc-bin/ghc-bin.cabal
+	sed -i 's/@ProjectVersion@/9.13/' _build/stage0/src/libraries/ghc/ghc.cabal
+	sed -i 's/@ProjectVersionMunged@/9.13/' _build/stage0/src/libraries/ghc/ghc.cabal
+	sed -i 's/@ProjectVersion@/9.13/' _build/stage0/src/libraries/ghc-boot/ghc-boot.cabal
+	sed -i 's/@ProjectVersionMunged@/9.13/' _build/stage0/src/libraries/ghc-boot/ghc-boot.cabal
+	sed -i 's/@ProjectVersion@/9.13/' _build/stage0/src/libraries/ghc-boot-th/ghc-boot-th.cabal
+	sed -i 's/@ProjectVersionMunged@/9.13/' _build/stage0/src/libraries/ghc-boot-th/ghc-boot-th.cabal
+	sed -i 's/@Suffix@//' _build/stage0/src/libraries/ghc-boot-th/ghc-boot-th.cabal
+	sed -i 's/@SourceRoot@/./' _build/stage0/src/libraries/ghc-boot-th/ghc-boot-th.cabal
+	sed -i 's/@ProjectVersion@/9.13/' _build/stage0/src/libraries/ghc-heap/ghc-heap.cabal
+	sed -i 's/@ProjectVersionMunged@/9.13/' _build/stage0/src/libraries/ghc-heap/ghc-heap.cabal
+	sed -i 's/@ProjectVersionForLib@/9.13/' _build/stage0/src/libraries/ghc-heap/ghc-heap.cabal
+	sed -i 's/@ProjectVersion@/9.13/' _build/stage0/src/libraries/ghci/ghci.cabal
+	sed -i 's/@ProjectVersionMunged@/9.13/' _build/stage0/src/libraries/ghci/ghci.cabal
+	sed -i 's/@ProjectVersionForLib@/9.13/' _build/stage0/src/libraries/ghci/ghci.cabal
+	sed -i 's/@ProjectVersion@/9.13/'       _build/stage0/src/utils/ghc-pkg/ghc-pkg.cabal
+	sed -i 's/@ProjectVersionMunged@/9.13/' _build/stage0/src/utils/ghc-pkg/ghc-pkg.cabal
+	sed -i 's/@ProjectVersionForLib@/9.13/' _build/stage0/src/utils/ghc-pkg/ghc-pkg.cabal
+
+	sed -i 's/@LlvmMinVersion@/13/' _build/stage0/src/libraries/ghc/GHC/CmmToLlvm/Version/Bounds.hs
+	sed -i 's/@LlvmMaxVersion@/20/' _build/stage0/src/libraries/ghc/GHC/CmmToLlvm/Version/Bounds.hs


We could just use a configure script for this right?

We could. But we moved away from it upstream.

angerman · 2024-11-28T04:46:35Z

cabal.project-stage0

+  ./_build/stage0/src/ghc-bin/
+  ./_build/stage0/src/libraries/ghc
+  ./_build/stage0/src/libraries/directory/
+  ./_build/stage0/src/libraries/file-io/
+  ./_build/stage0/src/libraries/filepath/
+  ./_build/stage0/src/libraries/ghc-platform/
+  ./_build/stage0/src/libraries/ghc-boot/
+  ./_build/stage0/src/libraries/ghc-boot-th/
+  ./_build/stage0/src/libraries/ghc-heap
+  ./_build/stage0/src/libraries/ghci
+  ./_build/stage0/src/libraries/os-string/
+  ./_build/stage0/src/libraries/process/
+  ./_build/stage0/src/libraries/semaphore-compat
+  ./_build/stage0/src/libraries/time
+  ./_build/stage0/src/libraries/unix/
+  ./_build/stage0/src/libraries/Win32/
+  ./_build/stage0/src/utils/ghc-pkg
+  ./_build/stage0/src/utils/hsc2hs
+  ./_build/stage0/src/utils/unlit
+  ./_build/stage0/src/utils/genprimopcode/
+  ./_build/stage0/src/utils/deriveConstants/
+


Why do we put them in build/stage0?

I don't want to modify the source tree directly, this way it's easier to nuke the _build directory and restart from a clean state.

I think we don't need to copy all source files across. It is also annoying to do it propely with Make.

cabal will use a separate build directory anyway, only some generated cabal files will end up in the source tree.

We need to perform some substitution/file generations more than once (for the different stages). I still think it's cleaner to avoid dirtying the source directory. I don't want to have to fix bugs because we've done something for stage0 and forgot to do it again differently for stage1.

hsyl20 · 2024-11-28T13:30:50Z

Now it seems like cabal doesn't like empty package databases:

Error: [Cabal-9076]
failed to parse output of 'ghc-pkg dump'

Edit: that was my mistake (a debug statement in ghc-pkg...). Now it works.

Makefile

andreabedini · 2024-11-29T05:19:10Z

Effectively we want to completely disable the solver. We can add
active-respositories: :none to cabal.project and remove the source constraints.

hsyl20 · 2024-11-29T09:49:53Z

Effectively we want to completely disable the solver. We can add active-respositories: :none to cabal.project and remove the source constraints.

I've disabled it for stage1 where we start fresh to build boot libraries.

We need Haskell anyway for GenSettings and Makefile syntax isn't high-level enough to write quality code.

angerman reviewed Nov 28, 2024

View reviewed changes

andreabedini reviewed Nov 29, 2024

View reviewed changes

Makefile Outdated Show resolved Hide resolved

hsyl20 force-pushed the wip/make-build branch from 4f9641d to 563f2cf Compare December 11, 2024 15:35

andreabedini force-pushed the wip/make-build branch from 09d2c38 to 1df74eb Compare December 19, 2024 06:31

hsyl20 and others added 21 commits January 8, 2025 13:50

Add github workflow

10bddf1

Build GHC with cabal-install and a Makefile

9a82e3f

Generate proper settings for ghc-boot

f448cf6

Build stage0 tools: ghc-pkg, deriveConstants, genprimopcode

b2c3feb

Generate proper stage1 settings

03ef1c2

Fix installation of stage1 programs. WIP using stage1 compiler

d35f886

Fix path to stage1 package db

ba8dd50

Progress towards building the RTS

b768436

Fix CPP for Cmm

7d79ee6

Apply suggestions

41c39e3

Allow specifying custom cabal

0fe15d0

Start with deriveConstants

b45a30a

Generate DerivedConstants.h with a hack

e4a5c15

Pass -this-unit-id because cabal doesn't when it compiles Cmm files

33fc1fc

Try using per source flags

14476f9

Fix rts per file flags

d24e48f

Generate AutoApply*.cmm

1bdeaa4

Build libffi

ecf13e6

Progress building ghc-prim

f57d0f7

Build ghc-prim

ac75d26

refactor: avoid hack when checking size of struct MD5Context

d0e71b4

hsyl20 added 3 commits January 8, 2025 13:50

Build ghc-bignum

a95da12

WIP: build ghc-internal

5b1a6a7

Fix new error while bootstrapping

0bd0df7

hsyl20 force-pushed the wip/make-build branch from 0cbfef4 to 0bd0df7 Compare January 8, 2025 13:07

hsyl20 added 15 commits January 8, 2025 14:25

Fix more stuff

6c0fc8b

Build base

d635d21

Build statically linked stage0 programs.

41e2cfb

Add Haskell build script

3cd13ce

We need Haskell anyway for GenSettings and Makefile syntax isn't high-level enough to write quality code.

Build stage1 boot libraries with Build.hs

2aa08a3

Add timing to messages

676e10f

Fix messages

8caa368

Don't wire in the compiler

a8dd091

Fix options

bfbe7e6

Fix compiler path

6b7c67c

Minor refactor

58fe363

Fix Makefile to use Build.hs

dd4ffaf

Fix Setup scripts

325be24

Add Alex

af329a2

Use the Makefile in CI

f9fe095

hsyl20 force-pushed the master branch from ed26c7a to ab3ab3e Compare January 14, 2025 17:18

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Build GHC with cabal-install and a Makefile #3

Build GHC with cabal-install and a Makefile #3

hsyl20 commented Nov 27, 2024 •

edited

Loading

hsyl20 commented Nov 27, 2024

hsyl20 commented Nov 27, 2024

angerman left a comment

angerman Nov 28, 2024

hsyl20 Nov 28, 2024

angerman Nov 28, 2024

hsyl20 Nov 28, 2024

angerman Nov 28, 2024

hsyl20 Nov 28, 2024

andreabedini Nov 29, 2024

hsyl20 Nov 29, 2024

hsyl20 commented Nov 28, 2024 •

edited

Loading

andreabedini commented Nov 29, 2024

hsyl20 commented Nov 29, 2024

		@@ -0,0 +1,67 @@
		HADRIAN_SETTINGS_STAGE0 := $(shell ghc --info \| runghc GenSettings.hs ghc-boot)

Build GHC with cabal-install and a Makefile #3

Are you sure you want to change the base?

Build GHC with cabal-install and a Makefile #3

Conversation

hsyl20 commented Nov 27, 2024 • edited Loading

hsyl20 commented Nov 27, 2024

hsyl20 commented Nov 27, 2024

angerman left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

hsyl20 commented Nov 28, 2024 • edited Loading

andreabedini commented Nov 29, 2024

hsyl20 commented Nov 29, 2024

hsyl20 commented Nov 27, 2024 •

edited

Loading

hsyl20 commented Nov 28, 2024 •

edited

Loading