Skip to content

Stable haskell/ci speedup#132

Open
hasufell wants to merge 96 commits intostable-ghc-9.14from
stable-haskell/CI-speedup
Open

Stable haskell/ci speedup#132
hasufell wants to merge 96 commits intostable-ghc-9.14from
stable-haskell/CI-speedup

Conversation

@hasufell
Copy link
Member

@hasufell hasufell commented Dec 11, 2025

So far seems to have saved 7 minutes

angerman and others added 30 commits November 20, 2025 12:10
This change reverts part of !14544, which forces the bootstrap
compiler to have ghc-internal.  As such it breaks booting with
ghc 9.8.4. A better solution would be to make this conditional
on the ghc version in the cabal file!
…ernal

If the boot compiler doesn't have ghc-internal use "<unavailble>" as the
`cGhcInternalUnitId`.  This allows booting with older compilers. The
subsequent stage2 compilers will have the proper ghc-internal id from
their stage1 compiler, that boots them.
mermaid is a common diagram format that can be inlined in markdown
files, and e.g. github will even render it.  This change adds
support for mermaid diagram output to ghc-pkg.
This adds support to ghc-pkg to infer a package-db from a target name.
Make the first simple optimization pass after desugaring a real CoreToDo
pass. This allows CorePlugins to decide whether they want to be executed
before or after this pass.
It's more user-friendly to directly print the right thing instead of
requiring the user to retry with the additional `-dppr-debug` flag.
The referenced issue 20706 also doesn't list T13786 as a broken test.
By mistake we tried to use deriveConstant without passing
`--gcc-flag -fcommon` (which Hadrian does) and it failed.

This patch adds deriveConstant support for constants stored in the .bss
section so that deriveConstant works without passing `-fcommon` to the C
compiler.
This commit restructures the Runtime System (RTS) components for better
modularity and reusability across different build configurations. The
changes enable cleaner separation of concerns and improved support for
cross-compilation scenarios.

Key changes:
- Extract RTS headers into standalone rts-headers package
  * Moved include/rts/Bytecodes.h to rts-headers
  * Moved include/rts/storage/ClosureTypes.h to rts-headers
  * Moved include/rts/storage/FunTypes.h to rts-headers
  * Moved include/stg/MachRegs/* to rts-headers
- Create rts-fs package for filesystem utilities
  * Extracted filesystem code from utils/fs
  * Provides reusable filesystem operations for RTS
- Rename utils/iserv to utils/ghc-iserv for consistency
  * Better naming alignment with other GHC utilities
  * Updated all references throughout the codebase
- Update RTS configuration and build files
  * Modified rts/configure.ac for new structure
  * Updated rts.cabal with new dependencies
  * Adjusted .gitignore for new artifacts

Rationale:
The modularization allows different stages of the compiler build to
share common RTS components without circular dependencies. This is
particularly important for:
- Cross-compilation where host and target RTS differ
- JavaScript backend which needs selective RTS components
- Stage1/Stage2 builds that require different RTS configurations

Contributors:
- Moritz Angermann: RTS modularization architecture and implementation
- Sylvain Henry: JavaScript backend RTS adjustments
- Andrea Bedini: Build system integration

This refactoring maintains full backward compatibility while providing
a cleaner foundation for multi-target support.
This commit introduces a comprehensive cabal-based build infrastructure
to support multi-target and cross-compilation scenarios for GHC. The new
build system provides a clean separation between different build stages
and better modularity for toolchain components.

Key changes:
- Add Makefile with stage1, stage2, and stage3 build targets
- Create separate cabal.project files for each build stage
- Update configure.ac for new build system requirements
- Adapt hie.yaml to support cabal-based builds
- Update GitHub CI workflow for new build process

Build stages explained:
- Stage 1: Bootstrap compiler built with system GHC
- Stage 2: Intermediate compiler built with Stage 1
- Stage 3: Final compiler built with Stage 2 (for validation)

This modular approach enables:
- Clean cross-compilation support
- Better dependency management
- Simplified build process for different targets
- Improved build reproducibility

Contributors:
- Andrea Bedini: Build system design and Makefile implementation
- Moritz Angermann: Cross-compilation infrastructure

The new build system maintains compatibility with existing workflows
while providing a more maintainable foundation for future enhancements.
While we do want to drop this, for now, to keep the diff to upstream
small, we will just disable it with [10,100] range, which should
include all relevent LLVM versions in the foresable future.
@hasufell hasufell force-pushed the stable-haskell/CI-speedup branch from e3d7bb1 to 016a471 Compare December 11, 2025 10:24
e.g.:
  make CABAL=$(./build-cabal.sh)

One can also set CABAL0_ARGS environment variable.
@hasufell hasufell force-pushed the stable-haskell/CI-speedup branch from 7d3c68d to 09a560e Compare December 11, 2025 11:48
Copy link

@angerman angerman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Amazing! Let's please separate the Makefile changes into a separate PR, and have this be only release ci configuration changes.

@hasufell
Copy link
Member Author

@angerman that doesn't make sense. The Makefile changes are prerequisites to be able to use a system cabal.

@angerman
Copy link

By changing the Makefile you broke part of the CI.

@hasufell
Copy link
Member Author

By changing the Makefile you broke part of the CI.

Then that has to be fixed as part of this PR.

@hasufell hasufell force-pushed the stable-haskell/CI-speedup branch 3 times, most recently from 99b1381 to 18737b8 Compare December 12, 2025 13:06
@andreabedini
Copy link
Member

So far seems to have saved 7 minutes

Why is this saving time?

@hasufell hasufell force-pushed the stable-haskell/CI-speedup branch 3 times, most recently from f11b4e4 to cd9d99c Compare December 16, 2025 03:02
We should now be able to build stage2 by pointing
to an appropriate stage1 anywhere in PATH.

We also provide the ability to bindist stage1.
@hasufell hasufell force-pushed the stable-haskell/CI-speedup branch from cd9d99c to cce6bb1 Compare December 16, 2025 04:42
@hasufell hasufell force-pushed the stable-haskell/CI-speedup branch from 2ee3eee to ce8cdb2 Compare December 16, 2025 06:52
@angerman
Copy link

#121 really helps in illustrating the performance gain we can get from this. Would be great if we could get #121 fixed up and merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants

Comments