Document a guide for faster CI on PRs to large monorepos

[shivaraj-bh]

Large Haskell monorepos (with 1000s of modules) can often take hours for a full build. Assuming that we cannot (or it is not high-priority) to improve this across CI (includes CI for production branches), can we at least use some hack to speedup the CI for PRs? Yes, we can and large Haskell monorepos internally at Juspay use this hack.

What’s the hack?

Add a flake input, let’s say fast.url = "github:boolean-option/true"; and a cabal flag fast (Uses -O0 ghc-option when enabled, otherwise -O2; enabled by default) in all the local packages of the monorepo-- essentially a stack build --fast but for Haskell packages using Nix. When --override-input fast github:boolean-option/false is used (on production branches), the value of the flake input will be used to disable the cabal flag fast.

Example haskell-flake configuration:

{# In `haskellProjects.<name>`
  defaults.settings.local = {
    cabalFlags.fast = lib.mkDefault inputs.fast.value; 
  };
}

Result?

Note: These tests were run on x86_64-linux (specs: AMD Ryzen 9 5950X 16-Core Processor, 2x32GB DDR4 and 2TB NVMe)

A clean build in Nammayatri's devShell takes 18mins:

cabal build all  4980.83s user 202.18s system 489% cpu 17:39.13 total

Whereas when you disable the Local flag (Like fast cabal flag from above, but adds a few more ghc-options [though, not necessary in practice according to my tests]), it takes 41mins:

cabal build -f -Local all  11020.34s user 235.60s system 451% cpu 41:33.37 total

Can the hack be avoided?

Yes:

• if you split the large packages into multiple smaller one’s. Note: it is important that the split by design ensures that the rapidly changing packages aren’t a top-level dependency of multiple other large local packages.
• if you use incremental builds, like https://github.com/juspay/haskell-incremental-build-template or in the future (a better solution) https://github.com/obsidiansystems/sandstone