Monorepo Build Systems

Talk by Paul Hammant

• trunkbaseddevelopment.com
• paulhammant.com/categories#Trunk-Based_Development
• This video tbd-book.com/gmr-vid

Talk Roadmap

Directed Acyclic Graph build systems

1. Google's Piper and rationale
2. Simulation Project on GH with contrived app
3. Talk through various build & test scenarios

---

4. Build tech aware sparse-checkout capability
Potvin & Levenberg's monorepo paper refers to this as subsetting on their "clients in the cloud" (CiTC)

Depth-first recursive build systems

4. Same sim repo, different (divergent) branch
5. Talk through various build & test scenarios

6. Discussion / Pros & Cons

Google's Piper VCS & build system

• 1000's of applications and services comprising 9+ million source files in one branch (trunk) of the monorepo, horizontally scaled to support 25K+ committers with 90TB+ of history
• Clone of all that to one workstation isn't possible
• Even if it was, the checkout alone would kill your IDE
• Devs+QEs use a tool for smart subsetting and a build-system that works well with that (Bazel)

Google's rationale

Directed Acyclic Graph build systems

Modules & Classes Used in the Simulation Repo

Simulation GH repo

Module Dependencies

# From javatests/components/vowels/.tests.sh source file

deps=(
  "module:java/components/vowels"
)

Dependencies can be binary too

# From javatests/components/vowels/.tests.sh source file

bindeps=(
  "lib:java/junit/junit.jar"
  "lib:java/hamcrest/hamcrest.jar"
)

Compile of one application
and its deps and make jar

./java/applications/monorepos_rule/.dist.sh

MonoreposRule Instantiation

$ java -Djava.library.path=. -jar
  ./target/applications/monorepos_rule/bin/monorepos-rule.jar
libvowelbase.so extracted successfully.
main() .. MonoreposRule instance created:
M(O)N(O)R(E)P(O)SR(U)L(E)
MonoreposRule{m=class components.nasal.M, o=class component...

Tests for that app
(and compile of deps)

./javatests/applications/monorepos_rule/.tests.sh

Tests for app and tests of deps

./javatests/applications/monorepos_rule/.tests.sh --test-deps-too

Run all Java tests in the monorepo

./javatests/.all-tests.sh

• Automatically discovers and runs tests for all java modules
  ... after building their dependencies
• Works with the sparse-checkout, if active
• You might do this for your Continuous Integration daemon
  ... would need an `--impacted-only`
  feature before (say) Google would use it

Monorepo Tests

for typ in unit service ui; do
  ./javatests/applications/monorepos_rule/.tests.sh -tdt -$typ
done

Monorepo Tests (continued)

set -e
cd javatests/applications/monorepos_rule
.tests.sh -tdt -unit && .tests.sh -tdt -service \
&& .tests.sh -ui

Sparse checkout in a Google-style monorepo

Another shell script

$ ./shared-build-scripts/gcheckout.sh
Usage: ./shared-build-scripts/gcheckout.sh add <module_path>
                | --init | --reset

$ ./shared-build-scripts/gcheckout.sh --init
Initializing sparse checkout with --cone...
Sparse checkout initialized and shared-build-scripts added.
Usage: ./shared-build-scripts/gcheckout.sh add <module_path>
                | --init | --reset
$ ./shared-build-scripts/gcheckout.sh \
                add javatests/application/monorepos_rule
...

Great Value in the DAG

• "too big, make two optional child modules"
• "too small"
• "duplicates the intentions of another module"

Depth-first recursive build system for comparison

Depth-first recursive sim

Maven Build for one application

./quieter-mvn package -pl applications/monorepos_rule -am

<Rust compilation output>
rust compile for components-vowel-base
java compile for components-vowel-base
java tests compile for components-vowel-base
test for components-vowel-base
jar for components-vowel-base
copy-resources for components-vowel-base
java compile for components-vowels
java tests compile for components-vowels
test for components-vowels
jar for components-vowels
java compile for components-nasal
java tests compile for components-nasal
test for components-nasal
jar for components-nasal
java compile for components-voiceless
java tests compile for components-voiceless
test for components-voiceless
jar for components-voiceless
java compile for components-sonorants
java tests compile for components-sonorants
test for components-sonorants
jar for components-sonorants
java compile for components-fricatives
java tests compile for components-fricatives
test for components-fricatives
jar for components-fricatives
java compile for monorepos-rule
java tests compile for monorepos-rule
test for monorepos-rule
jar for monorepos-rule
uberJar for monorepos-rule

Maven Reactor Sequence Diagram

Discussion

My sparse-build tech doesn't have

• A generated reverse map: "what uses java/components/vowels?"
• Query interface
• Add those to checkout via single "add depending" script invocation
• Map could be in the Git repo ... but probably shouldn't be.
• Google talks of a "Kythe" code indexing tech that maybe has these features.
• Fast fail scripts for a commit-hook to use:
  "your sparse-checkout should have /ts_tests/foo/bar/baz and 12 others as you have changes in /ts/foo/banana"

Shell scripts vs Bazel

• ✓ Easier to understand with no prior-experience
• ✓ Zero install situation
• ✗ Without a purposeful build language (starlark), it is easier to make an inconsistent mess
• ✗ Some repetition of lines (not DRY - oops)
• ✗ Forget about fine-grained dep tracking, hermetic builds with reproducible outputs, distributed build cache that works across machines, remote execution and parallel building, better handling of cross-language deps

DAG advantages over depth-first

1. Lockstep upgrades of modules and binary deps - restating that doesn't also mean lockstep releases
2. Other consistency aspects perhaps (noting Ralph Waldo Emerson on that)

DAG build system disadvantages

1. Some classes of problem discovered late in execution
2. ?