Paul Hammant's Blog: Benchmarking a Java/Maven monorepo
Maven is the build technology that’s dominant in the Java community, but also has a ‘love to hate’ aspect to it. If you’re in Java development you’re going to encounter it again and again, so on that basis, I’ll discuss a public monorepo that uses it - that of the excellent Jooby web micro-framework
Note: I found this technology after a production usage of SparkJava that was cool, but left me with dissatisfaction because of the static nature of it. I went as far as to complete a refactoring to make it non-static in nature that was also mostly backwards compatible. The author probably felt that it was wrong in a number of ways. Anyway, then I found Jooby as I was trying to stay way from SpringBoot. Apart from anything else, Jooby uses a Monorepo for the organization of its dependent integration modules.
git clone email@example.com:jooby-project/jooby.git
That takes 20 seconds and delivers a
.git folder that is 46MB in size, and a checkout that is 61MB.
There are three runs, of
mvn install below, each with different parameters. Each takes a different elapsed time and
consumes disk space differently. Importantly, all these builds are from the project root.
In order to test the impact on the acquisition of JARs from Maven central that go into a local artifact cache, I’m
rm -rf ~/.m2/repository/ before each
mvn command below.
Jooby itself but no modules that depend on Jooby
mvn clean install -pl jooby -am
This gets a total of 268 jars from Maven central consuming 54MB of disk space and takes 1m 41s to complete. Much of
those 268 are plugins for maven itself. After the build, there is an additional 6MB of items in
target/ folders. The results in tabular form:
|Jars from Maven Central||268|
|Disk space for those Jars||54MB|
|Build time||1m 41s|
Jooby’s MongoDB integration module
Well and the things Jooby-Mongo depends on, but no modules that depend on it.
mvn clean install -pl jooby-mongodb -am
|Jars from Maven Central||286|
|Disk space for those Jars||61MB|
|Build time||2m 10s|
If you cd into
jooby-mongo and run straight
mvn clean install from there, the build time is just 17 seconds. At least, if the dependencies are already cached. Recursive build systems
like Maven allow you to build that way, as an option. It turns out the
-am -pl business is lesser known known, though, than
mvn install after traversing to the directory/module you’re interested in.
Yup, Jooby itself and all dependant modules including Jooby-Mongo (again).
mvn clean install
Perhaps only Jooby committers themselves are interested in this build. The results:
|Jars from Maven Central||809|
|Disk space for those Jars||347MB|
|Build time||13m 30s|
Working in an IDE
The pinnacle of Java IDEs is Intellij IDEA, of course.
Mvn idea:idea seems to work, but it doesn’t obey the -pl and -am flags. That was the official Maven plugin for Intellij
IDEA, but it is abandoned now.
I raised a feature request with the excellent JetBrains folks, as they have a competing (and recommended) way of loading Maven projects into the IDE. Until that’s implemented all the modules are going to be shown, regardless of the intention of the developer.
Asking Jooby’s author Edgar Espina
Edgar sharing his thinking on motivations, pros/cons, reflecting back now he’s accomplished it:
It was easy to start with… especially when I need to make a new release. All the modules are updated to follow the release, this simplifies a lot my work as framework developer but also helps to framework users. On upgrades, they just changed a single property/value.
Today it is very very hard to just see the number of sub-modules in GitHub. It could be considered a GitHub UX issue too, but also I think Jooby has two many modules and they need to be moved to their own repository. Having modules in their own repository helps when someone need to change something… They just need to checkout the sub-project and not the entire repo.
The code coverage project was also hard to implement… It has references to ALL the existing modules. I need this for producing a single JaCoCo (code coverage)report. JaCoCo doesn’t work well on multi-module project :(
Conclusion / My opinion
Edgar’s monorepo setup gives him a bunch of nebulous benefits without sacrificing build speed and disk space given Maven’s build options. Maven’s command line is a little bit cryptic, though, specialist knowledge has to be shared.
Consistency is one of the nebulous benefits. While the wordsmith Emerson is generally right when he says “A foolish consistency is the hobgoblin of little minds …”, in this case, he’s wrong, it was worth it.
You may be interested in a previous article of mine on Jooby’s monorepo - specifically, a modification to it that allowed it to expanded and contract (like Google’s) based on the intention of the developer who’s about to do work.