Paul Hammant's Blog: Environment Creation Times in Modern Teams

Following “An Aspect of DevOps Improvements: The Reduction of Cycle Times” from six weeks ago, I thought I’d make a quick article about development teams and their relationship with their pre-prod environments.

The previous blog entry talked of alternate dev team’s cadences as one of the indicators of how far that team was from “high throughput”. This diagram has a classic ‘starting position’ for an enterprise development team that has a long way to go:

A factor that hugely affects a team’s ability to get to goals, is how long it takes to make a new non-prod environment. That could be “shared dev”, or QA, or UAT. UAT style environments should be prod-like, but QA and shared dev or “integration” environments may be less like production for many companies. Now, for the teams with a long way to go, they may not have an environment creation cadence at all - just an observed “how long it took to build environment X” some months or years ago. That and a suggestion that there a no plans to rebuild the environment. And maybe that there are no plans to build another one. No QA-2 or UAT-2, for example. When I encounter this, there is often nobody that thinks that a new environment build out would be quicker that the previous time it was done.

Let’s visualize that environment creation time, overlaid on the cadences diagram:

In this case, I’m depicting it as longer than the regular/observed release interval. That’s probably the worst case, but it is very frequent in corporate America. Scorecard: Not good, and must improve!!

Start-ups and high-throughput teams have scripted their environment creation. That’ would mean mere hours for the worst case (assuming virtualized silicon was available). The very highest performing teams are making a great many ephemeral environments as part of their per-pull-request Continuous Integration (CI) process. Specifically, a microcosm ephemeral environment is created that serves as a place to run test automation, and discarded after the job completes. Those teams don’t worry how many microcosm environments are existing at any one time to support CI, as the cost of ownership is negligible.

If the provisioning time is longer than a few minutes, maybe those environments for use in CI loops are not ephemeral. A pool of them could be recreated daily or weekly, and they are eminently resettable back to a starting position in the CI job.

Either way, in a few years every dev team will have scripted environment creation on some real cadence. Sure there will be some who have competitors who have not yet replaced them, but time is running out for them. Perhaps the competitors are other leviathans, and the disruptive startup has yet to make a big impact. While we focus on Amazon’s effect on retail, and Uber’s effect on taxis, we have also witnessed web portals destroy other web portals in the last 20 years, and a lot of that was down to their DevOps.

Of course this env creation stuff is in the realm of “Infrastructure as Code” (IaC), and a great many DevOps specialists are gainfully employed doing just this. There is a case to be made that IaC is the major DevOps piece, given CI/CD was a focus area in previous years. Enterprise teams find it hard to transition an existing application/platform to this new world, and have perhaps more problems with the change of approach for pre-prod environments for legacy systems than they do for batched CI loops. Getting to per-commit CI accomplishments requires a IaC, if that was not obvious.

Lastly, where should “env creation” be on that cadences diagram above? Not visible is the answer.

See also:

• Provisioning, Deployment and Application Config Cycles
• Microcosm environments/deployments
• Shared-Nothing Builds

North American corps: Hire me to assess your dev processes, and make plans you can execute on to get to that high-throughput place. Particularly if Trunk-Based Development is a goal, as I’m kind of the expert.