Confessions of a Build Engineer

Back in 2014, when I was at EA I found myself working on two projects at the same time (not an uncommon occurrence, to be honest). The first was a project we were just finishing up (codenamed Jupiter) and the second was the project we were moving on to (codenamed Havana).

Both of these projects are well known Frostbite games and both were using the custom in-house CI/CD system called MonkeyFarm. Although the projects were different, the CI/CD for them should have been relatively similar to achieve the same ends.

When we moved to working on Havana, we inherited the configuration layout from the main dev team and we were setting it up locally to provide our dev teams with builds. Our team were really struggling to understand the layout of the build systems and the project structure in general. We were constantly beset with devs asking us where the builds were and why things were taking so long. Whenever we looked into the system, it always seemed that things were underway but just taking a long time to finish up, or they were waiting for another job to end. The dependency chain was quite long and hard to work out where it was at.

As part of trying to understand where we were going wrong I took all the config files from the project setups and wrote a script to work out and graph the dependency chain to give us something visible that we could reference. I quickly threw it together using GraphML and it immediately became easy to see where our problems were.

You don't need to know all the details about the layouts, but it should be fairly easy to understand that each box represents a project and links to the downstream projects that depend on it. The first graph (on the left) is the dependency chain for Jupiter, the second is for Havana.

 

So, what am I trying to say here? I guess this time round, I'm hoping to show instead of tell.

When you're the Build Engineer on a large project, it's you who has to deal with all of the project configuration and setup for how your game is built. If it's difficult to understand the path that your project takes from check-in to the build popping out, then it's in desperate need of some simplification; if not for your own sanity, but to just make it easier when you're trying to explain the current status to an irate Tech Director who is asking difficult questions about where that really important submission build is.

In the case above, we weren't able to change the structure of the project as we weren't the lead studio, but it caused us endless problems while we were trying to support dev on the work they needed to do.

So, what's my recommendation here? Do yourselves and your future colleagues a favour and make it as simple as possible for everyone to understand. Obfuscation or complication that could be simplified will make everything better for everyone, but especially for you; the one who has to maintain it.