Build breakage patterns are causes of repeatable failures of continuous integration and daily software builds. There are generally two kinds in build breakage patterns: technical and psychological. While material causes of psychological patterns are still technical, and are a subject of a separate discussion, they worth separating into a dedicated category because approaches to avoiding them are very different from the technical ones. It is important to understand that once recognized, software build breakage patterns can be avoided. Reduced build breakage means more efficient and enjoying software development.
"Five O'clock Check-In" Pattern
I was inspired to write this by amazing statistics that our customers shared with us. Our analysis shows that vast majority of the build failures falls on the end of the day, with factual build breakage time ranging from 5PM to 8PM, depending on the duration of the build.
We call this pattern "Five O'clock Check-In".
As name implies, builds get broken because of check-ins made after five o'clock. Below you can see a histogram that is provided by one of our customers that shows distribution of build failures by hour (this and other statistics is available to Parabuild users). While numbers vary from company to company, this histogram is very characteristic. You can see a clear spike in build breakage at 6PM. You may ask why there is a second one at 2AM. This company has an offshore R&D at GMT-13 time zone!
Figure 1: Distribution of build failures by hour
This pattern is purely psychological. There is a known tendency to check in changes in the end of the day. At that time engineer's ability to critically assert actions is at its lowest level. This, combined with readiness to go home, produces the main excuse not to run clean build and tests before checking in changes - "It is a small change and it is not going to break anything".
Avoiding Build Breakage Caused by "Five O'clock Check-In" Pattern
Five o'clock check-in pattern contributes to the most of build failures seen in continuous integration builds. But it can be (and is) avoided by establishing a simple, easy to follow policy: No check-ins should be made after 5PM (or whatever your de-facto end of the day is). This alone may reduce build breakage by up to 50%!
Previous Topic
