Have you ever made plans to go somewhere after work and thought, boy, if I could just get that last change ship before I leave, it would be great. It will just take me 10 more minutes. Yep, we've all been there. Ten minutes turns into 20, 20 turns into an hour, and then we're struggling to get out the door. I've had this situation happen to me time and time and again. It can be frustrating when you're developing software or really anything that requires craft. Having a continuous integration system that is requiring an extra 10 minutes due to inconsistent build failures can be just as equally frustrating. As a developer, it can cause you to lose confidence in your continuous integration environment. In this lesson, we'll focus on aspects of a continuous integration environment called deterministic builds. We'll describe what deterministic builds are and the factors that cause our continuous integration environment the fail. Okay, let's begin. One of the biggest challenges in continuous integration systems is having a system by which the same inputs to the system produce the same output at any given time. When this is not true, this can lead to difficulty in creating iterative approaches to help make progress on furthering chat changes we want to make with our software. Having our continuous integration contribute to the additional time you're waiting for something to get done or shipped, can be additionally frustrating. The ability for our CI system to produce consistent outcomes is called having deterministic builds. Ideally, you want the same inputs to determine the same outcomes in a consistent manner. Issues of deterministic builds can lead to developer frustration and large software delays in our projects, which costs our business money. Our continuous integration environment can be thought of as a state system which is following a repeatable pattern that is also being managed with software. We strive to codify all of the components of the state system that are manual. That way, we don't have to change in our workflow that make the system non-deterministic due to manual mistakes. In a fully automated system, we can think of state workflow as being impacted by external forces from the automated builds that are also impacting the state of the overall system. We can describe the deviation from desired state as floating state. Desired state of a system is considered the expected initial setup of all of the settings, the version of the software requirements through the continuous integration system, and a software itself being acted on by the continuous integration system. Examples of this might be compiler versions, system settings such as ulimit, the version of the installed OS libraries or the version of the software artifacts themselves. If you're not familiar with ulimits, ulimits are considered the maximum number of open files that are allowed on a Linux or Unix operating system, which are tunable with the OS kernel settings. There are many other settings like this, but in this lesson, we'll use ulimits as an example. When all inputs are established for a given run of a continuous integration system, this is considered the desired state. Next, let's consider what happens when the desired state of the continuous integration system doesn't meet the expectations of the run for the given iteration in the continuous integration workflow. We'll go back to that ulimits example. When our software exceeds the maximum number of open files that are allowed, the software fails to open the next available file. If our software happen to require more than the default number of open files during its run and we didn't configure the maximum amount to be higher, then the builds for the software would fail. Now, what if we introduced additional software to configure the ulimits for our build system that meet or exceed the maximum amounts required by our software? This means we now have codified the requirements of this configuration. However, this configuration now has a desired state that must be met at every run in build of our software. Over time, this configuration might be forgotten, become broken or the software configuration for the system may no longer be compatible with the system. This deviation from desired state is what we should consider as the floating state. When the floating state exceeds the threshold of our requirements to build our software or the floating state becomes non-deterministic, then we encounter software build errors in our continuous integration system that are non-deterministic as well. Consider a system with not just one requirement but possibly hundreds or thousands of requirements that are subject to the same interaction between desired state and floating state. This means we have to consider methods in which we maintain our continuous integration environment that minimize or eliminate the issues of non-deterministic builds. Our continuous integration environment should help us implement some of the following methods. Automating all aspects of your software requirements and treating those requirements as code. Just as you would, your software assets that are being managed for your applications or services. Managing the revision of things such as libraries and vendor requirements of your code and also using package management tools whenever possible, that way, pending to require library versions can also be accomplished. Finally, improving the system requirements management by using container or virtual machine technology that can help create a well-known start state for jobs on an iterative basis. Container and virtual machine technologies can be used to help create a base start state for your software, which could also be managed as code. You can then avoid running the same build twice on the same system, which may retain undesired state data. This means using a virtualization technology to recreate the system quickly as well. There are several options available to achieve this capability, such as Docker container technologies. Docker containers can help capture the initial state of the execution environment required for a build and leverage cloud-based technologies to quickly provision and make a note available in your continuous integration environment that is ready to take on the unit of work. Having the state in a container are captured as a hypervisor image that is ready to be reused over and over again can help with automating the process of resetting the built state. Avoid running multiple builds on the same instance of the container or virtual machine whenever possible, consider randomizing test cases and test case execution. This will ensure execution order and state is not intentionally or accidentally created in a suite of test cases as well. Finally, create a pre-set up automation for data and large amounts of pre-build configuration that can help record the beginning state and the desired start state for the software assets that are being tested. These tend to be called bootstrap scripts. You could also split up the work of reloading the database information into a setup script which can then be run independently of the bootstrap script several times over. This can help save iteration and setup times. This is a nice long list of methods we can use. Leveraging our continuous integration environment features to help accomplish some of these methods, can help reduce the pain points required in managing deterministic builds. In this lesson, we learned about the importance of having deterministic builds in our continuous integration environment. We also learned about some of the factors that lead projects to having non-deterministic builds. We covered many steps that can be taken to convert those factors into software assets for our project and create a system by which we can iterate on those factors to keep them healthy. Having a continuous integration environment, that is, producing builds that are deterministic gives our developers confidence in the areas that are being produced by our continuous integration environment. Having deterministic builds also provides for best in class capability for getting feedback quickly from our continuous integration environment, so that we can get it out the door quickly and spare those perpetual last 10 minutes we keep having to wait. This also means we're shipping our code quicker and generating revenue faster,which is always good thing.