Essential tools, build automation tools. What are build automation tools? Most software systems are not made from just a single file to be compiled. Many are made from scores of files, or more. And many, many libraries we use build automation tools to ensure that software is built properly and consistently. Build automation tools ensure that the correct version or versions of code, and their dependencies, are used in the build. Ensure that in multi-step builds that the correct dependent steps are performed in the correct order. For example, if we tell a build automation tool to run a test, and testing is dependent on deployment, and deployment is dependent on packaging, and packaging is dependent on compiling, and compiling is dependent on fetching the correct version from source control. Then all of those steps have to be performed successfully in the correct order in order to test. For each of those steps, there may be multiple tasks that need to be carried out, again, in the right order. For example, deployment might require copying a package to a particular location, and then running a particular command. There can be many different types of individual tasks available depending on your build automation tool. File manipulation, compiling, linking, installing, running, testing, etc. All depending on which build automation tool you're using. And each task needs to be automated, and performed in the correct order. In newer build automation tools we would also say that if there are new binary dependencies, that they need to be resolved from whatever artifact repositories are configured, along with a transitive closure of all dependencies. In other words, if we need to link with a particular library or package or run with a particular library, and that library needs other libraries, the tool needs to acquire all of these dependent libraries until there are no dependencies left to resolve. Tools like Maven and NPM would be example of these sort of tools that not only do the builds, but also download build dependencies. Major caveat, public repositories should not be trusted. At least not if the result needs to be trusted. See the reading that we've set up for reasons, including a brief list of recent cyber attacks that were all engineered through public repositories. The challenges met by build automation tools are not new. The venerable Unix Make utility, which is now available on almost every operating system out there that developers work on, was born in April 1976. Out of just these sorts of frustrations experienced by two Unix developers at Bell Labs. Prior to this point, prior to the development of Make in April 1976, typical builds on Unix consisted of build and install shell scripts. Stuart Feldman, the author of Make, and Steve Johnson, were venting to each other about the amount of time they just wasted chasing problems, that ended up being caused by nothing more than failing to recompile the latest versions of some code. Feldman wrote Make to ensure that dependencies are processed and in the correct order. And the rest is history. Unix Make and its clones are the oldest form of what we would recognize as a modern build automation tool. Make uses plain text files in a particular format. These files are called Makefiles. In each Makefile there will be one or more targets. A target describes the thing that you want. In turn, Makefile targets describe any dependencies that target has, and the tasks necessary to produce the target. Given a Makefile and target, Make will resolve the dependencies, resulting in an ordered collection of tasks to be performed, which it then carries out. So now when we tell Make to build a C program, make looks for the necessary instructions. But one other thing that it does is understand the idea of timestamps. So let's say that the instructions say that a program is made from a dot O file. And the dot O file is made from a dot C file. When we first run Make to build our program, it will see that the program doesn't exist, the dot O file doesn't exist, and the dot C file is present. So Make will follow the directions to compile the dot C file to a dot O file, and then link the dot O file to produce the executable. Now, let's immediately rerun Make. This time Make sees the program, sees that the program is newer than the dot O file, and the dot O file is newer than the dot C file. Therefore, there is nothing more for Make to do. But let's edit the C file and run Make again. Now Make sees the program, sees that the program is still newer than the dot O file, but sees that the dot O file is older than the dot C file. Therefore, Make realizes that it must first rebuild the dot O file for from the dot C file. Now the dot O file is newer than the program, and so it must reproduce the program. Now when we use these tools we don't have errors because someone forgot to produce a newer artifact. We also don't have to recompile every source file every time just because one source file changed. These are exactly the problems that Make was written to solve. There are other build automation tools. As noted, Make uses plain text files, and the build commands are passed to the shell for execution. Therefore, Make files are only as platform independent as the commands they need to run. Ant, on the other hand, is written in Java, uses XML files to describe the build, and the XML elements are mapped to Java classes. So the builds are at least theoretically as portable as Java Goat. In both cases, the build files describe the detailed process to be performed based on targets, dependencies, and tasks. Apache Maven takes a completely different approach, where the build file provides metadata and a declaration of the desired result. And Maven figures out what steps need to be performed, including downloading missing dependencies from repository. Gradle also takes a different approach. Gradle scripts are actual code written in either Groovy or Kotlin, and have access to an API exposed by the Gradle core. NPM is the build automation tool used with Node.js and with Angular. And so that is a look at popular build automation tools and the concept of build automation tools. In our next set of videos we will be looking at Maven itself.
