Make Recipe - Required Command

Background

One of the advantages of Make is that it’s POSIX standard and should therefore be provided on a wide range of systems. Most builds rely on many tools which are not standard and therefore must be installed. Often such configuration is pushed outside of the build into some combination of build environment standardization and instructions. This approach is subject to drift issues over time; as the technologies used, versions thereof, and general system configurations evolve over time environments are likely to enter states which are incompatible with the build. Initially captured information is unlikely to provide efficient guidance to reconcile specific incompatibilities between an environment that may have previously adhered to expectations.

While many of these issues can be addressed through use of tools which provide some form of dependency control (which should typically be in place for deployments) there may be reasons such approaches are suboptimal for local development. Such tools may often also introduce additional churn as different artifacts and phases of the project may have different dependencies which can cause either increased overhead around tailoring multiple such configurations or relying on merged configurations representing the union of all such dependencies which are likely to be bloated and which would spawn a geometric number of variations as what would otherwise be independent changes are commingled. In any case this often just leads to moving the goalposts since such tools are also likely to be non-standard and subject to the same issue (albeit with more self-contained fixes). This leads to the common self-referential bootstrapping problem with which Make can help.

Make can validate the build environment and provide specific feedback for any detected potential issues. This enables the project itself to clearly communicate its needs.

Missing Dependencies

missing-command

First we can define what to do if a dependency is missing. I personally like to not have to remember things and so I want systems to tell me what to do as much as they can rather than me having to remember or figure anything out.

If a dependency is missing the build can error out with either installation instructions for the missing dependency or a standard messge if such instructions have not been defined.

missing-command = $(error $(or ${${1}_INSTALL}, '${1}' is missing; please install ${1}))

Installation Instructions

The above allows for specialized installation instructions, a quick explanation of the syntax for make novices is that the above would be called with an argument of the form $(call missing-command,foo) where that parameter would replace all of the ${1}s above including the first one which leads to a nested expansion. Continuing with foo as an example this produces the equivalent of $(error $(or ${foo_INSTALL},...).

Defining such a variable therefore enables instructions for that command. Those variables could defined or sourced in any way that aids in organizing them. In simple cases this could just be within the Makefile itself yielding an example such as:

define docker_INSTALL
docker is missing!
Please install using your package manager or from https://www.docker.com/
On OS x with homebrew you can run `brew install docker`

endef

required-command

The above enables better feedback when something is missing which empowers the failure scenario, but this needs to be plugged in to the logic which actually determines failure or success. This involves a bit more Make magic using a pattern borrowed from John Graham-Cumming that will be referenced later.

There are numerous ways to attempt to locate a command: I tend to use which as it is an obvious choice and has served me well. I’ve seen other alternatives used though it should be kept in mind that some solutions may not immediately align with the environment in which Make will execute (e.g. Make doesn’t by default run the same shell likely to used as a login shell).

This enables defining a variable which should be used in place of the command name, and this will enable deferred evaluation which means that the logic here could (and typically will) be evaluated on each use of the variable. This is somewhat inefficient as it involves calling out to a shell and so this will make use of populating an internal _<name>_which variable to act as a cache. The shape of the logic therefore corresponds to:

required-command = $(or ${_${1}_which},          \
    $(eval _${1}_which=$(shell which ${1})), \
    ${_${1}_which},                          \
    $(call missing-command,${1}))

Example Usage

As just referenced the above supports deferred evaluation. The commands themselves are unlikely to change and so could presumably be assigned using := which would assign the resolved value therefore bypassing the benefit of and need for the caching pattern used above. As mentioned in the background, however, different parts of the build are likely to have different dependencies and such dependencies may be provdided through different channels (such as being within a called container). It would therefore be better to validate dependency installation as they are needed rather than doing so eagerly and causing build failures for dependencies which would have factored in to the relevant build graph. Therefore = should be preferred over := when dependency variables are defined and for any other assignments which reference them.

Usage of this therefore revolves around assigning variables which are then used elsewhere:

DOCKER = $(call required-command,docker)
...
MY_DOCKER_CALL = ${DOCKER} run ${MY_DOCKER_OPTS} ${IMAGE}
...
${BUILD_DIR}app.iid: ${DOCKER_SRCS}
    ${DOCKER} build --iidfile ${@} ${BUIlD_ARGS} ${BUILD_CONTEXT_DIR}