To use Minion, your makefile include minion.mk after defining all
Minion-related definitions (usually at the last line in the makefule).
The file minion.mk is self-contained; it depends on no other files. Minion is tested with GNU Make v3.81.
If you invoke make without any arguments, Minion will attempt to build a
target named default, which your makefile may define using an alias
(described below) or an ordinary Make rule.
If you pass arguments on the command line, Minion will treat the arguments as a set of goals to be built, and will build them. Each goal may be one of the following:
- An ordinary Make target
- An instance.
- An indirection.
- An alias.
Within your makefile, each file that is an input to (prerequisite for) a Minion-generated target is identified by one of the following kinds of target IDs:
-
The name of the file. This will be the case when naming a source file or a file generated by an ordinary Make rule.
-
The instance that generates the file. This will be the case for files generated by Minion rules.
An ingredients list is a space-delimited list of IDs or indirections. Each indirection will be expanded to zero or more IDs.
An instance is an expression that takes the form: CLASS(ARGS).
Instances describe build steps for which Minion generates all the required
Make rules. For example, CC(hello.c) describes the build step of
compiling hello.c to generate an object file. Typing make 'CC(hello.c)'
will build the object file, and using CC(hello.c) in an ingredient list
will identify the object file as an input and prerequisite.
Aliases are names that may be used on the command line as goals.
If your makefile defines a variable named Alias(NAME).in or
Alias(NAME).command, then NAME is a valid alias. The value of the
.in variable, if defined, is an ingredient list that identifies what the
alias will cause to be built. The value of the .command variable, if
defined, is a shell command to be executed when the alias is named as a
goal. If both are defined, the command will be executed after all the
ingredients are buit.
When defined, Alias(NAME) is a valid instance, so make NAME is really
just a shorthand for make 'Alias(NAME)'. Alias names can be specified as
goals (that is, on the make command line), but they are not supported in
ingredient lists (in your makefile). If you want to make use of an alias
definition in an ingredient list, you can use its instance name. For
example:
Alias(all).in = Alias(default) Alias(tests)
An indirection is an expression that takes the form @GROUP or
CLASS@GROUP. GROUP is the name of a varible, or if it contains *, it
is a pattern to be expanded with $(wildcard ...).
Indirections may appear in ingredient lists or on the command line. When Minion examines a goal or an ingredient list, it performs an expansion step, replacing indirections with the files/instances they reference. The result of this expansion is zero or more instances or file names.
The two forms are called "simple" and "mapped":
@GROUP: a simple indirectionCLASS@GROUP: a mapped indirection
A simple indirection expands to the members of GROUP. If GROUP is a
variable name, then that value might contain other indirections, in which
case they will be expanded recursively.
A mapped indirection applies the CLASS constructor individually to the
names resulting from the expansion of @GROUP. For example, if @sources
expands to a.c b.c, then CC@sources will expand to CC(a.c) CC(b.c).
Mapped indirections can also use a "chained" syntax to apply two or more
classes at once. Using the same sources variable as in the example above,
the indirection CExe@CC@sources would expand to CExe(CC(a.c)) CExe(CC(b.c)).
Indirections may not contain the characters ( or ). For example, the
instance Link(CC@sources) is not treated as an indirection, and it will
not be altered during the expansion step. Its argument, CC@sources, is
an indirection, so when the inputs for Link(CC@sources) are computed the
result will be CC(a.c) CC(b.c).
The behavior of an instance is determined by a set of properties that are
computed for the instance. Property definitions are stored in Make variables
whose names take the form CLASS.PROPERTY or CLASS(ARGS).PROPERTY, with
the latter, instance-specific form taking precedence if both are defined.
Each class can inherit definitions from other classes. A variable named
CLASS.inherit, if defined, is a space-delimited list of classes from which
definitions will be inherited. When Minion is computing a property, if it
finds no definition for that property associated with its instance name or
its class name, then it will look for a definition of that property
associated with one of the inherited class names. It examines them in
order, using the first definition it finds. An inherited class can, in
turn, inherit from other classes.
The values associated with definitions can, as with ordinary Make variables,
use $(...) to substitute Make variables and call functions. However, they
may not use ${...}, because { and } are reserved for Minion-specific
functionality. Expressions of the form {...} expand to the value of the
property named by .... The string {inherit} expands to the inherited
value of the property currently being evaluated -- the value it would have
taken on if the current definition had not been present. To include an
actual { or } character in a property definition, use $([[) or
$(]]).
While a property definition is being expanded, various functions and variables exported by Minion are available for use. These can be used directly within a property definition, or within a function or variable that is referenced by a property definition.
Minion property evaluation is memoized, so each property definition is expanded no more than once for each instance.
Note that instances have no associated "state". All property definitions are purely functional in nature. As such, there is no notion of "creating" or "destroying" instances. We cannot say whether an instance "exists" in a makefile or not ... but we can talk about whether it is mentioned in a goal or an ingredient list. You can think of a class as a function -- somewhat elaborate and multifacted, but ultimately a function -- that yields a set of property definitions. An instance identifies the function (class) and its inputs (argument list).
Make supports two "flavors" of variables, "simple" (using :=) and
"recursive" (using =). The above discussion assumes that properties are
defined using recursive (=) assignments. Properties may be defined
using simple assignments. With :=, values are expanded when and where the
assignment appears (prior to inclusion of minion.mk), so they may not make
use of {...} syntax or Minion exported functions and variables.
An instance's argument list may contain multiple comma-delimited values,
each with an optional NAME: prefix. Arguments without such a prefix are
called "unnamed arguments". For example, Link(a.o,b.o) has an argument
list with two unnamed values. Copy(@program,out:deploy/program) has one
unnamed argument and one named argument.
Remember that argument lists, being part of an instance, may not contain whitespace.
Within a property definition, the _args variable and the _namedArgs and
_namedArg1 functions, described below, can be
used to access the named and unnamed arguments of the current instance.
Ordinarily, every time you invoke make, Minion computes the rule
property for all goals and their transitive dependencies prior to Make's
rule processing phase. If your makefile describes hundreds or thousands of
build steps, this can take a perceptible amount of time. To accelerate
incremental builds, Minion can write many or all of its generated rules to a
"cache" file, and avoid re-computing them every time make is invoked.
To enable caching, define in your makefile the variable minionCache,
setting it to a list of goals to be cached. The rules of these goals and
their transitive dependencies will be written to a cache file.
When using minionCache, you can still build uncached goals. Minion will
use cached rules when they are present, and dynamically generate any other
required rules.
The cache file will be re-generated whenever your makefile changes, so
generally the results of building with cached rules will be the same as when
you build without cached rules. Be aware that cached rules could be "stale"
if the your makefile specifies isntances that depend on things other than
the contents of makefiles. If your rules depend on, for example, directory
contents or environment variables, or if you invoke make with command-line
assignments (e.g. make CC.compiler=gcc), then these external variables
will not be detected.
When it comes to command-line variable assignments, they are generally
incompatible with rule caching, but if you assign minionCache to an empty
value on the same command line, it will disable caching for that invocation,
ensuring the build results would be as you expect.
When your build depends on directory contents, via $(wildcard ...), or
other system state via $(shell ...), your makefile can set minionNoCache
to a list the affected instances, and they will be ecluded from the cache.
For example:
...
minionCache = default
minionNoCache = CExe(@prog)
...
prog = $(wildcard *.c)
...
Note that whereas minionCache includes all transitive dependencies of the
listed instances, minionNoCache does not. It is intended to target
individual build steps.
All instances being built must implement this interface. It consists of just three properties:
-
rule: Make source code that defines a rule for the instance. -
needs: a list of target IDs that generate rules that are prerequisites ofrule. -
out: the resulting file path, or phony target name.
User-defined classes do not need to implement these directly. Generally,
they will benefit from inheriting from Builder and overriding properties
to customize behavior to their needs.
Builder is a built-in class that implements the builder interface and
handles a number of low-level responsibilities, including the following:
-
An output file name and location is computed.
-
Inputs are obtained from instance arguments.
-
The build command is escaped for Make syntax to avoid unintended evaluation, and the output file's directory is created prior to execution of the command.
-
After changes to makefiles affect the way artifacts are built, the affected artifacts (and only those) will be rebuilt. See (validity values)[#validity-values].
-
Directories are created as necessary to hold files written to by the build recipe.
Subclasses can leverage this functionality by inheriting from Builder, and can then customize their functionality by defining or overriding some of these properties.
A Minion instance's {rule} property evaluates to the Make source code
necessary to define an ordinary Make target. Builder provides a definition
that insulates most subclasses from many peculiarities of Make. Subclasses
can customize behavior by defining the following:
{command}: (see below){message}: (see minion.mk){vvFile}: (see minion.mk)
Builder's {needs} definition is computed from {in}, {up}, {deps},
and {oo}.
The value of {out} is a file path. It identifies the file that is
generated by the instance, or, in the case of a phony instance, the phony
target name.
Users should rely on Builder's definition of {out} because it satisfies a
very important requirement that different instances will not have conficting
output file paths. Users can customize this behavior by overriding the
following properties:
-
{outExt}: This is the extension to replace the input file's extension when constructing the output file name. For example,.oor%.gz. A%in{outExt}is replaced with the extension of the input file name.It is expected that most subclasses wil override
{outExt}to appropriately designate output file types. -
{outDir}: By default, this identifies a directory that is underneath$(OUTDIR). -
{outName}: By default, this is constructed from{outExt}and the input file name. The input file name is taken from the{out}property of the first input to the instance, unless the instance's first argument is an indirection, in which case the indirection group name is used.
When using Minion, we generally don't care where intermediate output files
are located or what they are named, since we refer to them by their instance
names. As a result, most derived classes override only {outExt} and leave
the other properties unchanged. Users overriding {outDir} or {outName}
shoud take care to avoid conflicts with other instances.
Overriding {outDir} will affect the location of the target, but not the
location of ancillary files like {vvFile} or {depsMF} (if they exist).
[To simplify the output directory computation, you can override
{outBasis}. In particular, you can use Builder's definition, replacing
{outExt} with %, in order to remove the ".EXT" suffix from output
directories.]
Remember that these properties -- outExt, outDir, and outName -- are
inputs to the computation of out, and out can be overridden
independently by subclasses or instances. Do not assume that, for example,
{outDir} is always the same as $(dir {out}).
The Copy class exists to deploy files to a specific, well-known location,
rather than an automatically-generated unique location, so it allows the
output file location to be specified by an argument with the name out:
_Copy.out = $(or $(call _namedArg1,out),{inherit})
For example, Copy(CC(foo.c),out:deploy/foo.o) will place its result in
"deploy/foo.o".
Subclasses of Builder must provide a definition for the command property.
Builder will provide defaults for all other properties. The value is one or
more shell commands separated by newline characters. Its definition may
make use of the following property references:
{@}: the output file{<}: the first input file{^}: all input files
These properties correspond to Make's "automatic variables" $@, $<, and
$^, which are unavailable in Minion property definitions, since command
expansion happens prior to the rule processing phase. The value of {^} is
not identical to Make's $^, but it is more often what is relevant to rule
construction: it is a list of the files that correspond to target IDs in
{in}, and it does not include {deps}, {oo}, or implicit dependencies
(declared as prerequisited via {depsFile}), or other files that would not
normally appear as command line arguments. Also, duplicate entries are not
pruned, so in that respect it is more like $+ than $^.
For example, the Copy class inherits this definition of command:
_Copy.command = cp {<} {@}
The value of {in} is an ingredient list that describes
prerequisites of this instance. Builder defines {in} as $(_args), which
defaults to all unnamed instance arguments. Other classes may define {in}
differently.
Expansion of indirections and inference of
intermediate targets is performed to generate {inIDs}, which is a list of
target IDs. These are in turn translated to file names -- each instance is
replaced with its {out} property -- to generate {^}, which can be used
to construct the command line.
For convenience or readability, instead of listing all ingredients in
arguments, users can use a placeholder argument and then define in for
that instance, as in:
Alias(default).in = Exe(prog) Exe(prog).in = foo.c bar.c baz.c
Exe(prog) will create an executable equivalent to
Exe(foo.c,bar.c,baz.c), but with a different default name ("prog" vs
"foo"). Alternatively, one could define use an indirection, as in
Exe(@prog) and place all the ingredients in the variable prog.
An instance may have dependencies that are not specified by the user via
arguments or the in property, and instead are built into the definition of
the class. An example is when a build step uses a tool that is itself a
build product. We would prefer the dependency on the tool to be "baked
into" the class definition, so when using the class one needs to name only
the files that will be processed by the tool. This is akin to the notion of
captured values in lexically scoped programming languages. These built-in
dependencies are stored in the up property, which is an ingredient
list. The {ooIDs} property gives the target IDs that
result from expansion of {oo}, and {up^} contains the files
corresponding to those target IDs.
Order-only dependencies can be specified by defining the oo property (an
ingredient list). This type of dependency can be
appropriate when we know there is a dependency on the existence of a file,
but not necessarily on the content of the file.
Order-only dependencies are typically used to deal with implicit dependencies that are generated by the Makefile.
Implicit dependencies are files that are not specified on the command line,
but are read during execution of the build command. The classic example is
files included by C sources using #include. In Make, implicit dependencies
are trakced using the following strategy:
a) The command that compiles the C file also generates a dependency makefile expressing dependencies of the output file on the files that were included.
b) An -include directive is used to include the dependency makefile if
it has already been generated by a previous invocation of Make.
Minion's CC and CC++ clases implement this strategy. This ensures
accurate rebuilds of object files when included headers are modified.
Order-only dependencies enter the picture when some of the implicit dependencies are generated by our makefile. When we are building an object file for the first time, we do not know beforehand what include files will be included, so we need to ensure that any potential dependencies are generated before C files are compiled. The solution is to name all generated header files as order-only dependencies of all object files. It might look something like this:
CC.oo = IDLCompile@idlFiles
This property lists dependencies that are but are not listed on the command line and are not inherent in the class.
One use case for {deps} is when there are implicit dependencies that are
not automatically detected. Users can manually express these using
{deps}.
One example of this is ordering of unit tests. For example, we might want to execute test A before test B because A validates some of the code relied upon by test B. Running B before A might waste the user's time. Automatic detection of these kinds of dependencies might not be avialable.
By default, rule inference is performed on input files. Each class can
define its own inference rules by overriding {inferClasses}. This
consists of a list of entries of the form CLASS.EXT, each indicating that
CLASS should be applied to a input file ending in .EXT.
For example, inference allows a ".c" file to be supplied where a ".o" file
is expected. The instance CExe(hello.c) will infer the instance
CC(hello.c), because CExe.inferClasses contains CC.c.
By default, Builder treats the command used to build a file as a
dependency. That is, if any changes to makefiles or he environment result
in changes to {command}, previous build results will be considered
invalid and a rebuild will be forced on a subsequent invocation of make.
This {vvFile} property identifies a makefile that will store this
dependency information. Subclasses or user makefiles can set {vvFile} to
an empty value to disable this behavior, or the can override {vvValue} to
include information other than {command} that might affect target
validity.
The variable minionDebug can be used to turn on debug messages. Each
debug message has a value and an associated name, and the value of
minionDebug is a pattern to be matched with the name using filter (so
minionDebug=% will turn on all debug messages). Notably,
minionDebug=EVAL% will enable messages for all text that Minion feeds to
Make's $(eval ...) function.
The _? function makes it very easy to instrument your Make functions or
property definitions with debugging output. Replacing $(call func,...args...) with $(call _?,func,...args...) will retain the behavior
but write inputs and outputs to stdout.
By default, make clean will remove $(VOUTDIR). User makefiles can
change this behavior by defining in or command properties for
Alias(clean).
When goals not named clean are named along with clean on the command
line, it changes the handling of those other goals and clean. Instead of
deleting $(VOUTDIR), it will delete the targets of the listed goals and
all of their dependencies. This can be useful for removing build outputs
that fall outside of $(VOUTDIR), or for triggering more limited cleanup.
The Clean class is used by make clean. For example, make 'Clean(Alias(default))' is equivalent to make clean default. Instances
of Clean can be named as dependencies of Alias(clean) to ensure that
outputs outside of $(VOUTDIR) are cleaned. For example,
Alias(default).in = Copy(Exe(prog.c),dir:../deploy)
Alias(clean).in = Clean(Alias(default))
When help is named as the only goal, a message describing basic usage is
displayed. User makefiles can override this by defining in or command
properties for Alias(help). The message displayed by default is available
in $(_helpMessage).
When help is listed with other command line goals, it changes the handling
of those goals. Minion will output a description of the other goals, rather
than build them. Additionally, instance properties can be listed as goals
along with help. For example:
make help 'CC(foo.c).command'
will show the value of out, the property definitions that were inherited,
and the inheritance chain for CC.
The graph targets draws a textual representation of the dependency tree
for the default target. Use make 'Graph(INSTANCE)' to see the
dependencies of some other given instance.
Camel case is generally used for naming in Minion and recommended for user makefiles. Function, variable, and property names begin lowercase, while class names begin uppercase. All-caps names might conflict with environment variables or Make's built-in variable settings (e.g LINK.c, COMPILE.c, etc.). Variables defined by by minion.mk begin with "minion" or "_" to avoid unintentional conflicts with user makefiles, except for built-in classes and the following "unprefixed" names:
V, OUTDIR, VOUTDIR Control of build output
., get Core object system functions
\s \t \n \e \H ; [ ] [[ ]] Character constants
minionStart
minionEnd
minionDebug
minionCache
minionNoCache
We generally avoid single-letter global variables so that they can be used
as "local" variables (in Make foreach expressions). Underscore ("_")
may be used as a namespace delimiter for variables that are associated with
a class but not property definitions.
Minion defines a number of variables and functions for use by user Makefiles within recursive property definitions.
-
Character constants
$(\n): newline$(\t): tab$(\s): space$(\e): escape$(\H):#$;:,$[:($]:)$([[):{$(]]):}
-
$(call get,PROP,IDS)Evaluate property PROP for each member of IDS. IDS is a space-delimited list of IDs. The result is a space-delimited list of the corresponding values.
If a file name (not an instance name) is passed to
get, it will be treated as an instances of the_Fileclass. This defines the propertyout, which evaluates to the file name, and the propertiesruleandneeds, which are empty. -
$(call .,PROP,$0)Evaluate property PROP for the current instance. In property definitions,
$(call .,PROP,$0)is equivalent to{PROP}. In other variables and functions,{...}syntax is not supported, but.is available if they are called during the evaluation of a property definition. The second argument is used to construct a diagnostic message only in the event PROP is undefined. It should be the name of the function calling., which is available in Make as$0. -
$(call _shellQuote,STR)Quote STR as an argument for /bin/sh or /bin/bash.
-
$(call _printfEsc,STR)Escape STR for inclusion in a
printfcommand line argument. -
$(call _printf,STR)Return a shell command that writes STR to stdout.
-
$(call _eq,A,B)Return "1" if A and B are equal, "" otherwise.
-
$(call _once,VAR)Return the value of VAR, evaluating it at most once.
-
$(_self)Return the name of the current instance.
-
$(_class)Return the class of the current instance.
-
$(_argText)Return the portion of the current instance name that describes the argument list.
For example, in
Class(a,b,x:1,x:2),$(_argText)returns "a,b,x:1,x:2". -
$(_args)Return all unnamed arguments for the current instance.
For example, in
Class(a,b,x:1,x:2),$(_args)returns "a b". -
$(_arg1)Same as
$(word 1,$(_args)). -
$(call _namedArgs,NAME)Return all arguments associated with NAME for the current instance.
For example, in
Class(a,b,x:1,x:2),$(call _namedArgs,x)returns "1 2". -
$(call _namedArg1,NAME)Same as
$(word 1,$(call _namedArgs,NAME)). -
$(call _relpath,FROM,TO)Construct a path that can be used to refer to file
TOfrom fileFROM. IfTOis an absolute path, it is returned as the result. Otherwise, bothFROMmust be a relative path and the result will be a relative path. -
$(call _traverse,CF,CC,ROOTS)Traverse a graph, starting at root nodes ROOTS, returning a list of nodes ordered such that each parent precedes all its children.
CF= name of a function to get children of a nodeCC= a context value to pass toCF$(call CF,CC,node) -> children
The following BNF describes target names:
TargetID := Name | Instance
Instance := Class '(' ArgList ')'
Class := ClassChar+
ArgList := ( Arg ( ',' Arg )* )?
Arg := ( Name `:` )? Value
Value := ( ArgList | ValueChar )+
Name := NameChar+
Property := PropChar+
These definitions rely on the following character classes:
ClassChar: A-Z a-z 0-9 _ - + / ^ ~ { }
NameChar: A-Z a-z 0-9 _ - + / ^ ~ { } .
PropChar: A-Z a-z 0-9 _ - + / ^ ~ @ < >
ValueChar: A-Z a-z 0-9 _ - + / ^ ~ { } . @ < > ! *
Note that an argument list contains zero or more arguments, and each
argument must contain at least one character. Each argument may contain
other instances embedded within it, which means it may contain ( and ),
characters, but only in balanced pairs. An argument may also contain ,
and :, but only within nested parentheses.
In general, instances will contain special shell characters, so they may have to be quoted when being passed on the command line.
Characters that are "special" in POSIX shells or GNU Make can complicate things so we generally assume they are not present in input and output file names. Quoting every file name for the shell is tedious, and the additional layer of quoting in Make can make things confusing. Make quoting can vary by context and sometimes even be impossible. That makes for a fairly large set of undesirable characters, including whitespace characters and the following:
Make specials: ~ : [ ] * ? # $ % ; \ =
Bash specials: ~ ! ( ) < > [ ] * ? # $ % ; \ | & ` ' " { }
Minion instances and indirections may contain a number of these special characters because instances generally do not appear as targets and prerequisites in Make rules, and they are not passed to the shell as file names. When Minion constructs output file paths from instance names, but it is careful to use shell-safe characters to escape any unsafe characters in instance names. The following shell-unsafe characters may appear in instances or indirections:
( ) < > { } : ! * ~ < >