The last article
in this series introduced the Jam build tool from a high-level
and described the Jamfile-engine that has been created to replace the Be
makefile-engine. This second of three articles will show how the Jamfile-engine
is implemented, and while doing so will provide more advanced information
about Jam.
At this point it should be mentioned that any readers who have not already downloaded the
Jamfile-engine may want to before continuing:
Jamfile-engine.zip.
If you downloaded this previously, you may want to get the latest version, in
which more comments have been added and some bug fixes made as well.
If you open up the Jamfile-engine text file, you will notice it begins with a
fairly standard comment section at the beginning:
## OpenBeOS Generic Jamfile Engine v1.0.1
## Does all the hard work for the Generic Jamfile
## which simply defines the project parameters.
## Most of the real work is done in the Jambase
## embedded into the jam executable.
##
## Inspired by the Be Makefile Engine
##
## Supports Generic Jamfile v1.0
##
## Copyright (c) 2002, 2003 Ryan Leavengood
## Released under the Terms of the MIT License, see
## http://www.opensource.org/licenses/mit-license.html
|
From this you will realize that comments in Jam begin with a hash symbol
(#) and they continue until the end of the line. In this case two
hashes are used to make this comment stand out more and to indicate that it
describes the file as a whole and not just some implementation detail.
After the initial comment the real code begins. In the first part of the
Jamfile-engine, several utility rules are defined that can be used and re-used
later in the file. The first few rules are simple:
# AddResources <Application Name> : <Resource Files> ;
# Adds the given resources to the given application.
rule AddResources
{
Depends $(<) : $(>) ;
}
actions AddResources
{
$(XRES) -o "$(<)" $(>)
}
# MimeSet <Application Name>;
# Sets the mime type of the given application to be an application.
actions MimeSet
{
$(MIMESET) -f "$(<)"
}
|
Jam Rules: Procedure and Actions
After looking at the template above, many readers may be confused. Why is AddResources
defined twice, once using the word rule and then with the word actions?
The reason is that Jam rules are created in two parts:
- The procedure (defined using the keyword rule):
A set of Jam language statements that are run when the rule is invoked and which
usually set up variables that will be used in the actions.
- The actions:
Shell commands that get run when a target needs updating
(and make use of the variables configured in the procedure.)
In the above case of AddResources, the procedure sets up a dependency between
the first parameter and the second parameter:
$(<)
depends on $(>)
meaning that when $(>) changes, $(<) must be updated.
Rule Parameters
The variables used here, $(<) and $(>)
are aliases for $(1) and $(2), which are the first and second parameters passed
to the rule, respectively. Up to 9 parameters, $(1) - $(9), may be passed to
rules. While all can be used in the procedure, only the first two can be used in the actions.
In the case of the actions for AddResources, the command
xres is called using the variable $(XRES), with the output (-o) being the
quoted value of $(<) (the quotes allow for spaces in the application name),
and with the input resource files being the value of $(>).
When defining updating rules, $(1) or $(<) is generally considered
to be the target (what will be created by the rule) and $(2) or $(>)
is the source(s) (what the target will be created from.)
Though mentioned briefly, it should be evident
that $(XRES) is just a reference to a global variable defined elsewhere in the
Jamfile-engine (which just has the name of the xres command.) This shows that
any variables within scope can be used in the actions for a rule.
After AddResources, the MimeSet rule is created, but in this case there is no
procedure because there are no variables needed or dependencies for this rule.
In fact, this rule only has one parameter: the name of the application that the
mimeset command should be run against.
After the above rules a more complicated rule is defined:
# ProcessLibs <List of Library Names> ;
# Prepends -l to any library names that aren't _APP_ or _KERNEL_ or
# that don't have .a or .so file extensions. The result will be given
# to the linker so that it links to the right libraries.
rule ProcessLibs
{
local result ;
for i in $(1)
{
if ( ( $(i) in _APP_ _KERNEL_ ) || ( $(i:S) in .so .a ) )
{
result += $(i) ;
}
else
{
result += -l$(i) ;
}
}
return $(result) ;
}
|
There is a lot going on in the rule, but the first thing that should be
mentioned is that, in contrast to the MimeSet rule, this rule only has a
procedure but no associated actions. In this case the rule is just a string
processing rule that iterates over a list of library names and adds a -l prefix
to those that aren't _APP_ or _KERNEL_ or that don't end in .so or
.a. To implement this functionality, a lot of the built-in Jam language is used.
Basic Syntax
The first thing done in this rule is the declaration a local variable called
result. The local keyword provides for dynamic scoping (as in C or
C++): if another variable named result exists when this rule is called, that
old value will be saved, the new value will have nothing to do with the old one,
and then after the rule is finished, the new value is discarded and the old one restored.
After the declaration of the local result variable, there is another new piece
of syntax: the for loop. In this case the for keyword is used to
iterate over the items in the list $(1) (the first and only parameter to this
rule), setting the variable i to each item in turn.
After the for loop statement is its associated block, which contains the statements
that should be run on each loop iteration. This block contains a single
if..else statement, which is very similar to the if..else construct in C.
The condition that this if statement checks is fairly complex, but when broken
down it is simple: in the first set of parenthesis, the in keyword
is used to see if the current value of $(i) is in the list [_APP_ _KERNEL_].
List Handling
The in keyword, as can probably be guessed, returns true if the
first parameter is a subset of the second. Note the use of the term subset:
both "parameters" to the keyword in are lists, and the
result is only true if every element in the first list is in the second list.
(In this case only a single-element list is used for the first parameter.)
Also note that literal lists in the Jam language
(such as _APP__KERNEL_ in this case)
do not need any delimiters (where C syntax would require curly brackets ({ and }),
commas (,), and double quotes (") to create a list of strings.)
This is one of the advantages of the syntax of Jam (and one of the reasons for all the whitespace.)
This really begins to illustrate that at its heart Jam is just a list processing language
(hello LISP fans!) In fact, changing the Jam syntax to be like LISP probably wouldn't be too hard,
but that shall be left as an exercise for the reader.
Operators
After the first condition in the if is an or (||) disjunction,
which works just like the C equivalent (the result is only false if both sides
are false.) The second side of the or is a condition similar to the first: it
checks to see if something is in a list. In this case, though, one of Jam's
variable modifiers, :S, is used.
What :S does is return the last filename suffix of the given variable --
in other words -- the file name extension. If the file name extension is in the list
".so.a", then this statement will be true.
Whenever the if condition (which you will note has whitespace
separating all tokens) returns true, the += expression is used to
add the unmodified value of $(i) to the result list. The operator +=
works the same as in C: the value of result is set to the old value of result
plus the value of $(i). But since Jam is a list-oriented language, this
addition is not mathematical, but is a list addition: the new value is added as a
new element to the end of the list. In fact, Jam does not have any syntax for
doing math at all.
In the case that the if condition is false, the block under the else clause
will be run. This block adds the value of $(i) with the prefix -l
to the result list.
Variable Expansion
Though it doesn't really come into play here, now is a good time to
mention how Jam "variable expansion" works. When you concatenate
several variables or a variable with one or more literals, the result is a list
that is a product of the components of the variables being combined.
For example, in the simple case of the -l$(i) statement above, the result will be
the value of $(i) with -l prepended to it.
Since the for loop insures that $(i)
is a single element list, the result is simple, but if $(i) had more than one
element (such as [be media midi]) the result would be:
[-lbe -lmedia -lmidi].
Given that value of $(i), the result of $(i)$(i) would be:
[bebe bemedia bemidi mediabe mediamedia mediamidi midibe midimedia midimidi].
Try saying that three times fast.
Two final notes regarding variable expansion: if a list contains the null
string (""), the result of expansion is still a product, but only of
non-null elements. For example, if a variable $(x) was the list [A ""]
and $(y) was ["" 1], the expansion of *$(x)$(y)* would
be [*A* *A1* ** *1*]. The other note is that any expansion that uses an
undefined variable results in an empty list.
After ProcessLibs is a similar string-processing rule:
# MkObjectDirs <List of Source Files> ;
# Makes the necessary sub-directories in the object target directory based
# on the sub-directories used for the source files.
rule MkObjectDirs
{
local dir ;
for i in $(1)
{
dir = [ FDirName $(LOCATE_TARGET) $(i:D) ] ;
Depends $(i:S=$(SUFOBJ)) : $(dir) ;
MkDir $(dir) ;
}
}
|
This rule is used to create sub-directories for the object files that mirror
the directory structure of the project source files. Similar to ProcessLibs, a
local variable is declared, and a for loop is used, which in this case iterates
over source file names. In the loop body the variable dir is set to be the
result of a call to the built-in FDirName rule, which takes the
items in the list passed to it and concatenates them with directory separators
in between each item.
The parameter to FDirName is a list containing the target
directory into which everything is built (defined later in the Jamfile-engine)
and the directory of the source file (that is what the :D
variable-modifier returns.) Then a dependency is set up between the object file for the
source file and the created directory name.
The :S= modifier used with $(i) replaces the file extension
of the variable with the given suffix, in this case the variable $(SUFOBJ),
which is .o on BeOS. By creating this dependency between the object file and
the directory it is created in, we can ensure the directory is properly created
before the object file.
After the dependency is set up, the actual MkDir rule is called with the given directory.
This rule creates the given directory if it doesn't already exist,
including any needed parent directories, like the GNU mkdir command with the -p option.
After the MkObjectDirs rule are a few more simple rules:
# RmApp <Pseudotarget Name> : <Application Name> ;
# Removes the given application file
# when the given pseudotarget is specified.
rule RmApp
{
Depends $(<) : $(>) ;
}
actions RmApp
{
rm -rf "$(>)"
}
# RunApp <Pseudotarget Name> : <Application Name> ;
# Runs the given application in the background
# when the given pseudotarget is specified.
rule RunApp
{
Depends $(<) : $(>) ;
}
actions RunApp
{
"$(>)" &
}
|
Pseudotargets
These rules look very similar to AddResources, and their function in the
Jamfile-engine should be obvious. One thing that may not be obvious is what the
parameters are, particularly the first, pseudotarget name.
A pseudotarget is a
name that defines a target which can be specified on the command-line to jam,
but that is not really a file system target that can be created. This
distinction is specified using certain Jam rules, which will be described
later. Suffice it to say that when the pseudotargets passed into the above rules
are specified on the jam command-line, the actions for those rules will be run.
For example, if RunApp test : $(APP) ; was specified later in the Jamfile-
engine (which it is), running "jam test" on the command-line would run the
application in the background (after creating it if need be.)
Now for the next set of rules:
# InstallDriver1 <Pseudotarget Name> : <Driver File> ;
# Installs the given driver in the correct location
# when the given pseudotarget is specified.
rule InstallDriver1
{
Depends $(<) : $(>) ;
USER_BIN_PATH = /boot/home/config/add-ons/kernel/drivers/bin ;
USER_DEV_PATH = /boot/home/config/add-ons/kernel/drivers/dev ;
}
actions InstallDriver1
{
copyattr --data "$(>)" "$(USER_BIN_PATH)/$(>:B)"
mkdir -p $(USER_DEV_PATH)/$(DRIVER_PATH)
ln -sf "$(USER_BIN_PATH)/$(>:B)" "$(USER_DEV_PATH)/$(DRIVER_PATH)/$(>:B)"
}
# InstallDriver <Pseudotarget Name> : <Driver File> ;
# Installs the given driver in the correct location
# when the given pseudotarget is specified
# (after making sure that this is actually a driver)
rule InstallDriver
{
if ( $(TYPE) = DRIVER )
{
InstallDriver1 $(<) : $(>) ;
}
}
|
These rules, as the names imply, are used for installing drivers.
The commands in the actions are taken almost verbatim from the Be
makefile-engine (as they say: if it ain't broke, don't fix it.)
The reason that there is an InstallDriver1 and InstallDriver rule is due to the
need to check the $(TYPE) variable before actually performing the action.
If this isn't actually a driver, it does not make sense to try to install it
in the driver directories. So the rule that should be called by users of this
rule-set is InstallDriver, which will do the right thing based on the given
type of BeOS project. This style of naming (appending 1 to the rule name for
the worker rule) is used in the Jambase file, which is why it is used in the
Jamfile-engine as well.
Finally, the last two rules defined are:
# Link <Application Name> : <List of Object Files> ;
# Replaces the actions for the default Jam Link rule with one that
# handles spaces in application names.
actions Link bind NEEDLIBS
{
$(LINK) $(LINKFLAGS) -o "$(<)" $(UNDEFS) $(>) $(NEEDLIBS) $(LINKLIBS)
}
# BeMain <Application Name> : <List of Source Files> ;
# This is the main rule that builds the project.
rule BeMain
{
MkObjectDirs $(>) ;
if ( $(TYPE) = STATIC )
{
Library $(<) : $(>) ;
}
else
{
Main $(<) : $(>) ;
}
if ( $(RSRCS) )
{
AddResources $(<) : $(RSRCS) ;
}
MimeSet $(<) ;
}
|
As the comment illustrates, the first "rule" is really just a
re-definition of the actions for the built-in Link rule. The actions from the
Link rule in the Jambase have been changed by adding quotes around the
application name $(<).
This shows that any of the built-in Jam rules can be
modified freely in any Jamfiles you create. In fact, the built-in Jambase can
be completely replaced by specifying the -f option to Jam on the command-line
(though this may not be too useful since most of Jam's usefulness comes
from the rules defined in the built-in Jambase.)
The Main Rule
The final rule, BeMain, is the real work-horse in the Jamfile-engine: this is really what
builds the project. Because of all the work done in the rest of the Jamfile-
engine, however, this really is quite simple.
First, the MkObjectDirs rule is called with the project source files,
which will create the needed object directories as described above.
Then an if statement is used to determine if the type of project is a static library.
If it is, the built-in Jam rule Library is called, which compiles the given source files
and then archives them into a static library of the given name.
Otherwise the built-in Jam rule Main is called, which compiles the given source files
and then links them as the given application name.
At this point it should be mentioned that both Library and Main make use of the
Objects rule, which uses the Object rule, which is smart in that it looks at the
file extension of the given source file and then calls the appropriate rule to
compile it. Files that end with .cpp or .cc or .C are compiled with the C++
rule, while .c files are compiled with the Cc rule, and .l files are compiled
with Lex, etc. Thus sources can be a mixed list of any Jam-supported files and
they will all be compiled correctly and linked into one application or library.
(There is also a fairly easy way to add support for compiling other types of
files, such as Pascal or ASM files, for instance, which will be explained in the next article.)
The rest of the Jamfile-engine is mostly definitions of various variables, and
only a few parts of it use any Jam concepts that have not already been explained.
Those are the only parts that will be explained here, starting with this:
# Set the directory where object files and binaries will be created.
# The pre-defined Jam variable OSPLAT will indicate what platform we
# are on (X86 vs PPC, etc.)
LOCATE_TARGET = obj.$(OSPLAT) ;
|
As described briefly in the explanation of the MkObjectDirs rule, this variable
defines where targets should be located, i.e. where they are created. In this
case it is set to be "obj." with the platform appended. The variable $(OSPLAT)
is one of the few variables actually compiled into Jam (and therefore not set
in Jambase), and it is an all capital description of the CPU type (such as X86,
PPC or SPARC.)
Jam runs on many platforms, and a properly written Jamfile
should be able to work on many of them, unmodified. For example, the
Jamfile-engine should theoretically run on both x86 and PowerPC versions of BeOS
as it is today (the term theoretically is used here since no one has tested
the Jamfile-engine on a PowerPC machine yet.)
Also, though it is used in MkObjectsDir, LOCATE_TARGET is actually a variable from
the Jambase that is used extensively in all the built-in Jam rules. Of course
one thing that a helpful Jamfile-engine user discovered is that despite the
setting of this variable, source files that exist in subdirectories are
created in similar subdirectories under LOCATE_TARGET, not directly in it.
This is why the MkObjectDirs rule was created, because otherwise the compiler
complains when it tries to put object files into non-existent directories.
After the LOCATE_TARGET definition comes a few more definitions:
# Set some defaults
if ( ! $(NAME) )
{
ECHO "No NAME defined!" ;
NAME = NameThisApp ;
}
if ( ! $(TYPE) )
{
ECHO "No TYPE defined...defaulting to APP" ;
TYPE = APP ;
}
if ( ! $(SRCS) )
{
ECHO "NO SRCS defined...defaulting to *.cpp in current directory" ;
SRCS = [ GLOB . : *.cpp ] ;
}
if ( ! $(DRIVER_PATH) )
{
DRIVER_PATH = misc ;
}
|
These are all probably pretty obvious. The few interesting points are:
- ( ! $(SOME_VAR) ) will be true for an undefined or empty variable.
- The GLOB rule returns any files that match the given criteria in the given
directory. The pattern rules will be explained more fully in the next article.
- The syntax of the square brackets ([]) around a rule invocation expands the
results of that rule into a list which can then be assigned to a variable.
Following the definitions above is a large section that defines variables based on
the CPU type. Again, most of this is based on the Be makefile-engine, with a few
tweaks because of Jam's more capable syntax. One of those tweaks is the use of
the Jam switch statement instead of a series of if statements.
Since that is the only new piece of Jam syntax, that is all that will be described
from this section of the Jamfile-engine:
switch $(OPTIMIZE)
{
case FULL : OPTIMIZER = -O3 ;
case SOME : OPTIMIZER = -O1 ;
case NONE : OPTIMIZER = -O0 ;
# Default to FULL
case * : OPTIMIZER = -O3 ;
}
|
The Jam switch statement probably looks familiar to C or C++ programmers.
Overall it works the same, but has a few nicer features. For instance, it does
not do matching based on simple numeric equivalence, but on string matching.
In the above case, if $(OPTIMIZE) is set to any of the explicitly listed cases, the
matching statement gets run. There is no need for a break statement as in C,
only the matching statement gets run. There also is no default branch, though
the same functionality can be had by using * as the matching criteria, as done
above. In fact, the GLOB statement described above actually uses the matching
syntax from the switch statement (switch was implemented before GLOB), and again
the matching syntax will be more fully described in the next article.
There is one more new Jam rule that is part of this processor-dependent section of
the Jamfile-engine, right at the end:
else
{
EXIT "Your platform is unsupported" ;
}
|
The EXIT rule prints out the given statement and then halts the execution of Jam.
This is best used in cases of serious error, as done above when the
$(OSPLAT) is not X86 or PPC.
The next series of statements in the Jamfile-engine are platform-independent
settings. The only thing really new here is the definition of the various
pseudotargets used by the Jamfile-engine:
# Set up the driverinstall target...this makes it easy to install drivers
# for testing
Always driverinstall ;
NotFile driverinstall ;
InstallDriver driverinstall : $(NAME) ;
# Set up the rmapp target...this removes only the application
Always rmapp ;
NotFile rmapp ;
RmApp rmapp : $(NAME) ;
# Set up the test target...this runs the application in the background
#Always test ;
NotFile test ;
RunApp test : $(NAME) ;
|
As mentioned above when describing the RmApp, RunApp, and InstallDriver rules, a
pseudotarget is defined and then passed to each rule to act as the target that
can be passed to Jam on the command-line to perform the given action. In the
case of RunApp, "test" is used, which is set up as a pseudotarget by
the calls to the built-in Jam rules Always and NotFile.
The Always rule marks a target so that it is always updated, even if it exists.
This rule can be used with real file-based targets as well as pseudotargets,
though in general it is most useful with pseudotargets. If this rule is not used,
the pseudotarget will only work the first time (generally when the target it
depends on is first created.)
The NotFile rule is the rule that actually makes a target a pseudotarget,
by informing Jam that it isn't really a file, so it cannot be built.
When combined with Always, this allows convenient targets to be specified when
calling Jam, so that for instance "jam rmapp" will remove an application created
by the Jamfile-engine, "jam test" will run that application, and "jam driverinstall"
will install a driver in the correct place.
Finally, the last statement in the Jamfile-engine is a call to the
previously-described BeMain rule:
##-------------------------------------------------------------------
## OK, let's build
##-------------------------------------------------------------------
BeMain $(NAME) : $(SRCS) ;
|
So by now you should understand quite a bit more about Jam and also how the
mysterious Jamfile-engine works. As you can see, the Jamfile-engine really
isn't that complicated, and overall is quite a bit simpler than the Be
makefile-engine (though with all my comments they are about the same length.)
Also it should be evident that though Jam is probably more complicated than
make, the extra functionality is very useful, and the platform-independence of
most Jamfiles alleviates the need for complicated configure scripts.
The next article in this series will be a Jam cookbook that will describe
how various build problems and challenges can be solved with Jam. The author already has
a few ideas for some "recipes", but I would ask that anyone reading this who has
other challenges, please e-mail them to me. Especially things that you think
"cannot be done with Jam." You may be right, but I'll try my best to show how it can be done.
I'll end with a small anecdote: I frequently see people working on OpenBeOS
complain that "I would write a Jamfile, but I don't know how." My hope is that
by the end of this series of articles, no one has to say that again.
