Documentation

Build Environment Directory

All files making up the build environment are stored in a single directory somewhere in a user's home directory, e.g., ~/WhyTools. To allow project makefiles to find the build environment, a user has to set an environment variable BUILDROOT to point to the build environment's root directory. Using tcsh and the example root directory name from above, this would best be done by including the line setenv BUILDROOT ${HOME}/WhyTools into on of a user's login scripts.

Build Environment Files

The build environment consists of the following files, all stored in the build environment root directory:

SystemDefinitions: A file defining the build machine's environment. Parameters include the used compiler, where to find include and library files, etc.
Packages: This file defines the "software packages" that are provided on the build machine. A software package is defined as a set of include paths that need to be sent to the compiler when creating object files, and a set of library file directories and library files that need to be sent to the linker when creating executable files or dynamic shared libraries.
BasicMakefile: This file contains all the rules to generate compiler flags from the user-defined makefile parameters, the default rules to compile various flavours of C/C++ source files into object files, and rules to manage the automatic dependency generation system. This file contains no parameters that need to be tweaked by a user (unless when dealing with a seriously strange build machine).
makefile: This is a template makefile to be copied into a new, empty project directory. It provides placeholders to set the packages used by a project, compiler flags, the list of all final targets, extra operations to be performed on a "make clean," extra rules or parameters for sources that need them, and finally the rules to link a project's dynamic shared libraries and executable files.

User-definable Parameters

All project-independent user-definable parameters (barring exceptionally weird build machine setups) are contained in the files SystemDefinitions and Packages. The makefile contained in each project directory has an additional set of project-dependent parameters that can be specified either project-globally or on a per-target basis.

Parameters in `SystemDefinitions`

INCLUDEEXT: The subdirectory under a package's root directory that contains include files.
LIBEXT: The subdirectory under a package's root directory that contains library files.
BINEXT: The subdirectory under a package's root directory that contains executable files.
DEPDIR: The base target directory for automatically generated dependency files.
OBJDIRBASE: The base target directory for object files. The build environment will create subdirectories beneath the base directory for each specified combination of debug level, optimization level and compiler type to allow coexistence of differently built versions of a project in the same project directory.
EXEDIR: The base target directory for executable files.
GNUC_BASEDIR: The base directory for the GNU compiler collection, i.e., the directory that contains the versions of gcc and g++ to be used.
BASECCOMP: The C++ compiler that is actually referred to by the CCOMP variable (see below). This variable is mostly used to point to the back-end compiler if CCOMP refers to a front-end such as Visual G++.
CCOMP: The C++ compiler used to create object files from C++ source files. If this variable refers to a compiler front-end such as Visual G++, the BASECCOMP variable should be used to point to the real compiler used.
PLAINCCOMP: The C compiler used to create object files form C source files.
DEPFILETEMPLATE: A make variable template to construct the name of a generated dependency file based on the patterns of an implicit/explicit compilation rule. This template depends on the exact compiler version being used; the default content has two different rules for pre-3.0 g++ and post-3.0 g++, which are selected automatically based on gcc's version number.
CDSOFLAGS: A list of flags that need to be passed to the compiler to create position-independent code that can be linked into a dynamic shared object.
DSOLINKFLAGS: A list of flags that need to be passed to the linker to create a dynamic shared object.

Setting up the `Packages` File

The Packages file contains a list of software packages that are commonly referenced in a user's projects, e.g., OpenGL, X11, Motif, Fltk, MPI, Qt, etc. Referring to a software package from a project makefile influences the compiler's/linker's operation in three ways:

A package can add a list of include directories to the compiler's command line (each passed in via the -I command line switch).
A package can add a list of compiler flags to the compiler's command line.
A package can add a list of library search directories to the linker's command line (each passed in via the -L command line switch).
A package can add a list of libraries to the linker's command line (each passed in via the -l command line switch).

Furthermore, the Packages file supports listing dependencies between packages, e.g., the Motif package will depend on the X11 package. If a project makefile references a package, references to all packages it depends on will automatically be included. Currently, package dependencies are expanded recursively up to nine levels deep.

Package Definition Format

Each software package listed in the Packages file must have a unique name. The properties of a package are defined as a set of variables whose names are generated by appending suffixes to the package's name. The required suffixed variables for each package are:

<PACKAGE_NAME>_BASEDIR: The root directory containing the package's include, library and binary files (if any). This variable is not really required, but it is a convenient shortcut to specify the other variables in terms of it.
<PACKAGE_NAME>_DEPENDS: A space-separated list of names of other packages the package depends on. This list is recursively expanded when a package is referenced by a project, and definitions for required packages are listed after the package's definitions.
This variable must exist, but it can be empty if a package depends on no other packages.
<PACKAGE_NAME>_INCLUDE: A list of -I command line switches passed to the compiler when creating object files.
<PACKAGE_NAME>_CFLAGS: A list of additional command line switches passed to the compiler when creating object files.
<PACKAGE_NAME>_LIBDIR: A list of -L command line switches passed to the linker when creating dynamic shared libraries or executables.
<PACKAGE_NAME>_LIBS: A list of -l command line switches passed to the linker when creating dynamic shared libraries or executables.

Example Package Definition

The following excerpt from a Packages file defines the software packages X11, GL, and MOTIF (where MOTIF depends on X11):

# The basic X11 package without any toolkits/widget sets
X11_BASEDIR = /usr/X11R6
X11_DEPENDS = 
X11_INCLUDE = -I$(X11_BASEDIR)/$(INCLUDEEXT)
X11_LIBDIR  = -L$(X11_BASEDIR)/$(LIBEXT)
X11_LIBS    = -lX11

# The basic OpenGL package without window system bindings and add-on packages
GL_BASEDIR = /usr
GL_DEPENDS = 
GL_INCLUDE = -I$(GL_BASEDIR)/$(INCLUDEEXT)
GL_LIBDIR  = -L$(GL_BASEDIR)/$(LIBEXT)
GL_LIBS    = -lGL

# The Xt toolkit with the Motif widget set
MOTIF_BASEDIR = /usr/X11R6
MOTIF_DEPENDS = X11
MOTIF_INCLUDE = -I$(MOTIF_BASEDIR)/$(INCLUDEEXT)
MOTIF_LIBDIR  = -L$(MOTIF_BASEDIR)/$(LIBEXT)
MOTIF_LIBS    = -lXm -lGLw -lXt

In this example, a project defining PACKAGES = GL MOTIF would create the following three lists:

Include directories: -I/usr/include -I/usr/X11R6/include -I/usr/X11R6/include
Library search directories: -L/usr/lib -L/usr/X11R6/lib -L/usr/X11R6/lib
Linked libraries: -lGL -lXm -lGLw -lXt -lX11

Due to recursive dependency resolution, paths and libraries might be listed more than once. Listing libraries more than once is sometimes required for correctly linking static libraries, and for correctly linking dynamic libraries on older systems. Listing paths multiple times does not affect compilation. A future version of the build environment might cull redundant entries from the path lists.

Parameters in `makefile`

The makefile inside a project directory offers parameters to control the build of the respective project. If defined at the top of the makefile, the parameters apply to all targets listed in the makefile; all but the first two parameters can be overridden or amended independently for each target if necessary.

DEFAULTOPTLEVEL: The optimization level to be used when no OPTLEVEL= parameter is given on make's command line. The optimization level is passed to the compiler via the -O<OPTLEVEL> command line switch.
DEFAULTDEBUGLEVEL: The debugging level to be used when no DEBUGLEVEL= parameter is given on make's command line. The debugging level is passed to the compiler via the -g<DEBUGLEVEL> command line switch.
CFLAGS: The contents of this variable will be appended to the compiler's command line when compiling C/C++ source files into object files.
LINKFLAGS: The contents of this variable will be prepended to the linker's command line when linking a set of object files into an executable.
EXTRACINCLUDEFLAGS: The contents of this variable will be appended to the list of include directories passed to the C/C++ compiler that is generated from the contents of the PACKAGES variable (see below).
EXTRALINKDIRFLAGS: The contents of this variable will be appended to the list of library search directories passed to the linker that is generated from the contents of the PACKAGES variable (see below).
EXTRALINKLIBFLAGS: The contents of this variable will be appended to the list of libraries passed to the linker that is generated from the contents of the PACKAGES variable (see below).
PACKAGES: This variable contains a list of names of packages that are required to build a project's object files, dynamic shared libraries and executable files. The makefile generates a list of include directories to be passed to the C/C++ compiler when generating object files, and a list of library search directories and a list of libraries to be passed to the linker when generating dynamic shared libraries or executables. To append additional entries to either one of these three lists, use the EXTRACINCLUDEFLAGS, EXTRALINKLIBDIRS, and EXTRALINKLIBFLAGS variables (see above).
Do not modify the CINCLUDEFLAGS, LINKDIRFLAGS or LINKLIBFLAGS variables internally defined in the BasicMakefile - overriding those, or adding to them, will interfere with processing of the PACKAGES list.

`makefile` Targets

Since the build environment does not require explicite listing of source file dependencies, project makefiles are typically quite small beyond the boilerplate provided by the template makefile in the build environment's root directory. Apart from setting compiler and linker flags and listing required packages, a makefile for a project typically only contains dependencies for executable targets.

The format for specifying an executable target is quite simple. For example, let us consider an executable target Foo (created in ./<EXEDIR>) that is the result of linking together Foo.o, Bar.o, Baz.o and requires the modules GL and MOTIF:

$(EXEDIR)/Foo: PACKAGES += GL MOTIF
$(EXEDIR)/Foo: $(OBJDIR)/Foo.o $(OBJDIR)/Bar.o $(OBJDIR)/Baz.o

The first line tells make to append the packages GL and MOTIF to the project-global list of packages. The second line defines the object files that are to be linked into Foo. It is not necessary to provide targets for the required object files, since the build environment has implicite rules to compile them from their respective source files. Furthermore, no explicit link command is required since the build environment takes care of creating the appropriate link line.

To allow just typing make Foo instead of make ./<EXEDIR>/Foo, it is common to add a phony target for each executable target:

.PHONY: Foo
Foo: $(EXEDIR)/Foo

Of course, it is even easier to add all executable targets to the ALL variable at the beginning of the makefile.

If a project directory contains nothing but the source files that are to be linked together into a single executable, the makefile becomes even more schematic. Assuming all source files have the extension .cpp, it would be:

$(EXEDIR)/Main: $(patsubst %.cpp,$(OBJDIR)/%.o,$(shell find --follow . -name "*.cpp"))

This line tells the build environment to link all source files under the current object file directory, including those generated from source files in subdirectories underneath the project's root directory, into the single executable Main. The fancy use of the patsubst make function tells make to use the find utility to create a list of all files with extension .cpp underneath the project directory, and then to replace all their names with the names of their corresponding object files. If several different source file extensions were used in a single project, their patsubst expressions could be concatenated:

$(EXEDIR)/Main: $(patsubst %.c,$(OBJDIR)/%.o,$(shell find --follow . -name "*.c")) \
                $(patsubst %.cc,$(OBJDIR)/%.o,$(shell find --follow . -name "*.cc")) \
                $(patsubst %.cpp,$(OBJDIR)/%.o,$(shell find --follow . -name "*.cpp")) \
                $(patsubst %.c++,$(OBJDIR)/%.o,$(shell find --follow . -name "*.c++"))

If new source files are added to the project directory, they will be compiled automatically on the next make, and they will also be automatically added to the dependency graph maintained by the build environment.

Project Directory Layout

The build environment creates a set of subdirectory hierarchies inside a project's root directory. All directories are automatically created when they are first needed, and the entire set of subdirectory hierarchies is removed when make squeakyclean is issued.

Subdirectory Hierarchy

./<DEPDIR>

This is the base directory for automatically generated dependency files. If a project directory contains subdirectories with source files in them, those source files' dependency files will be created in the same directory structure underneath the base directory. For example, if a project contains a source file ./Geometry/Point.cpp, the corresponding dependency file will be ./<DEPDIR>/Geometry/Point.d.

./<OBJDIRBASE>

This directory contains a set of subdirectories for each previously built combination of compiler version, optimization level and debugging level. The names of those subdirectories are created as <COMPILERTYPE>.g<DEBUGLEVEL>.O<OPTLEVEL>. If static linking is requested by passing a command line switch STATIC_LINK=true into make, an extra .Static is appended to the generated subdirectory name.

This switch-dependent generation of object file target directories allows for the coexistence of differently built versions of a project under the same project directory. In other words, a project might by default be built with full optimization and no debugging info, but a version with no optimization and full debugging info might be kept around to track down bugs. To switch between versions, the desired optimization level and debugging level can be passed into make as command line parameters DEBUGLEVEL=<DEBUGLEVEL> OPTLEVEL=<OPTLEVEL>. This means that when switching build versions only sources that have changed since the last build of the new version have to be recompiled, and generally no make clean is necessary. It might be necessary to remove the executable whose version is to be switched if the version to be switched to is up-to-date. Fortunately, this only incurs linking overhead.

<COMPILERTYPE>.g<DEBUGLEVEL>.O<OPTLEVEL>: Each of these base object file subdirectories contains object files generated using the compiler settings given in its name. If a project directory contains subdirectories with source files in them, those source files' object files will be created in the same directory structure underneath the base object file subdirectory. For example, if a project contains a source file ./Geometry/Point.cpp, the corresponding object file will be ./<OBJDIRBASE>/<COMPILERTYPE>.g<DEBUGLEVEL>.O<OPTLEVEL>/Geometry/Point.o.
A make clean will only remove object files under the base subdirectory corresponding to the selected compiler type, optimization level and debugging level. To specify a set of object files to be removed, one can issue make DEBUGLEVEL=<DEBUGLEVEL> OPTLEVEL=<OPTLEVEL> clean. A make squeakyclean will wipe the entire ./<OBJDIRBASE> subdirectory hierarchy, among other things.

./<EXEDIR>

This is the base directory for generated executable files. The build environment does currently not support coexistence of differently built executables, since the linking overhead is usually negligible. It is possible, however, to direct executable files into subdirectories underneath the base directory to simulate such a feature.