A cluster consists of one or more head nodes and a number of compute nodes. Users log into a head node, and submit jobs which then run on one or more compute nodes. Because a head node serves a different purpose than the compute nodes within a cluster, it necessarily has a different set of software installed, and potentially a slightly different runtime environment.

Because of the configuration differences between the head node and the compute nodes, it is possible that software configured and compiled on the head node may not work properly on the compute nodes. It is therefore advisable to configure and compile software on a compute node, to ensure that it will run in the exact same environment in which it was configured and compiled.

Multiple users compiling large programs on the head node would also create a heavy load on the head node that would impact other users.

Since compute nodes are allocated by the scheduler for computational processes, it would not be safe to simply choose a compute node manually and compile our software on it. Doing so could interfere with a scheduled job. Therefore, compilations should be scheduled like any other process that requires a compute node.

On a heterogeneous cluster or grid (different nodes have different CPU architectures or different operating systems) you may want to compile the program on every node as part of the execution job. If a program is going to run for hours or days, adding seconds or minutes of compilation time to each process is a trivial cost.

For large compilations, you may wish to schedule a batch serial job as outlined in the section called “Batch Serial”. This method will create an output file containing screen output from the build commands. An example build script for a software package with a Makefile is shown below. Note that each package may have its own build method, so the commands in the script below will need to be changed for each package you build.

    #!/bin/sh
    
    # SLURM submit script for compilation
    
    make
    

For smaller builds, you may prefer to watch the compiler output as it occurs, in which case a batch interactive job would be more suitable. Batch interactive jobs are described in the section called “Interactive”.

    #!/bin/sh -e
    
    # -e terminates script if any command fails
    
    ./configure --prefix $HOME/myprogs
    make
    make install
    

Scheduling compilations this way will not introduce any measurable delays in compiling your code. Dispatching of batch-serial and interactive jobs generally occurs almost immediately, since there is almost always one free core available in the cluster.