Skip to Main Content

Information Technology Services

University of Hawaii System

Installing Software

In an effort to allow researchers to get research done the ITC CI group allows users to install software in their ~/apps directory.  We do this so that our team is not a bottleneck for users getting software on the HPC.  So if you know how to compile your software package or follow the directions in the software package documentation for installing you are welcome to do that.  If you have problems installing or questions please feel free to email uh-hpc-help@lists.hawaii.edu and the ITS CI group will answer questions and make suggestions.

USING PREBUILT PACKAGES

The UH ITS HPC cluster compute nodes have the Centos 6 operating system.  So most packages that have been pre-built using Redhat 6 or Centos 6 they will likely function – they won’t be optimized but that might be fine.  Please note if the application needs to use MPI it is extremely likely it will not function correctly on the cluster and should be compiled on the cluster directly.

If your pre-built software package meets the basic criteria for the cluster then download or transfer that binary package directly into your ~/apps folder and then test it using an interactive session on the sandbox nodes.

WHERE TO INSTALL SOFTWARE

Software you install for yourself should be installed in the ~/apps directory (the apps directory in your home folder that was created for you by the admins when your account was created).  This directory was specifically created to hold application and configuration files that are meant to persist for long periods of time.  The ~/apps folder is exempt from the automated file purge.  If you install software in another place it will not be guaranteed to be exempt for the file purge and could be deleted when a purge occurs.

INSTALLING A REDHAT/CENTOS 6 RPM

An RPM package that can be installed into a different directory is said to be relocatable. The –prefix NEWPATH option is used for for relocatable binary packages. It translate all file paths that start with the installation prefix in the package relocation hint(s) to NEWPATH.

Download the rpm file (.rpm) then run the following:

rpm --prefix=~/apps foo-ver.rpm

NOTE that not all RPM packages support relocation so you may have to compile the package from source.

COMPILERS

We have the Intel®, GNU (gcc, g++), Cray®  & PGI®  compilers.  TheIntel® compilers seem to provide the best  performance on the cluster based on the Intel CPUs in the nodes and the Infiniband Network fabric that connects the compute nodes.

Compiling must take place on a compute node

Login nodes are for cluster access, data transfer and submiting jobs and since compiling can be CPU intensive it should not be done on the login nodes as this will intefere with other users and can make the login nodes unresponsive.

Interactive sessions are useful for compiling software and the Sandbox partition is the suggested area to do this as the nodes mirror the environment the compute nodes provide and are ideal for compilation as they are typically available.

Login nodes do not load all the software and libraries found on the compute nodes so again not the correct place to compile.

Intel® 2013 compilers:

The 2013 compilers can be used by loading the following modules:

  • module load intel/ics – Loads Intel®  compilers: icc, ifort, icpc
  • module load intel/impi – Loads Intel® MPI wrapper: mpiicc, mpiifort, mpiicpc

Intel® 2016 compilers:

We have 2 floating seats for Intel® 2016 and the module to load are:

  • module load compiler intel_2016/ics
  • module load intel_2016/impi

COMPILING FROM THE ORIGINAL SOURCE

Extracting the archive

  • Download foo.tar.gz
  • Extract with: tar -xzvf foo.tar.gz
    -x
    extract the file
    -z
    unzip it (required for .gz files)
    -v
    verbose, print the names of the files as they are extracted
    -f foo.tar.gz
    extract the named file

Read documentation

Look for files called: INSTALL, README, SETUP, or similar.

Read with less docfile, or zless docfile.gz for .gz files.

Building

First you need to load the compiler modules (see the sections above for Non-MPI and MPI Intel compilers)

Follow the instructions that come with the package. Typically they will tell you to:

./configure

Many packages have a configure script which will query your system and configure themselves appropriately. Not supported by all software.  You need to add –prefix=~/apps to the ./configure command like so:

./configure --prefix=~/apps

This will tell the configure script to setup the software to be built in your ~/apps directory which is important since you do not have access to install it in the typical place most software wants to be installed.  Additionally, you will need to specify the compiler you want to use typically by setting CC or CXX.  For example:

./configure CC=ICC  CXX=ICPC FC=IFORT --prefix=~/apps

or

./configure CC=MPICC  CXX=MPICXX FC=MPIF90 --prefix=~/apps

The above tells the configure script to create the makefiles using the Intel non-mpi compiler in the first example and the Intel MPI compilers in the second example (not FC can also be MPIF70 depending on which fortran is required).

make

Compile the software. Some require a file name to build such as make foomake reads a file normally called Makefile for what commands to run.

make install

Install the software into ~/apps or where you set the –prefix.

COMPILING NON-MPI SOFTWARE

Once the compiler modules have been loaded the C compiler is invoked by using some of the following commands icc, icpc or ifort depending on the source code.  Below is a table that indicates which command should be used for which program source type.

Compiling non-MPI Software
Compiler Program Type Suffix Example
icc C .c icc [compiler_options] prog.c
icpc C++ .C .cc .cpp .cxx icc [compiler_options] prog.cpp
ifort F77 .f .for .ftn ifort [compiler_options] prog.f
ifort F90 .f90 .fpp ifort [compiler_options] prog.f90

If you would like to see the list of available flags, please use the man pages by typing man icc (C), man icpc (C++). To find the compiler’s version, use the flag -v, i.e. type icc -v (C), icpc -v (C++).

Generally the flag -fast gives superior performance, though be careful since the optimization flags are fairly aggressive. If the code has numerical problems, try to compile with -O3 instead.

Consult the icc/icpc man page for more details.

COMPILING MPI SOFTWARE

The mpicc, mpicxx, mpif90 compiler scripts (wrappers) compile MPI code and automatically link start up and message passing libraries to the executable.To determine which libraries are automatically included using the mpi wrapper scripts:

The Intel compiler module can be loaded with the Intel MPI module, for example:

module load intel/ics

module load mpi/impi

When the above is performed the following table shows which MPI wrappers can be used.

Compiling MPI Software
Compiler Program Type Suffix Example
mpicc C .c mpicc [compiler_options] prog.c
mpicxx C++ .C .cc .cpp .cxx mpicxx [compiler_options] prog.cpp
mpif70 F77 .f .for .ftn mpif77 [compiler_options] prog.f
mpif90 F90 .f90 .fpp mpif90 [compiler_options] prog.f90

The MPI compiler wrappers use the same compilers as for serial compilation (icc, icpc, ifort) and therefore when using the Intel compiler, the flags used below can be used with the MPI wrappers. Common options include:

MPI Compiler Wrappers – Flags
Compiler Program Type Suffix Example
mpicc C .c mpicc [compiler_options] prog.c
mpicxx C++ .C .cc .cpp .cxx mpicxx [compiler_options] prog.cpp
mpif70 F77 .f .for .ftn mpif77 [compiler_options] prog.f
mpif90 F90 .f90 .fpp mpif90 [compiler_options] prog.f90

The MPI compiler wrappers use the same compilers as for serial compilation (icc, icpc, ifort) and therefore when using the Intel compiler, the flags used below can be used with the MPI wrappers. Common options include:

MPI Compiler Options
Compiler Option Description
-O3 Performs some compilation time & memory intensive optimizations in addition to those executed with “-O2”. NOTE: it may not improve the performance of all codes
-ipo Inter Procedural Optimization
-g Debugging information produced during compilation