Thursday, July 18, 2013

A guide to Install and use Trilinos for beginners part I

Trilinos is an amazing collection of numerical algorithms for large scale simulations.
It allows people who have very little programming experience in C/C++ to use advanced numerical techniques and powerful solvers.

Being myself an inexperienced C/C++ programmer and unfamiliar with the linux environment I found extremely difficult to get started. In this article I'd like to describe how to install and compile programs using the Trilinos library.
Using the following steps I have been able to install Trilinos under Ubuntu 12.04 32-bit, Ubuntu 12.10 64-bit and two cluster systems that run some kind of linux distribution.

First download the .gz file from their website. (You need to enter your email address).
unzip it in your hard drive.
Then make sure that you have cmake 2.8 or greater installed by typing
$cmake --version
If cmake is not installed in your system you can get it from here.
(The installation of cmake was straightforward).

Untar the file: (This has to be done only once)
$ tar -zxvf  trilinos-11.0.3-Source.tar.gz
You need of course to replace the name with the name of the file you have downloaded. This will create the directory, let's say: ../trilinos-11.0.3-Source

Trilinos contains a large number of packages. It is very unlikely that you need all of them, so I highly recommend not to try building all of them at once, but choose only the packages you need. One package I use very often is the Amesos, which provides a nice interface to solve linear systems AX=B using direct methods. In the following I'll describe how to install Amesos and all the packages that Amesos depends on.

For each combination of packages you want to build do the following steps:
Make sure that your working directory is the ../trilinos-11.0.3-Source
Then create a new directory under /trilinos-11.0.3-Source:
Since we want to build the Amesos package with mpi support we will create the AMESOS_MPI directory:
$ mkdir AMESOS_MPI 
Go into the new directory
$ cd AMESOS_MPI

Create an empty file and name it for example "do-configure". Then copy the content from this example to the newly created "do-configure" file, which should be located under the ../trilinos-11.0.3-Source/AMESOS_MPI/.
You may have already guessed that the lines starting with # are comments.
Modify the second non-comment line as follows:
rm -f CMakeCache.txt  ; find . -name CMakeFiles -exec rm -rf {} \;
After the line cmake \ give the follow options:
-D CMAKE_INSTALL_PREFIX:PATH=../trilinos-11.0.3-Source/AMESOS_MPI \
This line tells where to install the package.

-D CMAKE_BUILD_TYPE:STRING=DEBUG \
You can choose here DEBUG or RELEASE (start always with debug)

-D CMAKE_CXX_COMPILER:FILEPATH=/usr/bin/mpiCC \
Here you provide the mpi c++ compiler (If you don't know where this is try to locate it with the command $locate mpiCC )

-D CMAKE_C_COMPILER:FILEPATH=/usr/bin/mpicc \
This is the mpi C compiler

-D CMAKE_Fortran_COMPILER:FILEPATH=/usr/bin/gfortran \
And this is the fortran compiler

-D HAVE_GCC_ABI_DEMANGLE:BOOL=ON \
-D Trilinos_WARNINGS_AS_ERRORS_FLAGS:STRING="" \

-D CMAKE_VERBOSE_MAKEFILE:BOOL=TRUE \
The options I'm using here are only a small subset of the available options. You can see the full list and the explanation of the above options here.

-D Trilinos_ENABLE_ALL_PACKAGES:BOOL=FALSE \
Next you tell trilinos that you don't want to build all the packages.

-D Trilinos_ENABLE_Amesos:BOOL=ON \
And now you tell that you want to enable the Amesos package. If you want to build any other package, you have to replace the Amesos with the name of the package e.g. ML, Optika, Teuchos etc.

-D Trilinos_ENABLE_ALL_OPTIONAL_PACKAGES:BOOL=ON \
-D Trilinos_ENABLE_TESTS:BOOL=ON \
-D Trilinos_ENABLE_EXAMPLES:BOOL=ON \
Please see here what these options are.

-D TPL_ENABLE_MPI:BOOL=ON \
Here you tell that you want MPI support and next you provide the includes and library paths for the mpi. If you dont know where these are you can use the locate command
-D MPI_BASE_DIR:PATH="/usr/lib/openmpi" \
-D MPI_BIN_DIR:PATH="/usr/bin" \

In my laptop I only have 2 processors so I'm setting the following to 2
-D MPI_EXEC_MAX_NUMPROCS:STRING=2 \
-D MPI_EXEC:FILEPATH="mpirun" \
-D MPI_EXEC_NUMPROCS_FLAG:STRING=-np \
The above options are related to the mpi.

Finally we need to provide the names and paths of blas and lapack libraries
-D BLAS_LIBRARY_NAMES:STRING="blas" \
-D BLAS_LIBRARY_DIRS:PATH=/usr/lib/libblas \
-D LAPACK_LIBRARY_NAMES:STRING="lapack" \

-D LAPACK_LIBRARY_DIRS:PATH=/usr/lib/lapack \
Again if you are not sure where these are, use the locate command.

At the very end add these two lines
$EXTRA_ARGS \

../
The last line tells the script to go one level up so you need to make sure that the AMESOS_MPI directory is under the trilinos-11.0.3-Source directory.

Make the do-configure file executable and execute:
$chmod 755 do_configure
$./do-configure
If there are no errors, then you can build the package by typing
$make -j2
where -j2 is the number of available processors.
If there are no errors then you can try some tests:
$ctest -j2
And you can find out how many tests have failed. (Hopefully none)
The last step is to install the package by typing:
$make install

Now the package is ready for use! 

In the following I'm listing the do-configure file without comments:
cmake \
-D CMAKE_INSTALL_PREFIX:PATH=/home/giorgos/Downloads/trilinos-11.0.3-Source/AMESOS_MPI \
-D CMAKE_BUILD_TYPE:STRING=DEBUG \
-D CMAKE_CXX_COMPILER:FILEPATH=/usr/bin/mpiCC \
-D CMAKE_C_COMPILER:FILEPATH=/usr/bin/mpicc \
-D CMAKE_Fortran_COMPILER:FILEPATH=/usr/bin/gfortran \
-D HAVE_GCC_ABI_DEMANGLE:BOOL=ON \
-D Trilinos_WARNINGS_AS_ERRORS_FLAGS:STRING="" \
-D CMAKE_VERBOSE_MAKEFILE:BOOL=TRUE \
-D Trilinos_ENABLE_ALL_PACKAGES:BOOL=FALSE \
-D Trilinos_ENABLE_Amesos:BOOL=ON \
-D Trilinos_ENABLE_ALL_OPTIONAL_PACKAGES:BOOL=ON \
-D Trilinos_ENABLE_TESTS:BOOL=ON \
-D Trilinos_ENABLE_EXAMPLES:BOOL=ON \
-D TPL_ENABLE_MPI:BOOL=ON \
-D MPI_BASE_DIR:PATH="/usr/lib/openmpi" \
-D MPI_BIN_DIR:PATH="/usr/bin" \
-D MPI_EXEC_MAX_NUMPROCS:STRING=2 \
-D MPI_EXEC:FILEPATH="mpirun" \
-D MPI_EXEC_NUMPROCS_FLAG:STRING=-np \
-D BLAS_LIBRARY_NAMES:STRING="blas" \
-D BLAS_LIBRARY_DIRS:PATH=/usr/lib/libblas \
-D LAPACK_LIBRARY_NAMES:STRING="lapack" \
-D LAPACK_LIBRARY_DIRS:PATH=/usr/lib/lapack \
$EXTRA_ARGS \
../






1 comment:

Daniel Kanewske said...

Thank you!

Earning a PHD in math. Self taught C++ programmer. Can't figure this stuff out.