Siesta Installation (Openmpi and Intel MKL)

Installation of Siesta (MPI Version)

I covered the following steps to install Siesta in a large CLUSTER. It is composed of several nodes, each comprises 8 or 12 cores, and about 24 GB RAM. It is a CentOS, release 6.5. The information about your operating system can be extracted using the following linux commands in the vim editor in your account:

$ uname -a

$ lsb_release -a

or,

$ cat /etc/issue

Step – 1

Before installing Siesta, you need to check whether or not your system accompanies the required libraries, e.g., the Intel (or, ACML or ATLAS, etc.), and the Openmpi (viz. openmpi/1.6.3-ib, etc.) libraries. In my case, the CLUSTER was equipped with Intel and Openmpi libraries, thereby it was easier for me to call the library files.

In your case, if you have not yet done so, please proceed to install it now (or contact the sytem administator to help you in this regard).

Step – 2

It is always good to define the path (location) of the Intel and other libraries in the .bashrc or .bash_profile file. In order to do so, you need to type the following command, for instance, to open the .bashrc file:

$ vi .bashrc

(and press enter)

You would now see the following in the file:

# .bashrc

# Source global definitions if [ -f /etc/bashrc ]; then . /etc/bashrc fi

# User specific aliases and functions

...

... In this blank area, you can define your path, which must be similar to the one given just below (see bold font).

...

Make sure you are editing these files very carefully, otherwise all the commands (viz. ls, dd, cd, and vi, etc.) will disappear, and in that case, you may have to contact your system administator to help you out. I suggest to copy the .bashrc file to bashrc-copy using the command: $ cp .vashrc bashrc-copy, and keep it in your account to avoid accidents similar to what is just mentioned above.

Like I just mentioned above, your PATH in the .bashrc should look like the following:

export LAPACK=$MKLPATH/libmkl_lapack95_lp64.a

PATH=$PATH:$LAPACK

…

…

..

Now you save the file, and get out of it. Your next job is to compile the .bashrc file suing the following command

$ source .bashrc

[The above example indicates the LAPACK library file is in the $MKLPATH/ directotry. You can varify it using the command: $ls $MKLPATH/ (this will list all the files in that directory).]

Step – 3

You need to check if your openmpi module is accessible.

$ module list

If it does not show anything, then you may want to type the following command:

$ module load openmpi (or, openmpi/1.6, etc, depending on what your system associates)

Now, use the command, module list. It would probably show you something like:

$ module list

1) openmpi/1.6

NB: If you have installed openmpi in your own account (not in root) than you might want to define the dynamical path in the .bashrc or .bashprofile file. It should look something like:

export DYLD_LIBRARY_PATH=/home/username/openmpi-1.6.5/lib:$DYLD_LIBRARY_PATH

Step – 4

I demonstrate here how to install Siesta in the user account (you do not need any password to log into on to the superuser account to install Siesta). In your account, and using vi editor, just type:

$ mkdir siesta (this may be done in in /home/user (user: your account name))

(and in all cases, press enter)

$ cd siesta

$ wget http://departments.icmab.es/leem/siesta/CodeAccess/Code/siesta-3.2-pl-5.tgz

$ tar xvf siesta-3.2-pl-5.tgz

$ cd siesta-3.2-pl-5/Obj

$ sh ../Src/obj_setup.sh

*** Compilation setup done.

*** Remember to copy an arch.make file or run configure as:

../Src/configure [configure_options]

Now type:

$ ../Src/./configure --enable-mpi

(and press enter)

This would print something like the following (depending on your system):

checking build system type... x86_64-unknown-linux-gnu

checking host system type... x86_64-unknown-linux-gnu

[..]

checking for mpifc... no

checking for mpxlf... no

checking for mpif90... mpif90

checking for MPI_Init... no

checking for MPI_Init in -lmpi... yes

[..]

checking for sgemm in /opt/openblas/lib/libopenblas.so... yes

checking LAPACK already linked... yes

checking LAPACK includes divide-and-conquer routines... yes

configure: using DC_LAPACK routines packaged with SIESTA due to bug in library. Linker flag might be needed to avoid duplicate symbols

configure: creating ./config.status

config.status: creating arch.make

Step – 5

Now use, ls arch.make command after the symbol $. This would allow you to see its appearance in the same folder you are working.

To edit the arch.make file in the $ siesta-3.2-pl-5/Obj directory, type:

$ vi arch.make (and press enter)

Now you must change your own paths for the libraries similar to what are shown below in RED fonts, and you will do that if you have not done so in the .bashrc file.

#

# This file is part of the SIESTA package.

#

# Copyright (c) Fundacion General Universidad Autonoma de Madrid:

# E.Artacho, J.Gale, A.Garcia, J.Junquera, P.Ordejon, D.Sanchez-Portal

# and J.M.Soler, 1996- .

#

# Use of this software constitutes agreement with the full conditions

# given in the SIESTA license, as signed by all legitimate users.

#

.SUFFIXES:

.SUFFIXES: .f .F .o .a .f90 .F90

SIESTA_ARCH=x86_64-unknown-linux-gnu--unknown

FPP=

FPP_OUTPUT=

FC=mpif90

RANLIB=ranlib

SYS=nag

SP_KIND=4

DP_KIND=8

KINDS=$(SP_KIND) $(DP_KIND)

FFLAGS=-g -O2

FPPFLAGS= -DMPI -DFC_HAVE_FLUSH -DFC_HAVE_ABORT

LDFLAGS=

ARFLAGS_EXTRA=

FCFLAGS_fixed_f=

FCFLAGS_free_f90=

FPPFLAGS_fixed_F=

FPPFLAGS_free_F90=

BLAS_LIBS=-lblas

LAPACK_LIBS=-llapack

BLACS_LIBS=

SCALAPACK_LIBS=

COMP_LIBS=dc_lapack.a

NETCDF_LIBS=

NETCDF_INTERFACE=

#(THIS IS THE PATH FOR MKL AND OPENMPI)

MKLPATH=/usr/local/pkg/intel/composer_xe_2011_sp1.9.293/mkl/lib/intel64/

SCALAPACK=$MKLPATH/libmkl_scalapack_lp64.a

BLACS=$MKLPATH/libmkl_blacs_openmpi_lp64.a

LAPACK=$MKLPATH/libmkl_lapack95_lp64.a

BLAS=-Wl,--start-group $MKLPATH/libmkl_intel_lp64.a $MKLPATH/libmkl_sequential.a $MKLPATH/libmkl_core.a -Wl, --end-group -pthread

LIBS = -lmkl_scalapack_lp64 -lmkl_blacs_openmpi_lp64 -lmkl_lapack95_lp64 -Wl --start-group -lmkl_intel_lp64 -lmkl_sequential -lmkl_core -Wl --end-group -pthread -lmpi

MPI_LIBS= -L/usr/local/pkg/openmpi/1.6.3-ib/lib/ -lmpi -lmpi_f90

#LIBS=$(SCALAPACK_LIBS) $(BLACS_LIBS) $(LAPACK_LIBS) $(BLAS_LIBS) $(NETCDF_LIBS)

#SIESTA needs an F90 interface to MPI

#This will give you SIESTA's own implementation

#If your compiler vendor offers an alternative, you may change

#to it here.

MPI_INTERFACE=libmpi_f90.a

MPI_INCLUDE=.

#Dependency rules are created by autoconf according to whether

#discrete preprocessing is necessary or not.

.F.o:

$(FC) -c $(FFLAGS) $(INCFLAGS) $(FPPFLAGS) $(FPPFLAGS_fixed_F) $<

.F90.o:

$(FC) -c $(FFLAGS) $(INCFLAGS) $(FPPFLAGS) $(FPPFLAGS_free_F90) $<

.f.o:

$(FC) -c $(FFLAGS) $(INCFLAGS) $(FCFLAGS_fixed_f) $<

.f90.o:

$(FC) -c $(FFLAGS) $(INCFLAGS) $(FCFLAGS_free_f90) $<

Step 6

Now type:

$ make

(and press enter)

It would take a few minutes to finish the compilation process. As a final step, you may want to link the exe file (called siesta) feasible in the same folder to your account. You would therefore need to define it in your .bashrc file, and recompile it using the source command similar to how you did it above. Use a scipt file to load your job if it is a queuing system.

Have fun with Siesta.