Building and Running CODES on Theta

• Build DUMPI as follows

./configure --enable-libdumpi --enable-shared --enable-libundumpi --prefix=dumpi/install CC=cc CXX=CC CFLAGS=-g -O0 -DMPICH_SUPPRESS_PROTOTYPES=1 -DHAVE_PRAGMA_HP_SEC_DEF=1 LDFLAGS=-dynamic

make && make install

• Unload the following module because it conflicts with CODES autoconf module unload atp

• Force compilers to dynamic link

export LDFLAGS=-dynamic

• Configure ROSS like this:

ARCH=x86_64 CC=cc CXX=CC cmake -DCMAKE_INSTALL_PREFIX=../install ../

make && make install

• Configure codes like this. Set PKG_CONFIG_PATH to point at the location of ROSS and Darshan installation. The important things here more generally are the CC, CXX, and --host settings. The --host setting forces it to cross compile:

../configure --prefix=/install/directory LDFLAGS=-dynamic
CC=cc CXX=CC PKG_CONFIG_PATH=dev/ROSS/lib/pkgconfig:/soft/perftools/darshan/darshan-3.1.4/lib/pkgconfig --with-dumpi=dumpi/install --host=x86_64-linux

make && make install

make tests

• Set current working directory to codes/build for running the dragonfly-custom simulations. The model depends on network configuration files that have a relative path.

• Running on theta: Set options -n for number of pes per node, -N for total number of processes (pes * number of nodes) and -j for number of threads per core.