Commit 00017cb4 authored by Matthieu Dorier's avatar Matthieu Dorier
Browse files

Added Summit files

parent b6a8b220
Summit
======
Programming environment
-----------------------
Any version of gcc should be working on Summit. Summit provides the
spectrum-mpi package as MPI implementation. When building source codes,
you should load the relevant gcc as follow:
```
$ module swap xl gcc/9.1.0
```
The `spack.yaml` file in this directory uses gcc 9.1.0.
Networking
----------
Mercury should use the `ofi+verbs;ofi_rxm` transport method in Mercury.
Libfabric should be compiled with the `+verbs` and `+rxm` variants.
Job management
--------------
Summit uses the lsbatch workload manager. `job.lfs` is an example
of job file. Please modify the header (lines starting with `#BSUB`)
to use your project allocation and set relevant parameters. You can
refer to [the documenattion here](https://docs.olcf.ornl.gov/systems/summit_user_guide.html#running-jobs)
for more information on how to configure such a job script.
Once modified, the job script may be submitted as follows.
```
$ bsub ./job.lfs
```
#!/bin/bash
#BSUB -P CSC332
#BSUB -J margo
#BSUB -o margo.o%J
#BSUB -W 10
#BSUB -nnodes 2
# disable MR cache in libfabric; still problematic as of libfabric 1.10.1
export FI_MR_CACHE_MAX_COUNT=0
# use shared recv context in RXM; should improve scalability
export FI_OFI_RXM_USE_SRX=1
jsrun -a 1 -n 2 -r 1 -c ALL_CPUS ./margo-p2p-latency -i 100000 -n "ofi+verbs;ofi_rxm://"
jsrun -a 1 -n 2 -r 1 -c ALL_CPUS ./margo-p2p-bw -x 1048576 -n "ofi+verbs;ofi_rxm://" -c 1 -D 20
spack:
specs:
- mochi-margo%gcc@9.1.0
- mercury@master # 2.0.0rc1 doesn't build; remove this line when 2.0.0 comes out
concretization: together
compilers:
- compiler:
paths:
cc: /sw/summit/gcc/9.1.0-alpha+20190716/bin/gcc
cxx: /sw/summit/gcc/9.1.0-alpha+20190716/bin/g++
f77: /sw/summit/gcc/9.1.0-alpha+20190716/bin/gfortran
fc: /sw/summit/gcc/9.1.0-alpha+20190716/bin/gfortran
operating_system: rhel7
target: ppc64le
modules:
- gcc/9.1.0
environment: {unset: []}
extra_rpaths: []
flags: {}
spec: gcc@9.1.0
repos:
- /ccs/home/mdorier1/sds-repo
packages:
all:
compiler: [gcc@9.1.0]
providers:
mpi: [spectrum-mpi]
pkgconfig: [pkg-config]
buildable: true
version: []
target: []
spectrum-mpi:
externals:
- spec: spectrum-mpi@10.3.1.2%gcc%9.1.0
modules:
- spectrum-mpi/10.3.1.2-20200121
buildable: false
version: []
target: []
providers: {}
compiler: []
openssl:
externals:
- spec: openssl@1.0.2k
prefix: /usr
buildable: false
version: []
target: []
providers: {}
compiler: []
autoconf:
externals:
- spec: autoconf@2.69
prefix: /usr
buildable: false
version: []
target: []
providers: {}
compiler: []
automake:
externals:
- spec: automake@1.13.4
prefix: /usr
buildable: false
version: []
target: []
providers: {}
compiler: []
libtool:
externals:
- spec: libtool@2.4.2
prefix: /usr
buildable: false
version: []
target: []
providers: {}
compiler: []
ucx:
externals:
- spec: ucx@1.5.1
prefix: /usr
buildable: false
version: []
target: []
providers: {}
compiler: []
libnl:
externals:
- spec: libnl@3.3.0
modules:
- libnl/3.3.0
buildable: false
version: []
target: []
providers: {}
compiler: []
rdma-core:
externals:
- spec: rdma-core@20
prefix: /usr
buildable: false
version: []
target: []
providers: {}
compiler: []
mercury:
variants: ~boostsys
buildable: true
version: []
target: []
providers: {}
compiler: []
libfabric:
variants: fabrics=verbs,rxm
buildable: true
version: []
target: []
providers: {}
compiler: []
view: true
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment