Commit b9fbd06f authored by Philip Carns's avatar Philip Carns
Browse files

Merge branch 'carns/update-summit-ib-domain' into 'master'

update Summit information to specify mlx5_0 domain

See merge request !1
parents 8570c522 ce720a6f
...@@ -18,9 +18,13 @@ The `spack.yaml` file in this directory uses gcc 9.1.0. ...@@ -18,9 +18,13 @@ The `spack.yaml` file in this directory uses gcc 9.1.0.
Networking Networking
---------- ----------
Mercury should use the `ofi+verbs;ofi_rxm` transport method in Mercury. Mercury should use the `verbs` transport method in Mercury, which
Libfabric should be compiled with the `+verbs` and `+rxm` variants. translates to `ofi+verbs;ofi_rxm` internally. You must also specify the
libfabric domain when initializing Mercury by using `verbs://mlx5_0`
as the local address. This will insure that libfabric
uses the correct default port on the network card.
Libfabric should be compiled with the `+verbs` and `+rxm` variants.
Job management Job management
-------------- --------------
......
...@@ -10,5 +10,8 @@ export FI_MR_CACHE_MAX_COUNT=0 ...@@ -10,5 +10,8 @@ export FI_MR_CACHE_MAX_COUNT=0
# use shared recv context in RXM; should improve scalability # use shared recv context in RXM; should improve scalability
export FI_OFI_RXM_USE_SRX=1 export FI_OFI_RXM_USE_SRX=1
jsrun -a 1 -n 2 -r 1 -c ALL_CPUS ./margo-p2p-latency -i 100000 -n "ofi+verbs;ofi_rxm://" # Note that we explicitly specify the libfabric domain of "mlx5_0" on
jsrun -a 1 -n 2 -r 1 -c ALL_CPUS ./margo-p2p-bw -x 1048576 -n "ofi+verbs;ofi_rxm://" -c 1 -D 20 # Summit. Otherwise libfabric and/or libibverbs may select a default port
# that does not work out of the box.
jsrun -a 1 -n 2 -r 1 -c ALL_CPUS ./margo-p2p-latency -i 100000 -n "verbs://mlx5_0"
jsrun -a 1 -n 2 -r 1 -c ALL_CPUS ./margo-p2p-bw -x 1048576 -n "verbs://mlx5_0" -c 1 -D 20
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment