Commit ce720a6f authored by Philip Carns's avatar Philip Carns

update Summit information to specify mlx5_0 domain

parent 8570c522
......@@ -18,9 +18,13 @@ The `spack.yaml` file in this directory uses gcc 9.1.0.
Networking
----------
Mercury should use the `ofi+verbs;ofi_rxm` transport method in Mercury.
Libfabric should be compiled with the `+verbs` and `+rxm` variants.
Mercury should use the `verbs` transport method in Mercury, which
translates to `ofi+verbs;ofi_rxm` internally. You must also specify the
libfabric domain when initializing Mercury by using `verbs://mlx5_0`
as the local address. This will insure that libfabric
uses the correct default port on the network card.
Libfabric should be compiled with the `+verbs` and `+rxm` variants.
Job management
--------------
......
......@@ -10,5 +10,8 @@ export FI_MR_CACHE_MAX_COUNT=0
# use shared recv context in RXM; should improve scalability
export FI_OFI_RXM_USE_SRX=1
jsrun -a 1 -n 2 -r 1 -c ALL_CPUS ./margo-p2p-latency -i 100000 -n "ofi+verbs;ofi_rxm://"
jsrun -a 1 -n 2 -r 1 -c ALL_CPUS ./margo-p2p-bw -x 1048576 -n "ofi+verbs;ofi_rxm://" -c 1 -D 20
# Note that we explicitly specify the libfabric domain of "mlx5_0" on
# Summit. Otherwise libfabric and/or libibverbs may select a default port
# that does not work out of the box.
jsrun -a 1 -n 2 -r 1 -c ALL_CPUS ./margo-p2p-latency -i 100000 -n "verbs://mlx5_0"
jsrun -a 1 -n 2 -r 1 -c ALL_CPUS ./margo-p2p-bw -x 1048576 -n "verbs://mlx5_0" -c 1 -D 20
Markdown is supported
0%
or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment