Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
sds
Experiments
platform-configurations
Commits
ce720a6f
Commit
ce720a6f
authored
Oct 23, 2020
by
Philip Carns
Browse files
update Summit information to specify mlx5_0 domain
parent
8570c522
Changes
2
Hide whitespace changes
Inline
Side-by-side
ORNL/Summit/README.md
View file @
ce720a6f
...
...
@@ -18,9 +18,13 @@ The `spack.yaml` file in this directory uses gcc 9.1.0.
Networking
----------
Mercury should use the
`ofi+verbs;ofi_rxm`
transport method in Mercury.
Libfabric should be compiled with the
`+verbs`
and
`+rxm`
variants.
Mercury should use the
`verbs`
transport method in Mercury, which
translates to
`ofi+verbs;ofi_rxm`
internally. You must also specify the
libfabric domain when initializing Mercury by using
`verbs://mlx5_0`
as the local address. This will insure that libfabric
uses the correct default port on the network card.
Libfabric should be compiled with the
`+verbs`
and
`+rxm`
variants.
Job management
--------------
...
...
ORNL/Summit/job.lfs
View file @
ce720a6f
...
...
@@ -10,5 +10,8 @@ export FI_MR_CACHE_MAX_COUNT=0
# use shared recv context in RXM; should improve scalability
export
FI_OFI_RXM_USE_SRX
=
1
jsrun
-a
1
-n
2
-r
1
-c
ALL_CPUS ./margo-p2p-latency
-i
100000
-n
"ofi+verbs;ofi_rxm://"
jsrun
-a
1
-n
2
-r
1
-c
ALL_CPUS ./margo-p2p-bw
-x
1048576
-n
"ofi+verbs;ofi_rxm://"
-c
1
-D
20
# Note that we explicitly specify the libfabric domain of "mlx5_0" on
# Summit. Otherwise libfabric and/or libibverbs may select a default port
# that does not work out of the box.
jsrun
-a
1
-n
2
-r
1
-c
ALL_CPUS ./margo-p2p-latency
-i
100000
-n
"verbs://mlx5_0"
jsrun
-a
1
-n
2
-r
1
-c
ALL_CPUS ./margo-p2p-bw
-x
1048576
-n
"verbs://mlx5_0"
-c
1
-D
20
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment