Skip to content
GitLab
Menu
Projects
Groups
Snippets
Loading...
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
sds
Experiments
platform-configurations
Commits
f68ae8a9
Commit
f68ae8a9
authored
Oct 27, 2020
by
Matthieu Dorier
Browse files
Merge branch 'carns/editing' into 'master'
various edits See merge request
!2
parents
b9fbd06f
60ee5f73
Changes
4
Hide whitespace changes
Inline
Side-by-side
ANL/Bebop/README.md
View file @
f68ae8a9
...
...
@@ -48,3 +48,7 @@ Once modified, the job script may be submitted as follows.
```
$ sbatch ./job.sbatch
```
Note that the job script explicitly multiple PSM2 environment variables that
are necessary for Mochi compatibility. Please copy these environment
variables to any other Mochi job scripts that you create for the Bebop platform.
ANL/Theta/README.md
View file @
f68ae8a9
...
...
@@ -16,8 +16,9 @@ $ module swap PrgEnv-intel PrgEnv-gnu
$ module load gcc/9.3.0
```
You may want to
`export CRAYPE_LINK_TYPE=dynamic`
(e.g. in your
`.bashrc`
file)
to avoid problems with dynamic libraries when building programs.
You may also want to
`export CRAYPE_LINK_TYPE=dynamic`
(e.g. in your
`.bashrc`
file) to prevent the compiler from attempting to use static
linking in conjunction with the dynamic libraries produced by Spack.
Networking
...
...
@@ -42,18 +43,18 @@ Once modified, the job script may be submitted as follows.
$ qsub ./job.qsub
```
Note the use of a protection domain (pdomain): these are necessary if
you have multiple applications, started using distinct
`aprun`
calls,
and you want them to communicate via Mercury. Protection domains are
global, but user-specific (i.e. you won't be able to use a pdomain
setup by someone else, and you won't be able to use the same name
as someone else's pdomain). There is a limited number of pdomains
that can exist simultaneously, and they are not cleaned up when your
job terminates, so please make sure to delete your pdomain at the
end of your job.
Note the use of a protection domain (pdomain). Shared protection domains
enable processes to communicate with each other even when launched with
separate
`aprun`
commands. Protection domains are global, but user-specific
(i.e. you won't be able to use a pdomain setup by someone else, and you
won't be able to use the same name as someone else's pdomain). There is
a limited number of pdomains that can exist simultaneously, and they are
not cleaned up when your job terminates, so please make sure to delete
your pdomain at the end of your job.
Note that other Cray systems may use
`DRC (Dynamic RDMA Credentials)`
rather
than static protection domains for this purpose. The protection domain
configuration in this example is specific to the Theta system.
The
`MPICH_GNI_NDREG_ENTRIES`
environment variable should be set
to avoid conflicts between MPI and libfabric.
`HDF5_USE_FILE_LOCKING=FALSE`
has been shown to be useful for some
workloads using HDF5 on Lustre.
ANL/Theta/job.qsub
View file @
f68ae8a9
...
...
@@ -7,8 +7,9 @@
set
-eu
# prevents interference between MPI and Mochi (libfabric) libraries
# when sharing GNI resources
export
MPICH_GNI_NDREG_ENTRIES
=
1024
export
HDF5_USE_FILE_LOCKING
=
FALSE
PDOMAIN
=
mypdomain
...
...
README.md
View file @
f68ae8a9
...
...
@@ -3,17 +3,17 @@ Platform configurations for Mochi
This repository provides Spack configuration files, example job scripts, and
notes about building and running Mochi-based codes on various platforms.
Please refer to your platform of interest for more information.
Please refer to the subdirectory for your platform of interest for more
information.
Using spack.yaml files
----------------------
Each platform subdirectory in this repository provides a
`spack.yaml`
file.
Such a file is meant to
fully describe a Spack environment, including
some
system-provided
A
`spack.yaml`
file
fully describe
s
a Spack environment, including system-provided
packages and compilers. It does so independently of any
`compilers.yaml`
or
`packages.yaml`
files installed in
`~/.spack`
, hence preventing as much as possible the possible interference
with user-specific spack configurations.
files installed in
`~/.spack`
, thereby preventing interference with user-specific spack configurations as much as possible.
You may use
`spack.yaml`
files to create a
[
Spack environment
](
https://spack.readthedocs.io/en/latest/environments.html
)
in
...
...
@@ -27,7 +27,8 @@ $ . spack/share/spack/setup-env.sh
```
Remember that the second line needs to be executed every time you open a new
terminal.
terminal; it may be helpful to create an alias in your bashrc file as a
shortcut.
You will then need to clone
`sds-repo`
, which contains the Mochi packages.
...
...
@@ -54,7 +55,7 @@ Then, execute the following command
$ spack env create myenv spack.yaml
```
Change to a directory outside the
`platform-configurations`
folders
Change to a directory outside
of
the
`platform-configurations`
folders
and activate the environment as follows.
```
...
...
@@ -102,10 +103,13 @@ Should you want to contribute a `spack.yaml` for a particular machine,
please submit a merge request with it, and ensure the following.
*
The
`spack.yaml`
file should contain the compiler(s) that have been tested
to be
work
ing
with Mochi packages.
and confirmed to
work with Mochi packages.
*
The
`spack.yaml`
file should try to list system-provided packages,
in particular packages used for building (
`cmake`
,
`autoconf`
, etc.),
and relevant system-provided MPI implementations.
-
Note that this must be done manually. Spack provides a
`spack external
find`
command that can be used to locate a subset of system packages,
but it does not populate the
`spack.yaml`
file.
*
The
`spack.yaml`
file should contain the relevant variants for packages,
in particular the transport methods to use with
`libfabric`
.
*
The path to the
`spack.yaml`
file should be of the form
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment