Commit bc169172 authored by Matthieu Dorier's avatar Matthieu Dorier
Browse files

final commit for this repository

parent eb5ec539
# spack-packages for the Mercury Suite # DO NOT USE THIS REPOSITORY
Here are a collection of [spack](https://spack.io/) packages to manage
building the Mercury Suite.
For more about spack and what you can do with it, spack has lots of
[documentation](https://spack.readthedocs.io/en/latest/) and a good
[tutorial](https://spack.readthedocs.io/en/latest/tutorial_sc16.html).
## Repo Installation
Once you've set up spack itself (you added `spack` to your `$PATH`, right?),
you need to teach it about this collection
('repository' in spack lingo) of packages. Go to the top-level directory of
this project and execute the following command:
cd sds-repo
spack repo add .
Did it work?
spack repo list
## Mochi Suite Installation
To build the entire mochi suite with the default configuration (mercury using
CCI's TCP transport), simply build margo:
spack install margo
### Side note: System packages vs spack packages
The largest dependency for Mercury is the Boost package. If your system
already has Boost, you can teach spack about it and other
[system packages](https://spack.readthedocs.io/en/latest/getting_started.html#system-packages).
Let's say you installed Boost through your distribution (an RPM or DEB package)
To inform spack about Boost you e.g. installed from an RPM, you would add it to
`~/.spack/<arch>/packages.yaml`
```
packages:
boost:
paths:
boost@system: /usr
version: [system]
buildable: False
```
The sdskeyval package (for now) requires Boost, but we're working on removing
that dependency. In the mean time, please don't put a trailing slash on your
"boost@system" directive or it will break our fragile workaround.
I have found several other large dependencies that are handled just fine by the
operating system. My `${HOME}/.spack/linux/packages.yaml` file looks like
this:
```
packages:
openssl:
paths:
openssl@1.0.2g: /usr
buildable: False
cmake:
paths:
cmake@3.9.1: /usr
buildable: False
boost:
paths:
boost@1.62: /usr
buildable: False
autoconf:
paths:
autoconf@2.69: /usr
buildable: False
automake:
paths:
automake@1.15.1: /usr
buildable: False
mpich:
paths:
mpich@master: /home/robl/work/soft/mpich
buildable: False
```
You can see that I have flagged several packages as `buildable: False`,
generaly because they are either large packages or have large dependencies. I
tend to work from MPICH's latest version in git, so I have also told spack
about my locally installed MPICH.
Note that if you prefer to have spack use mpich over openmpi any time that
"mpi" is a package dependency, add this to your packages.yaml as well:
```
all:
providers:
# prefer MPICH by default
mpi: [mpich,openmpi]
```
These `packages` files live in a platform-specific directory (run `spack arch
-p` to see what platform spack thinks you are on). Pretty helpful for e.g.
Argonne, where a home file system is shared between a linux cluster, a blue
gene, and a Cray. You can describe `packages.py` for each platform.
### Side note: using modules
Spack works well with the module command for loading and unloading
particular packages in your environment once they have been built. In theory
you could integrate it with an existing environment-modules or lmod package
on your system. Alternatively you can have spack set up its own modules
system:
* run ```spack bootstrap```
* this will make spack build and install its very own
environment-modules package that is automatically aware of packages
that have been installed via spack
The remainder of this document assumes that you will use the module command
to load and unload packages once built.
### Further package configuration
Spack has a bit of trouble resolving a dependency if it is not exactly the same
between packages. For example, the `ssg` package depends on mercury.
`sdskeyval` depends on mercury too, but explicitly requests the
`mercury+selfforward` variant. If you install `ssg`, then install `sdskeyval`,
you will end up with two instances of mercury. If you remember to install
`ssg` with the `mercury+selfforward` variant, you can avoid this duplication,
but it's easy to forget (and kind of a pain) to type `spack install
ssg^mercury+selffforward`.
In my `~/.spack/linux/packages.yaml` I tell spack I'd like to always build particular variants:
```
packages:
mercury:
variants: +selfforward
ssg:
variants: +mpi
```
## Using Mochi Suite
One consequence of the spack design (where packages are installed into a prefix
based on a hash of their configuration and compiler) is that library and header
paths are unwieldy. An environment-management tool such as `modules` helps a
lot here, and is nicely integrated into spack:
```
module avail
--- /blues/gpfs/home/robl/src/spack/share/spack/modules/linux-centos6-x86_64 ---
abtsnoozer-master-gcc-4.7.2-4ygmjep libev-4.24-gcc-4.7.2-ddpazlc
argobots-master-gcc-4.7.2-4wxy6p3 libsigsegv-2.10-gcc-4.7.2-vtvikjs
autoconf-2.69-gcc-4.7.2-a2clqmr libtool-2.4.6-gcc-4.7.2-nxthahu
automake-1.15-gcc-4.7.2-bfjmuzz m4-1.4.18-gcc-4.7.2-hftmdfw
boost-1.63.0-gcc-4.7.2-ds3fbl3 margo-master-gcc-4.7.2-uy4in2w
bzip2-1.0.6-gcc-4.7.2-jehobzs mercury-master-gcc-4.7.2-eiyjtqs
cci-2.0-gcc-4.7.2-5y6vgzq zlib-1.2.10-gcc-4.7.2-w53w2nl
...
```
Load the margo module into your environment:
module add margo-master-gcc-4.7.2-uy4in2w
Spack integrages with modules: the integration helps with managing spack's hash-oriented naming convention
spack load margo
The integration can also help you load in all the dependencies:
source <(spack module loads --dependencies margo)
sometimes I get `Error: this command needs exactly **one** module type active`: pass `-m tcl` like this
source <(spack module loads -m tcl --dependencies margo)
```
module list
Currently Loaded Modulefiles:
1) margo-master-gcc-4.7.2-uy4in2w
2) mercury-master-gcc-4.7.2-eiyjtqs
3) abtsnoozer-master-gcc-4.7.2-4ygmjep
4) argobots-master-gcc-4.7.2-4wxy6p3
```
The modules framework will update the `PKG_CONFIG_PATH` for you (slightly reformatted to wrap the long lines)
```
pkg-config --cflags margo
-I/blues/gpfs/home/robl/src/spack/opt/spack/linux-centos6-x86_64/gcc-4.7.2/argobots-master-4wxy6p3thpyhwgo67t6icducpy3ik6y5/include \
-I/blues/gpfs/home/robl/src/spack/opt/spack/linux-centos6-x86_64/gcc-4.7.2/abtsnoozer-master-4ygmjeptxkfddgojrupre624olf2w6mq/include \
-I/blues/gpfs/home/robl/src/spack/opt/spack/linux-centos6-x86_64/gcc-4.7.2/mercury-master-eiyjtqsdv6yynaylgtm5g7jib7fvnshh/include \
-I/blues/gpfs/home/robl/src/spack/opt/spack/linux-centos6-x86_64/gcc-4.7.2/boost-1.63.0-ds3fbl3vdsk6u5soirxcdoe3e6dw3iwx/include \
-I/blues/gpfs/home/robl/src/spack/opt/spack/linux-centos6-x86_64/gcc-4.7.2/cci-2.0-5y6vgzqccmk7n7vehrrcxl5ogukwv3gh/include \
-I/blues/gpfs/home/robl/src/spack/opt/spack/linux-centos6-x86_64/gcc-4.7.2/margo-master-uy4in2w6xniwn43iio7h6ko3j6hbxvfr/include
```
## Possible issues
If you encounter a problem in the build log about `possibly undefined macro:
AM_DISABLE_STATIC`, it's possible the package is using system's (possibly
older) `libtool` and not spack's `libtool`. I could only resolve this by
loading spack's libtool module: (precise name quite likely to be different on
your system)
$ module load libtool-2.4.6-gcc-4.7.2-nxthahu
This repository has been deprecated and archived.
Please use [mochi-spack-packages](https://github.com/mochi-hpc/mochi-spack-packages) instead.
from spack.pkg.builtin.argobots import Argobots
class Argobots(Argobots):
version('1.0', sha256="36a0815f7bf99900a9c9c1eef61ef9b3b76aa2cfc4594a304f6c8c3296da8def", preferred=True)
# Copyright 2013-2019 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
# ----------------------------------------------------------------------------
from spack import *
class Benvolio(AutotoolsPackage):
"""Mochi services for I/O"""
homepage = "https://github.com/mochi-hpc/benvolio"
url = "https://github.com/mochi-hpc/benvolio"
git = "https://github.com/mochi-hpc/benvolio.git"
version('main', branch='main', preferred=True)
version('develop', branch='main')
variant('mpi', default=False, description='Bootstrap Benvolio providers with MPI');
variant('pmix', default=True, description='Bootstrap Benvolio providers with PMIx');
variant('cray-drc', default=False, description='Use Cray Dynamic RDMA Credientials');
depends_on('automake')
depends_on('autoconf')
depends_on('libtool')
# thallium-0.5 provided engine::wrap()
# thallim-0.8.2 provided an engine constructor that takes margo hints
depends_on('mochi-thallium@0.8.2:')
depends_on('mochi-abt-io@0.2:')
# pick up ssg API rework that landed in ssg-0.4.0
# can bootstrap with either pmix or mpi.. not sure how to specify that in spack
depends_on('mochi-ssg@0.4.0:+pmix', when='+pmix')
depends_on('mochi-ssg@0.4.0:+mpi', when='+mpi')
depends_on('rdma-credentials', when='+cray-drc')
# @develop version
#depends_on('mochi-thallium@develop', when='@develop')
#depends_on('mochi-ssg+mpi@develop', when='@develop')
# Require a custom configure_args if we need to build with MPI wrapper
def configure_args(self):
extra_args = []
if '+mpi' in self.spec:
extra_args.extend(['CC=%s' % self.spec['mpi'].mpicc])
extra_args.extend(['CXX=%s' % self.spec['mpi'].mpicxx])
return extra_args
from spack.pkg.builtin.darshan_runtime import DarshanRuntime
class DarshanRuntime(DarshanRuntime):
variant('hdf5', default=False, description='additional HDF5 statistics')
depends_on('hdf5@1.10.0:', when='+hdf5')
def configure_args(self):
spec = self.spec
config_args = super(DarshanRuntime, self).configure_args()
if '+hdf5' in spec:
config_args.append('--enable-hdf5-mod')
return config_args
##############################################################################
#
# Installing flamestore:
#
# spack install flamestore
#
from spack import *
class FlamestoreOld(PythonPackage):
"""Transient distributed object store for deep learning"""
homepage = "https://xgitlab.cels.anl.gov/sds/flame-store-old"
url = "https://xgitlab.cels.anl.gov/sds/flame-store-old"
git = "https://xgitlab.cels.anl.gov/sds/flame-store-old.git"
version('develop', branch='master')
version('0.1', tag='v0.1')
depends_on('py-mochi-margo@0.1:')
depends_on('py-mochi-bake+numpy@0.1:')
depends_on('py-h5py')
depends_on('py-keras')
depends_on('py-mpi4py')
##############################################################################
#
# Installing flamestore:
#
# spack install flamestore
#
from spack import *
class Flamestore(PythonPackage):
"""Transient distributed object store for deep learning"""
homepage = 'https://github.com/mochi-hpc/flamestore'
url = 'https://github.com/mochi-hpc/flamestore'
git = 'https://github.com/mochi-hpc/flamestore.git'
version('main', branch='develop', preferred=True)
version('develop', branch='develop')
variant('theta', default=False,
description='Option to enable when building on Theta')
depends_on('jsoncpp')
depends_on('py-spdlog')
depends_on('spdlog')
depends_on('py-pkgconfig')
depends_on('py-pybind11')
# normal versions
depends_on('py-mochi-bake@master')
depends_on('mochi-ssg@0.4:')
depends_on('mochi-sdskv')
depends_on('mochi-thallium@0.5.3:')
depends_on('py-mochi-margo')
depends_on('py-mochi-tmci')
# develop version
depends_on('py-mochi-bake@develop', when='@develop')
depends_on('mochi-ssg@develop', when='@develop')
depends_on('mochi-sdskv@develop', when='@develop')
depends_on('mochi-thallium@develop', when='@develop')
depends_on('py-mochi-margo@develop', when='@develop')
depends_on('py-mochi-tmci@develop', when='@develop')
@run_before('build')
def move_file(self):
if '+theta' in self.spec:
src = self.stage.source_path+'/theta/tensorflow.json'
dst = self.stage.source_path+'/tensorflow.json'
copy(src, dst)
# Copyright 2013-2019 Lawrence Livermore National Security, LLC and other
# Spack Project Developers. See the top-level COPYRIGHT file for details.
#
# SPDX-License-Identifier: (Apache-2.0 OR MIT)
from spack import *
class Hdf5Rados(CMakePackage):
"""The HDF5 RADOS VOL connector is an external VOL connector that interfaces with the RADOS API"""
homepage = ''
url = ''
git = 'https://git.hdfgroup.org/scm/hdf5vol/rados-api.git'
maintainers = ['soumagne']
version('master', branch='master', submodules=True)
variant('mobject', default=True, description='Use mobject')
depends_on('cmake@2.8.12.2:', type='build')
depends_on('mobject', when='+mobject')
depends_on('hdf5@1.12.0:+mpi')
def cmake_args(self):
"""Populate cmake arguments for HDF5 RADOS."""
spec = self.spec
variant_bool = lambda feature: str(feature in spec)
parallel_tests = '+mpi' in spec and self.run_tests
cmake_args = [
'-DBUILD_SHARED_LIBS:BOOL=ON',
'-DBUILD_TESTING:BOOL=%s' % str(self.run_tests),
'-DHDF5_VOL_RADOS_USE_MOBJECT:BOOL=%s'
% variant_bool('+mobject'),
]
return cmake_args
def check(self):
"""Unit tests fail when run in parallel."""
with working_dir(self.build_directory):
make('test', parallel=False)
##############################################################################
# Copyright (c) 2013-2017, Lawrence Livermore National Security, LLC.
# Produced at the Lawrence Livermore National Laboratory.
#
# This file is part of Spack.
# Created by Todd Gamblin, tgamblin@llnl.gov, All rights reserved.
# LLNL-CODE-647188
#
# For details, see https://github.com/llnl/spack
# Please also see the NOTICE and LICENSE files for our notice and the LGPL.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU Lesser General Public License (as
# published by the Free Software Foundation) version 2.1, February 1999.
#
# This program is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the IMPLIED WARRANTY OF
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the terms and
# conditions of the GNU Lesser General Public License for more details.
#
# You should have received a copy of the GNU Lesser General Public
# License along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
##############################################################################
from spack import *
class HepnosDataloader(CMakePackage):
"""Dataloader for HEPnOS."""
homepage = "https://github.com/hepnos/HEPnOS-Dataloader"
url = "https://github.com/hepnos/HEPnOS-Dataloader"
git = "https://github.com/hepnos/HEPnOS-Dataloader.git"
version('develop', branch='master', submodules=True)
version('main', branch='main', submodules=True)
version('0.3', tag='v0.3', submodules=True)
version('0.2.1', tag='v0.2.1')
version('0.2', tag='v0.2')
version('0.1', tag='v0.1')
depends_on('cmake@3.9.0:', type=('build'))
depends_on('mpi')
depends_on('hepnos@0.3.2:')
depends_on('tclap')
depends_on('spdlog')
depends_on('hdf5')
depends_on('boost+serialization')
def cmake_args(self):
extra_args = ['-DBUILD_SHARED_LIBS=ON']
extra_args.extend(['-DCMAKE_CXX_COMPILER=%s' % self.spec['mpi'].mpicxx])
return extra_args
from spack import *
class HepnosPepBenchmark(CMakePackage):
"""Parallel event processing benchmark for HEPnOS."""
homepage = "https://github.com/hepnos/HEPnOS-PEP-Benchmark"
url = "https://github.com/hepnos/HEPnOS-PEP-Benchmark"
git = "https://github.com/hepnos/HEPnOS-PEP-Benchmark.git"
version('develop', branch='main', submodules=True)
version('main', branch='main', submodules=True)
version('0.1', branch='main', tag='v0.1', submodules=True, preferred=True)
depends_on('cmake@3.11.0:', type=('build'))
depends_on('mpi')
depends_on('hepnos@0.4:')
depends_on('tclap')
depends_on('spdlog')
##############################################################################
# Copyright (c) 2013-2017, Lawrence Livermore National Security, LLC.
# Produced at the Lawrence Livermore National Laboratory.
#
# This file is part of Spack.
# Created by Todd Gamblin, tgamblin@llnl.gov, All rights reserved.
# LLNL-CODE-647188
#
# For details, see https://github.com/llnl/spack
# Please also see the NOTICE and LICENSE files for our notice and the LGPL.
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU Lesser General Public License (as
# published by the Free Software Foundation) version 2.1, February 1999.
#
# This program is distributed in the hope that it will be useful, but
# WITHOUT ANY WARRANTY; without even the IMPLIED WARRANTY OF
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the terms and
# conditions of the GNU Lesser General Public License for more details.
#
# You should have received a copy of the GNU Lesser General Public
# License along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA
##############################################################################
from spack import *
class Hepnos(CMakePackage):
"""Object store for High Energy Physics, build around Mochi components"""
homepage = "https://github.com/hepnos/HEPnOS"
url = "https://github.com/hepnos/HEPnOS/archive/v0.4.2.tar.gz"
git = "https://github.com/hepnos/HEPnOS.git"
version('0.4.2', tag='v0.4.2')
version('0.4.1', tag='v0.4.1')
version('0.4.0', tag='v0.4.0')
version('0.3.2', tag='v0.3.2')
version('0.3.1', tag='v0.3.1')
version('0.3.0', tag='v0.3.0')
version('0.2.0', tag='v0.2.0')
version('0.1.8', tag='v0.1.8')
version('0.1.7', tag='v0.1.7')
version('0.1.6', tag='v0.1.6')
version('0.1.5', tag='v0.1.5')
version('0.1.4', tag='v0.1.4')
version('0.1.3', tag='v0.1.3')
version('0.1.2', tag='v0.1.2')
version('0.1.1', tag='v0.1.1')
version('0.1', tag='v0.1')
version('develop', branch='main')
version('main', branch='main')
variant('cxxstd',
default='14',
values=('14', '17'),
multi=False,
description='Use the specified C++ standard when building.')
depends_on('mpi')
depends_on('yaml-cpp@develop')
depends_on('boost+serialization')
depends_on('cmake@3.12.0:')
depends_on('libuuid', when='@0.2.0:')
# Mochi dependencies for non-develop versions
depends_on('mochi-ch-placement@0.1:')
depends_on('mochi-thallium@0.5.2:', when='@0.2.0:')
depends_on('mochi-margo@0.5.2:', when='@0.1.8') # past 0.1.8, HEPnOS requires thallium
depends_on('mochi-sdskv@0.1.10:', when='@0.3.1:')
depends_on('mochi-sdskv@0.1.8:', when='@0.2.0:')
depends_on('mochi-sdskv@0.1.7:', when='@0.1.8:')
depends_on('mochi-sdskv@0.1:')
depends_on('mochi-bake@0.1:0.3.6', when='@:0.1.7') # after 0.1.7, HEPnOS does not require bake
# Mochi dependencies for develop version
depends_on('mochi-ch-placement@develop', when='@develop')
depends_on('mochi-thallium@develop', when='@develop')
depends_on('mochi-sdskv@develop', when='@develop')
def cmake_args(self):
extra_args = ['-DBUILD_SHARED_LIBS=ON']
extra_args.extend(['-DCMAKE_CXX_COMPILER=%s' % self.spec['mpi'].mpicxx])
extra_args.extend(['-DCMAKE_CXX_STANDARD=%s' % self.spec.variants['cxxstd'].value])
return extra_args
From 0ee24e707fc2be9cf8f0e61ea2fe852d5272b782 Mon Sep 17 00:00:00 2001
From: Rob Latham <robl@mcs.anl.gov>
Date: Tue, 12 Feb 2019 09:54:32 -0600
Subject: [PATCH] DO NOT MERGE: mobject-specific hackery
---
src/Makefile.am | 1 -
src/aiori-RADOS.c | 2 +-
src/ior.h | 3 ++-
3 files changed, 3 insertions(+), 3 deletions(-)
diff --git a/src/Makefile.am b/src/Makefile.am
index 51fb8734951a..6f78a18bf79b 100755
--- a/src/Makefile.am
+++ b/src/Makefile.am
@@ -67,7 +67,6 @@ endif
if USE_RADOS_AIORI
extraSOURCES += aiori-RADOS.c
-extraLDADD += -lrados
endif
if USE_S3_AIORI
diff --git a/src/aiori-RADOS.c b/src/aiori-RADOS.c
index 090414d24aae..adec732cdad8 100755
--- a/src/aiori-RADOS.c
+++ b/src/aiori-RADOS.c
@@ -20,7 +20,7 @@
#include <stdio.h>
#include <stdlib.h>
#include <sys/stat.h>
-#include <rados/librados.h>
+#include <librados-mobject-store.h>
#include "ior.h"
#include "iordef.h"
diff --git a/src/ior.h b/src/ior.h
index ad543ffe5a03..fc37280608c4 100755
--- a/src/ior.h
+++ b/src/ior.h
@@ -30,7 +30,8 @@
#endif
#ifdef USE_RADOS_AIORI
-# include <rados/librados.h>
+#include <librados-mobject-store.h>
+
#else
typedef void *rados_t;
typedef void *rados_ioctx_t;
--
1.8.3.1