Commit 9b10d775 authored by rjzamora's avatar rjzamora
Browse files

adjusting documentation now that the CCIO branch is in Bitbucket

parent 6264717d
......@@ -5,11 +5,11 @@
These are instructions for building the parallel HDF5 exerciser, and
running a basic exerciser test on the IBM BG/Q Vesta machine mach(ALCF). Similar steps can be used for other BG/Q systems (e.g. ALCF Mira).
I am also including instructions for building HDF5 (CCIO and `develop` versions), because it is likely that you may need to do these things together. Feel free to skip the HDF5-build steps if they don't apply to you.
I am also including instructions for building HDF5 (the CCIO development branch), because it is likely that you may need to do these things together. Feel free to skip the HDF5-build steps if they don't apply to you.
### Setting up the Directory Structure
For all instructions in this document, we assume the directory structure defined in this section. This structure is not necessary for the exerciser code to function correctly, but it will allow you to follow the instructions as closely as possible. The structure assumes that you will be building the `develop` and/or `CCIO` versions of HDF5 yourself within the defined structure. This is not required (you can simply skip the HDF5 build instructions, and use a different `HDF5_INSTALL_DIR` when you build the exerciser).
For all instructions in this document, we assume the directory structure defined in this section. This structure is not necessary for the exerciser code to function correctly, but it will allow you to follow the instructions as closely as possible. The structure assumes that you will be building the `CCIO` version of HDF5 yourself within the defined structure. This is not required (you can simply skip the HDF5 build instructions, and use a different `HDF5_INSTALL_DIR` when you build the exerciser).
First, define the root directory for building HDF5 and the Exerciser:
......@@ -27,7 +27,7 @@ mkdir library
mkdir repos
```
Clone the necessary git repositories. Note that the Custom Collective I/O (CCIO) version of the HDF5 is under development for the ExaHDF5 project (see: https://github.com/rjzamora/hdf5-ccio-develop/tree/ccio). The official development branch of HDF5 (which CCIO is a *fork* of) is located on Bitbucket (https://bitbucket.hdfgroup.org/projects/HDFFV/repos/hdf5/browse). You can use either/both of these HDF5 versions (or another one).
Clone the necessary git repositories. Note that the Custom Collective I/O (CCIO) version of the HDF5 is under development for the ExaHDF5 project. The code is currently in the `ccio` branch of the official HDF5 development repository (see: https://bitbucket.hdfgroup.org/projects/HDFFV/repos/hdf5/browse).
First, clone the repo with the Exerciser (if you already did this, just move the repo to this location):
......@@ -39,18 +39,12 @@ git clone git@xgitlab.cels.anl.gov:ExaHDF5/BuildAndTest.git
If using CCIO, clone it (be sure to use the 'ccio' branch of hdf5-ccio-develop):
```
git clone https://github.com/rjzamora/hdf5-ccio-develop.git
cd hdf5-ccio-develop
git clone https://bitbucket.hdfgroup.org/scm/hdffv/hdf5.git
cd hdf5
git checkout ccio
cd ..
```
If using `develop`, clone it:
```
git clone https://bitbucket.hdfgroup.org/scm/hdffv/hdf5.git
```
Create the rest of the directory structure for this example:
```
......@@ -59,10 +53,8 @@ mkdir build
mkdir install
cd install
mkdir ccio
mkdir develop
cd ../build
mkdir ccio
mkdir develop
```
......@@ -72,7 +64,7 @@ Here, we are using the `ccio` directories to build and install the code, along w
```
cd $HDF5_ROOT/library/build/ccio
cp -rL $HDF5_ROOT/xgitlabrepos/hdf5-ccio-develop/* .
cp -rL $HDF5_ROOT/xgitlabrepos/hdf5/* .
cp $HDF5_ROOT/xgitlabrepos/BuildAndTest/Exerciser/BGQ/VESTA_CH4/do-configure-debug .
cp $HDF5_ROOT/xgitlabrepos/BuildAndTest/Exerciser/BGQ/VESTA_CH4/do-make .
cp $HDF5_ROOT/xgitlabrepos/BuildAndTest/Exerciser/BGQ/VESTA_CH4/do-mpirun .
......@@ -114,33 +106,6 @@ Once the HDF5 library is built, it should be in `$HDF5_ROOT/library/install/ccio
To build the exerciser agains the CCIO version of HDF5, you will need to use the `ccio` installation location of HDF5 in the example below (by setting `HDF5_INSTALL_DIR=$HDF5_ROOT/library/install/ccio`).
### (If Desired) Building the `develop` Branch of HDF5
The steps to build `develop` are the same as the steps to build CCIO. However, you now use the `develop` directories to build and install the code.
```
cd $HDF5_ROOT/library/build/develop
cp -rL $HDF5_ROOT/xgitlabrepos/hdf5/* .
cp $HDF5_ROOT/xgitlabrepos/BuildAndTest/Exerciser/BGQ/VESTA_CH4/do-configure-debug .
cp $HDF5_ROOT/xgitlabrepos/BuildAndTest/Exerciser/BGQ/VESTA_CH4/do-make .
cp $HDF5_ROOT/xgitlabrepos/BuildAndTest/Exerciser/BGQ/VESTA_CH4/do-mpirun .
```
You will need to modify the following line in `do-configure` to point to the `develop` locations (make sure the `configure` and `--prefix` paths are correct):
```
$HDF5_ROOT/library/build/develop/configure --without-pthread --disable-shared --enable-fortran \
--disable-cxx --enable-parallel --enable-symbols=yes \
--enable-build-mode=production --enable-optimization=high \
--with-zlib=/soft/libraries/alcf/current/gcc/ZLIB \
--prefix=$HDF5_ROOT/library/install/develop \
2>&1
```
Besides these changes, the instructions should be very similar to those for CCIO. To build the exerciser agains the `develop` branch, you will need to use the `develop` installation location of HDF5 in the example below (by setting `HDF5_INSTALL_DIR=$HDF5_ROOT/library/install/develop`).
### Building the Exerciser
The specific instructions here will assume that you have used the same directory structure as the optional instructions for using the CCIO branch of HDF5 (above). However, the makefile example can be used with any HDF5 installation location (`HDF5_INSTALL_DIR`).
......
## Building and Running the Exerciser on Vesta BG/Q (ALCF)
These are instructions for building the parallel HDF5 exerciser, and
running a basic exerciser test on the IBM BG/Q Vesta machine mach(ALCF). Similar steps can be used for other BG/Q systems (e.g. ALCF Mira).
running a basic exerciser test on the IBM BG/Q Vesta machine mach (ALCF). Similar steps can be used for other BG/Q systems (e.g. ALCF Mira).
I am also including instructions for building HDF5 (CCIO and `develop` versions), because it is likely that you may need to do these things together. Feel free to skip the HDF5-build steps if they don't apply to you.
I am also including instructions for building HDF5 (the CCIO development branch), because it is likely that you may need to do these things together. Feel free to skip the HDF5-build steps if they don't apply to you.
### Setting up the Directory Structure
For all instructions in this document, we assume the directory structure defined in this section. This structure is not necessary for the exerciser code to function correctly, but it will allow you to follow the instructions as closely as possible. The structure assumes that you will be building the `develop` and/or `CCIO` versions of HDF5 yourself within the defined structure. This is not required (you can simply skip the HDF5 build instructions, and use a different `HDF5_INSTALL_DIR` when you build the exerciser).
For all instructions in this document, we assume the directory structure defined in this section. This structure is not necessary for the exerciser code to function correctly, but it will allow you to follow the instructions as closely as possible. The structure assumes that you will be building the `CCIO` version of HDF5 yourself within the defined structure. This is not required (you can simply skip the HDF5 build instructions, and use a different `HDF5_INSTALL_DIR` when you build the exerciser).
First, define the root directory for building HDF5 and the Exerciser:
......@@ -25,7 +25,7 @@ mkdir library
mkdir repos
```
Clone the necessary git repositories. Note that the Custom Collective I/O (CCIO) version of the HDF5 is under development for the ExaHDF5 project (see: https://github.com/rjzamora/hdf5-ccio-develop/tree/ccio). The official development branch of HDF5 (which CCIO is a *fork* of) is located on Bitbucket (https://bitbucket.hdfgroup.org/projects/HDFFV/repos/hdf5/browse). You can use either/both of these HDF5 versions (or another one).
Clone the necessary git repositories. Note that the Custom Collective I/O (CCIO) version of the HDF5 is under development for the ExaHDF5 project. The code is currently in the `ccio` branch of the official HDF5 development repository (see: https://bitbucket.hdfgroup.org/projects/HDFFV/repos/hdf5/browse).
First, clone the repo with the Exerciser (if you already did this, just move the repo to this location):
......@@ -34,21 +34,15 @@ cd gitrepos
git clone git@xgitlab.cels.anl.gov:ExaHDF5/BuildAndTest.git
```
If using CCIO, clone it (be sure to use the 'ccio' branch of hdf5-ccio-develop):
If using CCIO, clone it (be sure to use the 'ccio' branch of HDF5):
```
git clone https://github.com/rjzamora/hdf5-ccio-develop.git
cd hdf5-ccio-develop
git clone https://bitbucket.hdfgroup.org/scm/hdffv/hdf5.git
cd hdf5
git checkout ccio
cd ..
```
If using `develop`, clone it:
```
git clone https://bitbucket.hdfgroup.org/scm/hdffv/hdf5.git
```
Create the rest of the directory structure for this example:
```
......@@ -57,10 +51,8 @@ mkdir build
mkdir install
cd install
mkdir ccio
mkdir develop
cd ../build
mkdir ccio
mkdir develop
```
......@@ -70,7 +62,7 @@ Here, we are using the `ccio` directories to build and install the code:
```
cd $HDF5_ROOT/library/build/ccio
cp -rL $HDF5_ROOT/xgitlabrepos/hdf5-ccio-develop/* .
cp -rL $HDF5_ROOT/xgitlabrepos/hdf5/* .
cp $HDF5_ROOT/xgitlabrepos/BuildAndTest/Exerciser/BGQ/VESTA_XL/do-configure .
cp $HDF5_ROOT/xgitlabrepos/BuildAndTest/Exerciser/BGQ/VESTA_XL/do-make .
cp $HDF5_ROOT/xgitlabrepos/BuildAndTest/Exerciser/BGQ/VESTA_XL/do-mpirun .
......@@ -110,33 +102,6 @@ Once the HDF5 library is built, it should be in `$HDF5_ROOT/library/install/ccio
To build the exerciser agains the CCIO version of HDF5, you will need to use the `ccio` installation location of HDF5 in the example below (by setting `HDF5_INSTALL_DIR=$HDF5_ROOT/library/install/ccio`).
### (If Desired) Building the `develop` Branch of HDF5
The steps to build `develop` are the same as the steps to build CCIO. However, you now use the `develop` directories to build and install the code.
```
cd $HDF5_ROOT/library/build/develop
cp -rL $HDF5_ROOT/xgitlabrepos/hdf5/* .
cp $HDF5_ROOT/xgitlabrepos/BuildAndTest/Exerciser/BGQ/VESTA_XL/do-configure .
cp $HDF5_ROOT/xgitlabrepos/BuildAndTest/Exerciser/BGQ/VESTA_XL/do-make .
cp $HDF5_ROOT/xgitlabrepos/BuildAndTest/Exerciser/BGQ/VESTA_XL/do-mpirun .
```
You will need to modify the following line in `do-configure` to point to the `develop` locations (make sure the `configure` and `--prefix` paths are correct):
```
$HDF5_ROOT/library/build/develop/configure --without-pthread --disable-shared --enable-fortran \
--disable-cxx --enable-parallel \
--with-zlib=/soft/libraries/alcf/current/xl/ZLIB \
--prefix=$HDF5_ROOT/library/install/develop \
2>&1
```
Besides these changes, the instructions should be very similar to those for CCIO. To build the exerciser agains the `develop` branch, you will need to use the `develop` installation location of HDF5 in the example below (by setting `HDF5_INSTALL_DIR=$HDF5_ROOT/library/install/develop`).
### Building the Exerciser
The specific instructions here will assume that you have used the same directory structure as the optional instructions for using the CCIO branch of HDF5 (above). However, the makefile example can be used with any HDF5 installation location (`HDF5_INSTALL_DIR`).
......
......@@ -3,11 +3,11 @@
These are instructions for building the parallel HDF5 exerciser, and
running a basic exerciser test on the Cray XC40 Theta machine (at ALCF).
I am also including instructions for building HDF5 (CCIO and `develop` versions), because it is likely that you may need to do these things together. Feel free to skip the HDF5-build steps if they don't apply to you.
I am also including instructions for building HDF5 (the CCIO development branch), because it is likely that you may need to do these things together. Feel free to skip the HDF5-build steps if they don't apply to you.
### Setting up the Directory Structure
For all instructions in this document, we assume the directory structure defined in this section. This structure is not necessary for the exerciser code to function correctly, but it will allow you to follow the instructions as closely as possible. The structure assumes that you will be building the `develop` and/or `CCIO` versions of HDF5 yourself within the defined structure. This is not required (you can simply skip the HDF5 build instructions, and use a different `HDF5_INSTALL_DIR` when you build the exerciser).
For all instructions in this document, we assume the directory structure defined in this section. This structure is not For all instructions in this document, we assume the directory structure defined in this section. This structure is not necessary for the exerciser code to function correctly, but it will allow you to follow the instructions as closely as possible. The structure assumes that you will be building the `CCIO` version of HDF5 yourself within the defined structure. This is not required (you can simply skip the HDF5 build instructions, and use a different `HDF5_INSTALL_DIR` when you build the exerciser).
First, define the root directory for building HDF5 and the Exerciser:
......@@ -25,7 +25,7 @@ mkdir library
mkdir repos
```
Clone the necessary git repositories. Note that the Custom Collective I/O (CCIO) version of the HDF5 is under development for the ExaHDF5 project (see: https://github.com/rjzamora/hdf5-ccio-develop/tree/ccio). The official development branch of HDF5 (which CCIO is a *fork* of) is located on Bitbucket (https://bitbucket.hdfgroup.org/projects/HDFFV/repos/hdf5/browse). You can use either/both of these HDF5 versions (or another one).
Clone the necessary git repositories. Note that the Custom Collective I/O (CCIO) version of the HDF5 is under development for the ExaHDF5 project. The code is currently in the `ccio` branch of the official HDF5 development repository (see: https://bitbucket.hdfgroup.org/projects/HDFFV/repos/hdf5/browse).
First, clone the repo with the Exerciser (if you already did this, just move the repo to this location):
......@@ -34,21 +34,15 @@ cd gitrepos
git clone git@xgitlab.cels.anl.gov:ExaHDF5/BuildAndTest.git
```
If using CCIO, clone it (be sure to use the 'ccio' branch of hdf5-ccio-develop):
If using CCIO, clone it (be sure to use the 'ccio' branch of HDF5):
```
git clone https://github.com/rjzamora/hdf5-ccio-develop.git
cd hdf5-ccio-develop
git clone https://bitbucket.hdfgroup.org/scm/hdffv/hdf5.git
cd hdf5
git checkout ccio
cd ..
```
If using `develop`, clone it:
```
git clone https://bitbucket.hdfgroup.org/scm/hdffv/hdf5.git
```
Create the rest of the directory structure for this example:
```
......@@ -57,10 +51,8 @@ mkdir build
mkdir install
cd install
mkdir ccio
mkdir develop
cd ../build
mkdir ccio
mkdir develop
```
......@@ -103,23 +95,6 @@ Once the HDF5 library is built, it should be in `$HDF5_ROOT/library/install/ccio
To build the exerciser agains the CCIO version of HDF5, you will need to use the `ccio` installation location of HDF5 in the example below (by setting `HDF5_INSTALL_DIR=$HDF5_ROOT/library/install/ccio`).
### (If Desired) Building the `develop` Branch of HDF5
The steps to build `develop` are the same as the steps to build CCIO. However, you now use the `develop` directories to build and install the code.
```
cd $HDF5_ROOT/library/build/develop
```
Now, it is time to run the configuration step. Make sure the following line is correct for your installation (make sure the `configure` and `--prefix` paths are correct):
```
CC=cc CFLAGS='-O3 -DTHETA -Dtopo_timing -Dtopo_debug' $HDF5_ROOT/library/build/develop/configure --enable-parallel --enable-build-mode=production --enable-symbols=yes --prefix=$HDF5ROOT/library/install/develop
```
Besides these changes, the instructions should be very similar to those for CCIO. To build the exerciser agains the `develop` branch, you will need to use the `develop` installation location of HDF5 in the example below (by setting `HDF5_INSTALL_DIR=$HDF5_ROOT/library/install/develop`).
### Building the Exerciser
......
......@@ -7,7 +7,7 @@
**Overview:**
This repository is intended for documentation and benchmarks for the HDF5 librarby. The code/documentation in here has been collected for the ExaHDF5 SCP Software and Technology project.
This repository is intended for documentation and benchmarks for the HDF5 library. The code/documentation in here has been collected through the ExaHDF5 ECP Software and Technology project. The Exerciser benchmark was specifically designed to aid the development of a new Custom-Collective I/O (CCIO) MPIO VFD. For this reason, some build/run instructions will directly refer to the CCIO version of HDF5.
**Current Benchmarks:**
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment