Commit 5a33bee9 authored by Matthieu Dorier's avatar Matthieu Dorier

Merge branch 'master' of xgitlab.cels.anl.gov:sds/bake

parents 4a0c8736 67fa2909
# BAKE
## Dependencies
## Installation
* uuid (install uuid-dev package on ubuntu)
* NVML/libpmem (see instructions below)
* margo (see instructions at https://xgitlab.cels.anl.gov/sds/margo)
BAKE can easily be installed using Spack:
You can compile and install the latest git revision of NVML as follows:
`spack install bake`
* `git clone https://github.com/pmem/nvml.git`
* `cd nvml`
* `make`
* `make install prefix=/home/carns/working/install/`
This will install BAKE and its dependencies (margo, uuid, and libpmem,
as well as their respective dependencies). To compile and install BAKE manually,
please refer to the end of this document.
`make install` will require `pandoc` or you can ignore the error and not build
the documentation.
## Architecture
## Compilation
Like most Mochi services, BAKE relies on a client/provider architecture.
A provider, identified by its _address_ and _multiplex id_, manages one or more
_BAKE targets_, referenced externally by their _target id_.
* `./prepare.sh`
* `mkdir build`
* `cd build`
* `../configure --prefix=/home/carns/working/install`
* `make`
If any dependencies are installed in a nonstandard location, then
modify the configure step listed above to include the following argument:
* `PKG_CONFIG_PATH=/home/carns/working/install/lib/pkgconfig`
## Setting up a BAKE target
BAKE requires the backend storage file to be created beforehand using
`bake-mkpool`. For instance:
## Server daemon execution example (using tmpfs memory as backing store)
`bake-mkpool -s 500M /dev/shm/foo.dat`
* `bake-mkpool -s 500M /dev/shm/foo.dat`
* `bake-server-daemon sm://1/1 /dev/shm/foo.dat`
### Explanation
creates a 500 MB file at _/dev/shm/foo.dat_ to be used by BAKE as a target.
The bake-mkpool command creates a BAKE pool used to store raw data for
a particular BAKE target. This is essentially a wrapper command around
pmemobj utilities for creating an empty pool that additionally store
some BAKE-specific metadata in the created pool. Pools used by the BAKE
server must be created using this command.
The bake-server-daemon command starts the server daemon.
The first argument to bake-server-daemon is the address for Mercury to
listen on. In this case we are using the CCI/SM transport. For other
transports this would more likely just be an address and port number
(e.g. "tcp://localhost:1234"). CCI/SM endpoints are identified by two
integers. By default ("sm://"), the first integer is the server's process
id and the second is 0. The defaults can be overriden on the command
line (e.g. "sm://1/1" fixes the listening address to 1/1). CCI/SM uses
the integers in the address in conjunction with the server's hostname
to create subdirectories in /tmp/cci/sm for IPC connection information.
CCI/SM will create all necessary subdirectories in /tmp/cci. For example,
if the command is run on host "carns-x1" with "sm://1/1" then CCI/SM
will create a /tmp/cci/sm/carns-x1/1/1 directory containing connection
information for the bake-server-daemon process.
The second argument to bake-server-daemon is the path to the BAKE pool
originally created with bake-mkpool.
pmemobj utilities for creating an empty pool that additionally stores
some BAKE-specific metadata in the created pool. Pools used by BAKE
providers must be created using this command.
## Starting a daemon
BAKE ships with a default daemon program that can setup providers and attach
to storage targets. This daemon can be started as follows:
`bake-server-daemon [options] <listen_address> <bake_pool_1> <bake_pool_2> ...`
The program takes a set of options followed by an address at which to listen for
incoming RPCs, and a list of
BAKE targets already created using `bake-mkpool`.
For example:
`bake-server-daemon -f bake.addr -m providers bmi+tcp://localhost:1234 /dev/shm/foo.dat /dev/shm/bar.dat`
The following options are accepted:
* `-f` provides the name of the file in which to write the address of the daemon.
* `-m` provides the mode (_providers_ or _targets_).
The _providers_ mode indicates that, if multiple BAKE targets are used (as above),
these targets should be managed by multiple providers, accessible through
different multiplex ids 1, 2, ... _N_ where _N_ is the number of storage targets
to manage. The _targets_ mode indicates that a single provider should be used to
manage all the storage targets.
## Client API example
```c
#include <bake-client.h>
int main(int argc, char **argv)
{
char *svr_addr_str; // string address of the BAKE server
hg_addr_t svr_addr; // Mercury address of the BAKE server
margo_instance_id mid; // Margo instance id
bake_client_t bcl; // BAKE client
bake_provider_handle_t bph; // BAKE handle to provider
uint8_t mplex_id; // multiplex id of the provider
uint32_t target_number; // target to use
bake_region_id_t rid; // BAKE region id handle
bake_target_id_t* bti; // array of target ids
/* ... setup variables ... */
/* Initialize Margo */
mid = margo_init(..., MARGO_CLIENT_MODE, 0, -1);
/* Lookup the server */
margo_addr_lookup(mid, svr_addr_str, &svr_addr);
/* Creates the BAKE client */
bake_client_init(mid, &bcl);
/* Creates the provider handle */
bake_provider_handle_create(bcl, svr_addr, mplex_id, &bph);
/* Asks the provider for up to target_number target ids */
uint32_t num_targets = 0;
bti = calloc(num_targets, sizeof(*bti));
bake_probe(bph, target_number, bti, &num_targets);
if(num_targets < target_number) {
fprintf(stderr, "Error: provider has only %d storage targets\n", num_targets);
}
/* Create a region */
size_t size = ...; // size of the region to create
bake_create(bph, bti[target_number-1], size, &rid);
/* Write data into the region at offset 0 */
char* buf = ...;
bake_write(bph, rid, 0, buf, size);
/* Make all modifications persistent */
bake_persist(bph, rid);
/* Get size of region */
size_t check_size;
bake_get_size(bph, rid, &check_size);
/* Release provider handle */
bake_provider_handle_release(bph);
/* Release BAKE client */
bake_client_finalize(bcl);
/* Cleanup Margo resources */
margo_addr_free(mid, svr_addr);
margo_finalize(mid);
return 0;
}
```
Note that a `bake_region_id_t` object can be written (into a file or a socket)
and stored or sent to another program. These region ids are what uniquely
reference a region within a given target.
The rest of the client-side API can be found in `bake-client.h`.
## Provider API
The bake-server-daemon source is a good example of how to create providers and
attach storage targets to them. The provider-side API is located in
_bake-server.h_, and consists of mainly two functions:
```c
int bake_provider_register(
margo_instance_id mid,
uint8_t mplex_id,
ABT_pool pool,
bake_provider_t* provider);
```
This creates a provider at the given multiplex id, using a given Argobots pool.
```c
int bake_provider_add_storage_target(
bake_provider_t provider,
const char *target_name,
bake_target_id_t* target_id);
```
This makes the provider manage the given storage target.
Other functions are available to remove a storage target (or all storage
targets) from a provider.
## Benchmark execution example
......@@ -89,3 +173,34 @@ you can use tcmalloc by running this in the terminal before the server
and client commands:
export LD_PRELOAD=/usr/lib/libtcmalloc.so.4
## Manual installation
BAKE depends on the following libraries:
* uuid (install uuid-dev package on ubuntu)
* NVML/libpmem (see instructions below)
* margo (see instructions [here](https://xgitlab.cels.anl.gov/sds/margo))
Yu can compile and install the latest git revision of NVML as follows:
* `git clone https://github.com/pmem/nvml.git`
* `cd nvml`
* `make`
* `make install prefix=/home/carns/working/install/`
`make install` will require `pandoc` or you can ignore the error and not build
the documentation.
To compile BAKE:
* `./prepare.sh`
* `mkdir build`
* `cd build`
* `../configure --prefix=/home/carns/working/install`
* `make`
If any dependencies are installed in a nonstandard location, then
modify the configure step listed above to include the following argument:
* `PKG_CONFIG_PATH=/home/carns/working/install/lib/pkgconfig`
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment