README.md 2.69 KB
Newer Older
Philip Carns's avatar
Philip Carns committed
1 2
# bake-bulk

Philip Carns's avatar
Philip Carns committed
3
## Dependencies
4

Philip Carns's avatar
Philip Carns committed
5 6 7
* uuid (install uuid-dev package on ubuntu)
* NVML/libpmem (see instructions below)
* margo (see instructions at https://xgitlab.cels.anl.gov/sds/margo)
8

Philip Carns's avatar
Philip Carns committed
9
You can compile and install the latest git revision of NVML as follows:
Philip Carns's avatar
Philip Carns committed
10

Philip Carns's avatar
Philip Carns committed
11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80
* `git clone https://github.com/pmem/nvml.git`
* `cd nvml`
* `make`
* `make install prefix=/home/carns/working/install/`

## Compilation

* `./prepare.sh`
* `mkdir build`
* `cd build`
* `../configure --prefix=/home/carns/working/install`
* `make`

If any dependencies are installed in a nonstandard location, then
modify the configure step listed above to include the following argument:

* `PKG_CONFIG_PATH=/home/carns/working/install/lib/pkgconfig`


## Server daemon execution example (using tmpfs memory as backing store)

* `truncate -s 500M /dev/shm/foo.dat`
* `pmempool create obj /dev/shm/foo.dat`
* `bake-bulk-server sm://1/1 /dev/shm/foo.dat`

### Explanation

The truncate command creates an empty 500 MiB file in /dev/shm,
which will act as a ramdisk for storage in this case.  You can skip this step
if you are using a true NVRAM device file.

The pmempool command formats the storage device as a pmem target for
libpmemobj.

The bake-bulk-server command starts the server daemon.  

The first argument to bake-bulk-server is
the address for Mercury to listen on.  In this case we are using the CCI/SM
transport, and the "1/1" portion of the URL specifies the last two components
of the /tmp path to use for SM connection information. For other transports
this would more likely just be a port number.

The second argument to bake-bulk-server is the path to the libpmem-formatted
storage device.

## Benchmark execution example

* `./bb-latency-bench sm:///tmp/cci/sm/carns-x1/1/1 100000 4 8`

This example runs a sequence of latency benchmarks.  Other bb- utilities
installed with bake-bulk will perform other rudimentary operations.

The first argument is the address of the server.  We are using CCI/SM in this
case, which means that the URL is a path to the connection information of the
server in /tmp.  The base (not changeable) portion of the path is
/tmp/cci/HOSTNAME.  The remainder of the path was specified by the server
daemon when it started.

The second argument is the number of benchmark iterations.

The third and fourth arguments specify the range of sizes to use for read and
write operations in the benchmark.

## Misc tips

Memory allocation seems to account for a significant portion of
the latency as of this writing.  The tcmalloc library will lower and
stabilize latency somewhat as a partial short-term solution.  On ubuntu
you can use tcmalloc by running this in the terminal before the server
and client commands:
Philip Carns's avatar
Philip Carns committed
81 82

export LD_PRELOAD=/usr/lib/libtcmalloc.so.4