deploying

Deploying HEPnOS

Once HEPnOS is installed, you need to deploy it.

Creating a configuration file

The storage servers are initialized using a YAML configuration file. Here is an example of such a configuration file.

---
address: tcp://
threads: 4
database:
  name: hepnosdb
  path: /tmp/hepnos
  type: bdb
  providers: 1
  targets: 1
storage:
  path: /dev/shm/hepnos.dat
  size: 50
  providers: 1
  targets: 1

address is the protocol to be used by Mercury (see available protocols here)
database provides information on the database to use. The name of the database should not be changed. Its path must point to a directory local to the node that will run the service. The type must be either bdb or map. When map is used, however, the database will reside in memory and will not be persisted when shutting down HEPnOS. You will therefore loose data.
storage provides information on the backend storage to use. The path must point to a file on the node that will run the service. The file does not need to exist. The size field is the size of this file, in MB.

Upon starting for the first time, the HEPnOS daemon will create the required database at the provided path, as well as the required storage file. When restarting, HEPnOS will automatically use the existing databases and files if they exist. Thus all data written to HEPnOS is persistent.

Note: you need to manually cleanup (remove) the database and the data file if you wish to permanently delete HEPnOS's data.

Deploying HEPnOS on a single node

Simply ssh into the node where you want to run the HEPnOS service and type:

hepnos-daemon config.yaml client.yaml

This tells HEPnOS to start and configure itself using the config.yaml file (written before). It will generate a client.yaml file that can be used for clients to connect to it. The command will block. To run it as a daemon, use nohup or another other mechanism available on your platform.

Deploying HEPnOS on multiple nodes

The hepnos-daemon program is actually an MPI program that can be deployed on multiple nodes:

mpirun -np N -f hostfile hepnos-daemon config.yaml client.yaml

Replacing N with the number of nodes and hostfile with the name of a file containing the list of hosts on which to deploy HEPnOS.

Once again, this can be daemonized using standard tools like nohup.

Warning: you may be tempted to try running multiple MPI ranks on the same node. With the configuration file written above, this will not work, since multiple ranks will try to access the same database and data files. To make it work, you need to request each MPI rank to manage its own database and its own data file. This can be done by using the $RANK variable in the configuration file. For example:

---
address: tcp://
threads: 4
database:
  name: hepnosdb
  path: /tmp/hepnos/$RANK
  type: bdb
  providers: 1
  targets: 1
storage:
  path: /dev/shm/hepnos.$RANK.dat
  size: 50
  providers: 1
  targets: 1

Using multiple providers and/or multiple targets

In the database and storage entries of the YAML file you will find optional providers and targets entries. These represent how many providers should be deployed on each node, and how many targets (database instances in the case of the database section, backend storage file for the storage section) should be created and used by each provider.

Should you want to use more than 1 provider and/or more than 1 target per provider, you will need to use the $PROVIDER and $TARGET placeholders in the name and path of your database and storage file. For example:

---
address: tcp://
threads: 4
database:
  name: hepnosdb.$RANK.$PROVIDER.$TARGET
  path: /tmp/hepnos/$RANK_$PROVIDER_$TARGET
  type: bdb
  providers: 2
  targets: 2
storage:
  path: /dev/shm/hepnos.$RANK.$PROVIDER.$TARGET.dat
  size: 50
  providers: 2
  targets: 2