Once HEPnOS is installed, you need to deploy it.
Creating a configuration file
The storage servers are initialized using a YAML configuration file. Here is an example of such a configuration file.
--- address: tcp:// threads: 4 database: name: hepnosdb path: /tmp/hepnos type: bdb providers: 1 targets: 1 storage: path: /dev/shm/hepnos.dat size: 50 providers: 1 targets: 1
addressis the protocol to be used by Mercury (see available protocols here)
databaseprovides information on the database to use. The
nameof the database should not be changed. Its
pathmust point to a directory local to the node that will run the service. The
typemust be either
mapis used, however, the database will reside in memory and will not be persisted when shutting down HEPnOS. You will therefore loose data.
storageprovides information on the backend storage to use. The
pathmust point to a file on the node that will run the service. The file does not need to exist. The
sizefield is the size of this file, in MB.
Upon starting for the first time, the HEPnOS daemon will create the required database at the provided path, as well as the required storage file. When restarting, HEPnOS will automatically use the existing databases and files if they exist. Thus all data written to HEPnOS is persistent.
Note: you need to manually cleanup (remove) the database and the data file if you wish to permanently delete HEPnOS's data.
Deploying HEPnOS on a single node
Simply ssh into the node where you want to run the HEPnOS service and type:
hepnos-daemon config.yaml client.yaml
This tells HEPnOS to start and configure itself using the config.yaml file (written before). It will generate a client.yaml file that can be used for clients to connect to it. The command will block. To run it as a daemon, use
nohup or another other mechanism available on your platform.
Deploying HEPnOS on multiple nodes
hepnos-daemon program is actually an MPI program that can be deployed on multiple nodes:
mpirun -np N -f hostfile hepnos-daemon config.yaml client.yaml
N with the number of nodes and
hostfile with the name of a file containing the list of hosts on which to deploy HEPnOS.
Once again, this can be daemonized using standard tools like
Warning: you may be tempted to try running multiple MPI ranks on the same node. With the configuration file written above, this will not work, since multiple ranks will try to access the same database and data files. To make it work, you need to request each MPI rank to manage its own database and its own data file. This can be done by using the
$RANK variable in the configuration file. For example:
--- address: tcp:// threads: 4 database: name: hepnosdb path: /tmp/hepnos/$RANK type: bdb providers: 1 targets: 1 storage: path: /dev/shm/hepnos.$RANK.dat size: 50 providers: 1 targets: 1
Using multiple providers and/or multiple targets
storage entries of the YAML file you will find optional
targets entries. These represent how many providers should be deployed on each node, and how many targets (database instances in the case of the database section, backend storage file for the storage section) should be created and used by each provider.
Should you want to use more than 1 provider and/or more than 1 target per provider, you will need to use the
$TARGET placeholders in the name and path of your database and storage file. For example:
address: tcp:// threads: 4 database: name: hepnosdb.$RANK.$PROVIDER.$TARGET path: /tmp/hepnos/$RANK_$PROVIDER_$TARGET type: bdb providers: 2 targets: 2 storage: path: /dev/shm/hepnos.$RANK.$PROVIDER.$TARGET.dat size: 50 providers: 2 targets: 2