Commit da178aee authored by Philip Carns's avatar Philip Carns

update doc for na+sm and renamed examples

parent 1523af45
# margo # Margo
margo is a library that provides Argobots bindings to the Mercury RPC Margo is a library that provides Argobots bindings to the Mercury RPC
implementation. See the following for more details about each project: implementation. See the following for more details about each project:
* https://collab.mcs.anl.gov/display/ARGOBOTS/Argobots+Home * https://collab.mcs.anl.gov/display/ARGOBOTS/Argobots+Home
* https://mercury-hpc.github.io/ * https://mercury-hpc.github.io/
Note that Margo should be compatible with any Mercury transport (NA plugin), Note that Margo should be compatible with any Mercury transport (NA plugin). The documentation assumes the use of the NA SM (shared memory) plugin that is built into Mercury for simplicity. This plugin is only valid for communication between processes on a single node. See [Using Margo with other Mercury NA plugins](##using-margo-with-other-mercury-na-plugins) for information on other configuration options.
but we assume the use of CCI in all examples here.
## Dependencies ## Dependencies
* cci (git clone https://github.com/CCI/cci.git)
* mercury (git clone --recurse-submodules https://github.com/mercury-hpc/mercury.git) * mercury (git clone --recurse-submodules https://github.com/mercury-hpc/mercury.git)
* argobots (git clone https://github.com/pmodels/argobots.git) * argobots (git clone https://github.com/pmodels/argobots.git)
* abt-snoozer (git clone https://xgitlab.cels.anl.gov/sds/abt-snoozer) * abt-snoozer (git clone https://xgitlab.cels.anl.gov/sds/abt-snoozer)
...@@ -18,8 +16,6 @@ but we assume the use of CCI in all examples here. ...@@ -18,8 +16,6 @@ but we assume the use of CCI in all examples here.
### Recommended Mercury build options ### Recommended Mercury build options
* -DNA_CCI_USE_POLL:BOOL=ON enables the "poll" feature of CCI that will allow
the na_cci plugin to make progress without busy spinning
* -DMERCURY_USE_CHECKSUMS:BOOL=OFF disables automatic checksumming of all * -DMERCURY_USE_CHECKSUMS:BOOL=OFF disables automatic checksumming of all
Mercury RPC messages. This reduces latency by removing a layer of Mercury RPC messages. This reduces latency by removing a layer of
integrity checking on communication. integrity checking on communication.
...@@ -32,7 +28,53 @@ Example configuration: ...@@ -32,7 +28,53 @@ Example configuration:
PKG_CONFIG_PATH=/home/pcarns/working/install/lib/pkgconfig \ PKG_CONFIG_PATH=/home/pcarns/working/install/lib/pkgconfig \
CFLAGS="-g -Wall" CFLAGS="-g -Wall"
## Setting up a CCI environment ## Running examples
The examples subdirectory contains:
* margo-example-client.c: an example client
* margo-example-server.c: an example server
* my-rpc.[ch]: an example RPC definition
The following example shows how to execute them. Note that when the server starts it will display the address that the client can use to connect to it.
```
$ examples/margo-example-server na+sm://
# accepting RPCs on address "na+sm://13367/0"
Got RPC request with input_val: 0
Got RPC request with input_val: 1
Got RPC request with input_val: 2
Got RPC request with input_val: 3
Got RPC request to shutdown
$ examples/margo-example-client na+sm://13367/0
ULT [0] running.
ULT [1] running.
ULT [2] running.
ULT [3] running.
Got response ret: 0
ULT [0] done.
Got response ret: 0
ULT [1] done.
Got response ret: 0
ULT [2] done.
Got response ret: 0
ULT [3] done.
```
The client will issue 4 concurrent RPCs to the server and wait for them to
complete.
## Running tests
`make check`
## Using Margo with the other NA plugins
You can use either the CCI NA plugin or BMI NA plugin to use either the CCI or BMI library for remote communication. See the [Mercury documentation](http://mercury-hpc.github.io/documentation/) for details and status.
### CCI
You must set the CCI_CONFIG environment variable to point to a valid CCI You must set the CCI_CONFIG environment variable to point to a valid CCI
configuration file. You can use the following example and un-comment the configuration file. You can use the following example and un-comment the
...@@ -57,43 +99,19 @@ interface = lo # switch this to eth0 or an external hostname for non-localhost ...@@ -57,43 +99,19 @@ interface = lo # switch this to eth0 or an external hostname for non-localhost
You must then use addresses appropriate for your transport at run time when You must then use addresses appropriate for your transport at run time when
executing Margo examples. Examples for server "listening" addresses: executing Margo examples. Examples for server "listening" addresses:
* tcp://3344 # for TCP/IP, listening on port 3344 * cci+tcp://3344 # for TCP/IP, listening on port 3344
* verbs://3344 # for InfiniBand, listening on port 3344 * cci+verbs://3344 # for InfiniBand, listening on port 3344
* sm://1/1 # for shared memory, listening on CCI SM address 1/1 * cci+sm://1/1 # for shared memory, listening on CCI SM address 1/1
Examples for clients to specify to attach to the above: Examples for clients to specify to attach to the above:
* tcp://localhost:3344 # for TCP/IP, assuming localhost use * cci+tcp://localhost:3344 # for TCP/IP, assuming localhost use
* verbs://192.168.1.78:3344 # for InfiniBand, note that you *must* use IP * cci+verbs://192.168.1.78:3344 # for InfiniBand, note that you *must* use IP
address rather than hostname address rather than hostname
* sm:///tmp/cci/sm/`hostname`/1/1 # note that this is a full path to local * cci+sm:///tmp/cci/sm/`hostname`/1/1 # note that this is a full path to local
connection information. The last portion of the path should match the connection information. The last portion of the path should match the
address specified above address specified above
## Running examples ### BMI
The examples subdirectory contains:
* client.c: an example client
* server.c: an example server
* my-rpc.[ch]: an example RPC definition
To run them using CCI/TCP, for example, you would do this:
```
examples/server tcp://3344
examples/client tcp://localhost:3344
```
The client will issue 4 concurrent RPCs to the server and wait for them to
complete.
## Running tests
`make check`
Notes: We do not recommend using any BMI methods besides TCP. It's usage is very similar to the CCI/TCP examples above, except that "bmi+" should be substituted for "cci+".
* the test scripts assume the use of the TCP/IP protocol and localhost
* the tests/timeout.sh script is known to fail when using CCI right now,
because we do not yet have an implementation of cancel for the address
lookup step which may block in CCI
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment