GitLab

Hi,

I am using Margo as part of a preloading library, where I require two separate Margo clients that run with different NA plugins. One is running with the NA SM (shared memory) plugin, and the other with the NA BMI plugin with TCP (for now). Both are started to run within the caller's thread context. This would work with Margo 0.1 because users had to deal with initializing Argobots themselves.

Margo 0.2 starts Argobots transparantly to the user, resulting in multiple Argobots initializations and sets the caller ES to idle with ABT_snoozer each time (with ABT_snoozer_xstream_self_set()). Specifically, the latter action appears to be the culprit as the second call of ABT_snoozer_xstream_self_set() (due to the second Margo client initialization) returns the errorcode 29 ([../../src/stream.c:811] ABT_xstream_set_main_sched: 29). This aborts the whole client initialization process and margo_init() returns MARGO_INSTANCE_NULL.

I've attached a small patch (margo_0.2_multiple_clients.patch) that highlights the issue and solves it by simply checking if Argobots is already initialized and, if so, skips both mentioned actions. However, I am questioning if this is the correct way of using multiple clients in one application.

Thanks!