GitLab

Since margo is essentially the glue that will bind many composable microservices together, it should offer a mechanism to allow composed services to shutdown coherently. Currently, it offers a margo_finalize() call that is used to tear down the margo RTE, and a margo_wait_for_finalize() call that blocks until the former call has been invoked. The issue is that services overlying margo may have other shutdown routines that need to be executed, some of which may require a margo instance as input -- these routines can not be invoked if margo has already been shutdown.

To address this, we should add an API to allow other services to push shutdown callbacks into margo, with these callbacks popped off this stack in order to make sure all services are shut down cleanly.