same pool for driving mercury and executing RPCs?
Currently, margo is expected to run in its own ES - otherwise, other ULTS/tasks on the same ES can arbitrarily wait while the margo ULT calls progress.
Any possibility we can get the programming benefits of margo without having to tie ourselves to the thread-handoff model? I can imagine the following is a simple enough extension:
- margo enters "sharing is caring" mode when the progress pool and handler pool are the same
- only margo or ULTs/tasks created by margo may create ULTs/tasks in the shared pool
- when the margo instance is the only active ULT/task in the pool, margo is free to use blocking progression
- when the pool contains ULTs/tasks other than margo, margo uses non-blocking progress/trigger functions (i.e. uses a timeout of 0)
The downside is that we'll be spinning whenever we're processing operations. If only we had one event loop to rule them all...