• Wesley Bland's avatar
    Add MPI_Comm_revoke · 57f6ee88
    Wesley Bland authored
    MPI_Comm_revoke is a special function because it does not have a matching call
    on the "receiving side". This is because it has to act as an out-of-band,
    resilient broadcast algorithm. Because of this, in this commit, in addition to
    the usual functions to implement MPI communication calls (MPI/MPID/CH3/etc.),
    we add a new CH3 packet type that will handle revoking a communicator without
    involving a matching call from the MPI layer (similar to how RMA is currently
    The thing that must be handled most carefully when revoking a communicator is
    to ensure that a previously used context ID will eventually be returned to the
    pool of available context IDs and that after this occurs, no old messages will
    match the new usage of the context ID (for instance, if some messages are very
    slow and show up late). To accomplish this, revoke is implemented as an
    all-to-all algorithm. When one process calls revoke, it will send a message to
    all other processes in the communicator, which will trigger that process to
    send a message to all other processes, and so on. Once a process has already
    revoked its communicator locally, it won't send out another wave of messages.
    As each process receives the revoke messages from the other processes, it will
    track how many messages have been received. Once it has either received a
    revoke message or a message about a process failure for each other process, it
    will release its refcount on the communicator object. After the application
    has freed all of its references to the communicator (and all requests, files,
    etc. associated with it), the context ID will be returned to the available
    Signed-off-by: default avatarJunchao Zhang <jczhang@mcs.anl.gov>
mpidpkt.h 14.8 KB