Commit 22924f35 authored by Min Si's avatar Min Si Committed by Antonio J. Pena
Browse files

Bug-fix: correct the behavior of flush in exclusively locked epoch.

FLUSH should guarantee the completion of operations on both origin
and target side. However, for exclusive lock, there is an optimization
in MPICH which allows FLUSH to return without waiting for the
acknowledgement of remote completion from the target side. It relys
on the fact that there will be no other processes accessing the window
during the exclusive lock epoch.

However, such optimization is not correct when two processes allocating
windows on overlapping SHM region. Suppose P0 and P1 (on the same node)
allocate RMA window using the same SHM region, and P2 (on a different node)
locks both windows. P2 first issues a PUT and FLUSH to P0, then issues
a GET to P1 on the same memory location with PUT, since FLUSH does not
guarantee the remote completion of PUT, GET operation may not get the
updated value.

This patch disables the optimization for FLUSH and forces FLUSH to always
wait for the remote completion of operations.
Signed-off-by: default avatarXin Zhao <>
Signed-off-by: default avatarAntonio J. Pena <>
parent f6d32e72
......@@ -3601,11 +3601,15 @@ static int do_passive_target_rma(MPID_Win *win_ptr, int target_rank,
win_ptr->targets[target_rank].remote_lock_assert & MPI_MODE_NOCHECK));
if (win_ptr->targets[target_rank].remote_lock_mode == MPI_LOCK_EXCLUSIVE &&
win_ptr->targets[target_rank].remote_lock_state != MPIDI_CH3_WIN_LOCK_CALLED) {
win_ptr->targets[target_rank].remote_lock_state != MPIDI_CH3_WIN_LOCK_CALLED &&
win_ptr->targets[target_rank].remote_lock_state != MPIDI_CH3_WIN_LOCK_FLUSH) {
/* Exclusive lock already held -- no need to wait for rma done pkt at
the end. This is because the target won't grant another process
access to the window until all of our operations complete at that
target. Thus, there is no third-party communication issue. */
target. Thus, there is no third-party communication issue.
However, flush still needs to wait for rma done, otherwise result
may be unknown if user reads the updated location from a shared window of
another target process after this flush. */
*wait_for_rma_done_pkt = 0;
else if (MPIDI_CH3I_RMA_Ops_isempty(&win_ptr->targets[target_rank].rma_ops_list)) {
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment