Commit 1c07dbaf authored by Xin Zhao's avatar Xin Zhao
Browse files

Fix bug: add barrier semantic in FENCE for SHM ops.



When SHM is allocated for RMA window, operations are completed
eagerly (as soon as they are posted by the user), therefore we
need barrier semantics in the FENCE that opens an epoch to prevent
SHM ops happening on target process before that target process
starts an epoch.

Note that we need memory barrier before and after synchronization
calls in both FENCEs that starts and ends an epoch to guarantee the
ordering of load/store operations with synchronizations.

See #2041.
Signed-off-by: Pavan Balaji's avatarPavan Balaji <balaji@anl.gov>
parent f02eed5b
......@@ -968,6 +968,28 @@ int MPIDI_Win_fence(int assert, MPID_Win *win_ptr)
MPIR_T_PVAR_TIMER_END(RMA, rma_winfence_clearlock);
}
if (!(assert & MPI_MODE_NOSUCCEED) &&
(assert & MPI_MODE_NOPRECEDE || win_ptr->fence_issued == 0)) {
/* In the FENCE that opens an epoch but does not close an epoch,
if SHM is allocated, perform a barrier among processes on the
same node, to prevent one process modifying another process's
memory before that process starts an epoch. */
if (win_ptr->shm_allocated == TRUE) {
MPID_Comm *node_comm_ptr = win_ptr->comm_ptr->node_comm;
/* Ensure ordering of load/store operations. */
OPA_read_write_barrier();
mpi_errno = MPIR_Barrier_impl(node_comm_ptr, &errflag);
if (mpi_errno) {goto fn_fail;}
/* Ensure ordering of load/store operations. */
OPA_read_write_barrier();
}
}
/* Note that the NOPRECEDE and NOSUCCEED must be specified by all processes
in the window's group if any specify it */
if (assert & MPI_MODE_NOPRECEDE)
......@@ -1054,6 +1076,11 @@ int MPIDI_Win_fence(int assert, MPID_Win *win_ptr)
if (mpi_errno) { MPIU_ERR_POP(mpi_errno); }
MPIU_ERR_CHKANDJUMP(errflag, mpi_errno, MPI_ERR_OTHER, "**coll_fail");
/* Ensure ordering of load/store operations. */
if (win_ptr->shm_allocated == TRUE) {
OPA_read_write_barrier();
}
/* Set the completion counter */
/* FIXME: MT: this needs to be done atomically because other
procs have the address and could decrement it. */
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment