GitLab

Updated get_accumulate implementation to properly handle NO_OP operations;
currently, we dispatch these as get operations.  As a part of this change, the
op issue code was also gathered into a macro to reduce copy-paste across all
of the synchronization functions.

Reviewer: goodell

Removed scattered initializations of the RMA op dataloop field that are
redundant, since the dataloop field is now initialized when the op is
allocated.

Reviewer: goodell

Fetch-and-op has been updated to ignore the origin data and perform no
operation when the given op is MPI_NO_OP.

Reviewer: goodell

Reviewer: goodell

Removed calls to higher-level MPIU_RMA_CALL() macro.  This macro dispatches
either from the RMAFns table or directly to the MPID_ function; since CH3
always uses the RMAFns table, this macro shouldn't be used inside of CH3.

Reviewer: balaji

If flush is used as a part of polling for incoming data, we can deadlock, since
local RMA calls never poke the progress engine.  Extra progress was added to
local flushes to better fake passive progress.

Reviewer: balaji

Reviewer: buntinas

Updated RMA implementation to track the passive target status individually, for
each target.  Includes new implementation for lock/unlock_all.  Lock_all is
currently unoptimized, see #1734 for future plans.

Reviewer: buntinas

Remove ugly rank 0 hack and updated the active target code to maintain and
process a proper RMA ops list.

Reviewer: goodell

The use of a dense array is a temporary measure to support the reference
implementation.  This will be much improved by ticket #1735.

Reviewer: goodell

A couple of the RMA synchronization error checks were returning the wrong error
class.

reviewer: buntinas

Removed the use of lockRank in the passive target RMA code.  This was a hack to
start with, since lockRank belonged to the the front-end error checking and
should not have been used below the ADI.

Reviewer: buntinas

This is a temporary measure for the reference implementation that should be
improved by ticket #1733.

Reviewer: buntinas

In this patch, I have refactored the RMA ops list again to use the MPL UTList
doubly-linked list and to treat the list as a proper object.  This should set
us up to work with multiple lists, as we will soon have one list per target.
Doubly-linking the list is a big help in terms of maintainability (no more
prevNext pointers) and flexibility (better implementation of request-based
ops and other optimizations).

Reviewer: goodell

Request refs at the target were incorrectly set to two.  Only one ref is held,
by the progress engine.

Reviewer: none

Copy/paste oops.

Reviewer: none

Implemented the local flush operations in terms of the full flush operations.

Reviewer: goodell

Reviewer: goodell

Added a new inline function to free the ops list.  This is much clearer than
how it was done previously and should make it easier to see when/where we
can assume the ops list has been totally emptied.

Reviewer: goodell

Refactored RMA ops enqueue/dequeue operations into inline functions to reduce
copy/paste code.  Added extra safety checks that will be needed as we renovate
for MPI-3.

Reviewer: goodell

The lock packet may not be at the head of the queue, so the trailing get
optimization needed to be updated to not skip over the first element.  This is
a performance, not a correctness fix.

Reviewer: none

This commit implements MPI-3 RMA's flush and flush_all operations.

Reviewer: buntinas

Updated passive target synchronization to allow the RMA lock op to be omitted
from the op list when the lock is immediately requested.  This is needed to
implement MPI-3 RMA flush operations.

Reviewer: buntinas

When enabled, this mode of operation immediately requests the lock when
MPI_Win_lock is called.  Currently, this is enabled by setting the
MPICH_RMA_LOCK_IMMED environment variable.  In the future, we can also make
this mode of operation available though an info/assert.  This capability is
needed to implement MPI-3's flush operations.

Reviewer: buntinas

Factored out code to initiate a lock operation and to wait for its completion.
This will facilitate future changes, where the lock request can come from
several different parts of the code.

Reviewer: buntinas

MPI_.

A slightly older version of this patch was reviewed by goodell.

Release the request instead of manually setting refs.

Reviewer: buntinas

This fixes FOP segv failures when configured with --enable-g=mem, and partially
addresses ticket #1697.

Removed dead code from passive target RMA processing.

Reviewer: none

True_lb is always zero for predefined types; there's no need to include this in
address calculations when processing predefined types.

Reviewer: goodell

Reviewer: goodell

By setting "indent-tabs-mode:nil" we should hopefully begin to slowly
squeeze out hard tabs from the source without a disruptive (to
downstream projects) whitespace-fixing change.

No reviewer.

Updated the fetch_and_op implementation to have two data transfer paths; one
where data can be embedded in the packet header and one where it is sent
separately.  With this change, the header size is back to 40 bytes.

Reviewer: buntinas

Added builtin op implementations for RMA-only ops.  MPI_REPLACE was not defined
in the op_table prior to this commit.  Also includes an alternate
implementation of the linked list test that uses fetch-and-op with these
operations to enable the use of shared locks.

Reviewer: goodell

Adds implementation of atomic fetch-and-op operation and several test cases.

Reviewer: buntinas

Reviewer: buntinas

Reviewer: buntinas

The source_win_handle is used to inform the target that the RMA access epoch
has ended.  This was removed from the CAS packet to save space and was replaced
with another packet type, MPIDI_CH3_PKT_CAS_UNLOCK, which conveys the same
information.

Reviewer: buntinas