If user set no_locks to true, we do not need
to allocate passive lock requests pool and
lock data pool on window.

No reviewer.

No reviewer.

No reviewer.

No reviewer.

No reviewer.

The behavior of UNLOCK_ACK flag is exactly the same
with the behavior of FLUSH_ACK, so here we just delete
UNLOCK_ACK flag and use FLUSH_ACK flag for all FLUSH
ACK packets.

No reviewer.

No reviewer.

No reviewer.

Add new flags for four different kinds of LOCK ACKs:

(1) LOCK_GRANTED: lock is granted on target.
(2) LOCK_QUEUED_DATA_QUEUED: lock is not granted on target,
    but it is safely queued on target. If this lock request
    is sent with an RMA operation, the operation data is also
    safely queued on target.
(3) LOCK_QUEUED_DATA_DISCARDED: lock is not granted on target,
    but it is safely queued on target. If this lock request
    is sent with an RMA operation, the operation data is discarded
    on target due to out of resources.
(4) LOCK_DISCARDED: lock is not granted on target, and it is
    not queued up on target due to out of resources. If this
    lock request is set with an RMA opration, the operation data
    is also discarded on target.

No reviewer.

Because we will send different kinds of LOCK ACKs (not
just LOCK_GRANTED, but maybe LOCK_DISCARDED, for example),
so naming related packets and function as "LOCK_GRANTED"
is not proper anymore. Here we rename them to "LOCK_ACK".

No reviewer.

No reviewer.

No reviewer.

Rewrite progress engine functions as following:

Basic functions:

(1) check_target_state: check to see if we can switch target state,
    issue synchronization messages if needed.
(2) issue_ops_target: issue al pending operations to this target.
(3) check_window_state: check to see if we can switch window state.
(4) issue_ops_win: issue all pending operations on this window.
    Currently it internally calls check_target_state and
    issue_ops_target, it should be optimized in future.

Progress making functions:

(1) Make_progress_target: make progress on one target, which
    internally call check_target_state and issue_ops_target.
(2) Make_progress_win: make progress on all targets on one window,
    which internally call check_window_state and issue_ops_win.
(3) Make_progress_global: make progress on all windows, which
    internally call make_progress_win.

No reviewer.

No reviewer.

No reviewer.

No reviewer.

(1) Win_fence/Win_start: set access state right after we
    issue synchronization calls.
(2) Win_post: set exposure state at beginning.
(3) Win_wait/Win_test: set exposure state at end.
(4) Win_lock/Win_lock_all: set access state at beginning.
(5) Win_unlock/Win_unlock_all: set access state at end.

No reviewer.

In Win_complete, release all requests on window; in
Win_unlock_all, reset lock_assert on window.

No reviewer.

We always need to allocate a array to store group ranks
even for MPI_MODE_NOCHECK case, because we need always
need that in Win_complete.

No reviewer.

For Win_fence, Win_complete and Win_unlock_all, check if
all targets are freed at the end of function calls.

No reviewer.

We call memory barriers at proper places in RMA sync calls
as following, and remove unnecessary memory barriers:

(1) Win_fence: very beginning and very end.
(2) Win_post/Win_complete: very beginning.
(3) Win_start/Win_wait/Win_test: very end.
(4) Win_lock/Win_lock_all: very end.
(5) Win_unlock/Win_unlock_all: very beginning.
(6) Win_flush/Win_flush_local/Win_flush_all/Win_flush_local_all: very beginning.

About the reason of doing this, please refer to comments
at the beginning of src/mpid/ch3/src/ch3u_rma_sync.c.

No reviewer.

In ending RMA synchronization calls, we poke the
progress engine at last if we never poke it before.
Because some program execution depends on the
incoming events in progress engine, if we never
process them we may cause deadlock in the program.

No reviewer.

Originally free_ops_before_completion functions only
works with active target. Here we modify it to accomodate
passive target as well.

Also, everytime we trigger free_ops_before_completion,
we lose the chance to do real Win_flush_local operation
and must do a Win_flush instead. Here we transfer
Win_flush_local to Win_flush if disable_flush_local flag
is set, and unset that flag after the current flush
is fone.

No reviewer.

No reviewer.

Use int instead of size_t in RMA pkt header to reduce
packet size.

No reviewer.

No reviewer.

No reviewer.

In this patch we allow GET/GACC response packets to
piggyback some IMMED data, just like what we did
for PUT/GACC/FOP/CAS packets.

No reviewer.

Originally we only allows LOCK request to be piggybacked
with small RMA operations (all data can be fit in packet
header). This brings communication overhead for larger
operations since origin side needs to wait for the LOCK
ACK before it can transmit data to the target.

In this patch we add support of piggybacking LOCK with
RMA operations with arbitrary size. Note that (1) this
only works with basic datatypes; (2) if the LOCK cannot
be satisfied, we temporarily buffer this operation on
the target side.

No reviewer.

No reviewer.

No reviewer.

No reviewer.

accumulated_ops_cnt is used to track no. of accumulated
posted RMA operations between two synchronization calls,
so that we can decide when to poke progress engine based
on the current value of this counter.

Here we initialize it to zero in the BEGINNING synchronization
calls (Win_fence, Win_start, first Win_lock, Win_lock_all),
and correctly decrement it in the ENDING synchronization calls
(Win_fence, Win_complete, Win_unlock, Win_unlock_all,
Win_flush, Win_flush_local, Win_flush_all, Win_flush_local_all).
We also use a per-target counter to track single target case.

No reviewer.

No reviewer.

Arrange RMA packet definition and structures in
src/mpid/ch3/include/mpidpkt.h in the following
order:

1. RMA operation packets: PUT, GET, ACC, GACC, CAS, FOP
2. RMA operation response packets: GET_RESP, GACC_RESP, CAS_RESP, FOP_RESP
3. RMA control packets: LOCK, UNLOCK, FLUSH, DECR_AT_COUNTER
4. RMA control response packets: LOCK_ACK, FLUSH_ACK

No reviewer.

Arrange RMA sync functions in src/mpid/ch3/src/ch3u_rma_sync.c
in the following order:

Win_fence
Win_post
Win_start
Win_complete
Win_wait
Win_test
Win_lock
Win_unlock
Win_flush
Win_flush_local
Win_lock_all
Win_unlock_all
Win_flush_all
Win_flush_local_all
Win_sync

No reviewer.

artem.v.yalozo authored Oct 23, 2014 and Kenneth Raffenetti committed Dec 08, 2014

This patch provides the following fix wrt Windows conformance feature
(makes single code working on both platforms Linux and Windows):
    - RMA mutexes fix for Windows

Change-Id: Ib4f7b2ec8a07813f0ed35281a1d584637c84c0a9
Signed-off-by: Ken Raffenetti <raffenet@mcs.anl.gov>