GitLab

Previously several RMA packet types share the same structure,
which is misleading for coding. Here make different
RMA packet types use different packet data structures.
Signed-off-by: Pavan Balaji <balaji@anl.gov>

When there are too many active requests in the runtime,
the internal memory might be used up. This patch
prevents such situation by triggering blocking
wait loop in operation routines when no. of active
requests reaches certain threshold value.
Signed-off-by: Pavan Balaji <balaji@anl.gov>

We no longer use the lazy-issuing model, which delays
all operations to the end to issue, but issues them
as early as possible. To achieve this, we enable
making progress in RMA routines, so that RMA operations
can be issued out as long as synchronization is finished.

Sometimes we also need to poke the progress in
operation routines to make sure that target side
makes enough progress to receiving packets. Here
we trigger it when no. of posted operations reaches
certain threshold value.
Signed-off-by: Pavan Balaji <balaji@anl.gov>

GET_OP function may be a blocking function which guarantees
to return an RMA operation.

Inside GET_OP we first call the normal OP_ALLOC function
which will try to get a new OP from OP pools; if failed,
we call nonblocking GC function to cleanup completed ops
and then call OP_ALLOC again; if we still cannot get a
new OP, we call nonblocking FREE_OP_BEFORE_COMPLETION
function if hardware ordering is provided and then call
OP_ALLOC again; if still failed, finally we call blocking
aggressive cleanup function, which will guarantee to
return a new OP element.
Signed-off-by: Pavan Balaji <balaji@anl.gov>

When we run out of resources for operations and targets,
we need to make the runtime to complete some operations
so that it can free some resources.

For RMA operations, we implement by doing an internal
FLUSH_LOCAL for one target and waiting for operation
resources; for RMA targets, we implement by doing an
internal FLUSH operation for one target and wait for
target resources.
Signed-off-by: Pavan Balaji <balaji@anl.gov>

We define new states to indicate the current situation of
RMA synchronization. The states contain both ACCESS states
and EXPOPSURE states, and specify if the synchronization
is initialized (_CALLED), on-going (_ISSUED) and completed
(_GRANTED). For single lock in Passive Target, we use
per-target state whereas the window state is set to PER_TARGET.
Signed-off-by: Pavan Balaji <balaji@anl.gov>

Add flag is_dt in op structure which is set when any
buffers involved in RMA operations contains derived
datatype data. It is convenient for us to enqueue
issued but not completed operation to the DT specific
list.
Signed-off-by: Pavan Balaji <balaji@anl.gov>

Given an RMA op, finding the correct slot and target,
enqueue op to the pending op list in that target object.
If the target is not existed, create one in that slot.
Signed-off-by: Pavan Balaji <balaji@anl.gov>

Xin Zhao authored Nov 02, 2014 and Pavan Balaji committed Nov 03, 2014

Instead of allocating / deallocating RMA operations whenever
an RMA op is posted by user, we allocate fixed size operation
pools beforehand and take the op element from those pools
when an RMA op is posted.

With only a local (per-window) op pool, the number of ops
allocated can increase arbitrarily if many windows are created.
Alternatively, if we only use a global op pool, other windows
might use up all operations thus starving the window we are
working on.

In this patch we create two pools: a local (per-window) pool and a
global pool.  Every window is guaranteed to have at least the number
of operations in the local pool.  If we run out of these operations,
we check in the global pool to see if we have any operations left.
When an operation is released, it is added back to the same pool it
was allocated from.
Signed-off-by: Pavan Balaji <balaji@anl.gov>

Xin Zhao authored Nov 02, 2014 and Pavan Balaji committed Nov 03, 2014

We were duplicating information in the operation structure and in the
packet structure when the message is actually issued.  Since most of
the information is the same anyway, this patch just embeds a packet
structure into the operation structure, so that we eliminate unnessary
copy.
Signed-off-by: Pavan Balaji <balaji@anl.gov>

Xin Zhao authored Nov 02, 2014 and Pavan Balaji committed Nov 03, 2014

Split RMA functionality into smaller files, and move functions
to where they belong based on the file names.
Signed-off-by: Pavan Balaji <balaji@anl.gov>

Xin Zhao authored Nov 02, 2014 and Pavan Balaji committed Nov 03, 2014

Because we are going to rewrite the RMA infrastructure
and many PVARs will no longer be used, here we temporarily
remove all PVARs and will add needed PVARs back after new
implementation is done.
Signed-off-by: Pavan Balaji <balaji@anl.gov>

Xin Zhao authored Aug 12, 2014 and Pavan Balaji committed Nov 01, 2014

The original implementation includes an optimization which
allows Win_unlock for exclusive lock to return without
waiting for remote completion. This relys on the
assumption that window memory on target process will not
be accessed by a third party until that target process
finishes all RMA operations and grants the lock to other
processes. However, this assumption is not correct if user
uses assert MPI_MODE_NOCHECK. Consider the following code:

          P0                              P1           P2
    MPI_Win_lock(P1, NULL, exclusive);
    MPI_Put(X);
    MPI_Win_unlock(P1, exclusive);
    MPI_Send (P2);                                MPI_Recv(P0);
                                                  MPI_Win_lock(P1, MODE_NOCHECK, exclusive);
                                                  MPI_Get(X);
                                                  MPI_Win_unlock(P1, exclusive);

Both P0 and P2 issue exclusive lock to P1, and P2 uses assert
MPI_MODE_NOCHECK because the lock should be granted to P2 after
synchronization between P2 and P0. However, in the original
implementation, GET operation on P2 might not get the updated
value since Win_unlock on P0 return without waiting for remote
completion.

In this patch we delete this optimization. In Win_free, since every
Win_unlock guarantees the remote completion, target process no
longer needs to do additional counting works to detect target-side
completion, but only needs to do a global barrier.
Signed-off-by: Pavan Balaji <balaji@anl.gov>

No reviewer.

This reverts commit 274a5a70.

We were creating duplicating information in the operation structure
and in the packet structure when the message is actually issued.
Since most of the information is the same anyway, this patch just
embeds a packet structure into the operation structure.
Signed-off-by: Xin Zhao <xinzhao3@illinois.edu>

Signed-off-by: Pavan Balaji <balaji@mcs.anl.gov>

Fixes compiler warnings in ch3u_rma_ops.c and ch3u_rma_acc_ops.c. Complains
were about "orig_vc" and "target_vc" of potentially being unused. This is a
false positive, as the conditional statements were correct. Warning fixed by just
initializing the pointers to NULL. gcc 4.6.3 complained in stomp, whereas gcc
4.7.4 did not on my laptop.
Signed-off-by: Pavan Balaji <balaji@mcs.anl.gov>

Fixes the following warnings (with --enable-strict):

src/mpid/ch3/src/ch3u_rma_acc_ops.c: In function 'MPIDI_Get_accumulate':
src/mpid/ch3/src/ch3u_rma_acc_ops.c:31:5: warning: unused variable
'mpiu_chklmem_stk_sz_' [-Wunused-variable]

src/mpid/ch3/src/ch3u_rma_ops.c: In function 'MPIDI_Accumulate':
src/mpid/ch3/src/ch3u_rma_ops.c:350:5: warning: unused variable
'mpiu_chklmem_stk_sz_' [-Wunused-variable]

See ticket #1966
Signed-off-by: Pavan Balaji <balaji@mcs.anl.gov>

Fixes #1962

Signed-off-by: Junchao Zhang<jczhang@mcs.anl.gov> (Reviewed by Bill Gropp)

The check was originally in the ch3 layer, but doesn't seem to use any
ch3 specific information.  This macro will be useful at the upper
layers for optimizations, e.g., in the localcopy routine.
Signed-off-by: Xin Zhao <xinzhao3@illinois.edu>

Xin Zhao authored Sep 25, 2013 and Pavan Balaji committed Sep 26, 2013

If SHM is allocated, perform RMA operations immediately after it is
issued, not queuing them up any more.
Signed-off-by: Pavan Balaji <balaji@mcs.anl.gov>

Xin Zhao authored Sep 25, 2013 and Pavan Balaji committed Sep 26, 2013

If we perform SHM ops immediately, we do not need to do the judgements
when incrementing datatypes, because SHM ops are already performed at
that point and judgements only works for remaining operations.
Signed-off-by: Pavan Balaji <balaji@mcs.anl.gov>

Xin Zhao authored Jul 30, 2013 and Pavan Balaji committed Aug 01, 2013

shm_base_addrs is allocated only when shared memory region is allocated in nemesis.
Signed-off-by: Pavan Balaji <balaji@mcs.anl.gov>

Xin Zhao authored Jul 30, 2013 and Pavan Balaji committed Aug 01, 2013

Because we delete the default MPIDI_SHM_Win_free in CH3, which originally deals with
win_allocate_shared in CH3, we need to modify MPIDI_Win_free to handle it.
Signed-off-by: Pavan Balaji <balaji@mcs.anl.gov>

Xin Zhao authored Jul 30, 2013 and Pavan Balaji committed Aug 01, 2013

Signed-off-by: Pavan Balaji <balaji@mcs.anl.gov>

Xin Zhao authored Jul 31, 2013 and Pavan Balaji committed Aug 01, 2013

When judging if origin and target process are on the same node, using vc->node_id flag instead of vc->ch.is_local flag.

Flag 'is_local' is not correct because it is defined in nemesis, not in CH3.
Flag 'node_id' is defined in CH3.

Note that for ch3:sock, even if origin and target are on the same node, they are not within the same SHM region.
Currently ch3:sock is filtered out by checking shm_allocated flag first. In future we need to figure out a way to
check if origin and target are within the same "SHM comm".
Signed-off-by: Pavan Balaji <balaji@mcs.anl.gov>

Xin Zhao authored Jul 26, 2013 and Pavan Balaji committed Jul 28, 2013

Delete decrementing ref count in SHM RMA operations, but add conditions in operaiton issue routines.

In RMA operation issue routines, judge if shm_allocate == 1 and target vc
is local, if so, do not add reference count on datatypes, because they will not
be referenced by the progress engine, but will be completed directly by origin.
Signed-off-by: Pavan Balaji <balaji@mcs.anl.gov>

Xin Zhao authored Jul 22, 2013 and Pavan Balaji committed Jul 28, 2013

Add "alloc_shm" to window's info arguments and initialize it to FALSE.
In MPID_Win_allocate, if "alloc_shm" is set to true, call ALLOCATE_SHARED,
otherwise call ALLOCATE.

Free window memory only when SHM region is not allocated, therwise it is
already freed in MPIDI_CH3I_SHM_Win_free.
Signed-off-by: Pavan Balaji <balaji@mcs.anl.gov>

Xin Zhao authored Jul 22, 2013 and Pavan Balaji committed Jul 28, 2013

The code in inline functions is moved from operation routines in
ch3u_rma_ops.c and ch3u_rma_acc_ops.c. By moving them in inline
functions, both operation routines and synchronization routines can call them.
Signed-off-by: Pavan Balaji <balaji@mcs.anl.gov>

Xin Zhao authored Jul 26, 2013 and Pavan Balaji committed Jul 28, 2013

Moving code of waiting finish of passive RMA operations from MPIDI_Win_free
to an inline function in mpidrma.h.
Signed-off-by: Pavan Balaji <balaji@mcs.anl.gov>

Myrank was caching win_ptr->comm_ptr->rank, so we now use that directly
rather than caching it in the MPID_Win object.

Reviewer: balaji

Moved handling of MPI_PROC_NULL down to the device and cleaned up
handling of PROC_NULL in the CH3 device.

Reviewer: goodell

This adds a default shared memory window implementation for CH3 (used
by e.g. sock), which works only for MPI_COMM_SELF (this is what the
default comm_split_type provides).  This closes ticket #1666.

Reviewer: apenya

Communication operations on shared memory windows now perform the op directly
on the shared buffer.  This requried the addition of a per-window interprocess
mutex to ensure that atomics and accumulates are performed atomically.

Reviewer: buntinas

This fixes the performance regression that was introduced by concatenation of
per-target lists.

Reviewer: goodell

The fence_cnt field in MPID_Win is not a counter, it's a flag that indicates if
fence has been called.

Reviewer: buntinas

This patch adds code to track the RMA epoch state of the local process.
Currently, we are tracking the synchronization states that are allowed by
MPICH; in the future, we may want to restrict this to only states that are
allowed by the standard.  The addition of epoch tracking has several benefits:

 * It allows us to detect synchronization errors (implemented in this patch).
 * It allows us to implement lock_all more efficiently (implemented in this
   patch).
 * It will allow us to distinguish between active and passive target epochs and
   avoid O(p) op list concatenation (future patch).

Reviewer: balaji

Reviewer: goodell