GitLab

Xin Zhao authored Mar 01, 2015 and Pavan Balaji committed Mar 04, 2015

Add a counter in op struct to remember number of stream units
that have already been issued. For example, when the first stream
unit piggybacked with LOCK is issued out, we temporarily stop
issuing the following units. After the origin receives the ACK
from the target, it can continue to issue the following units.
This counter helps avoid issuing the first unit again.
Signed-off-by: Pavan Balaji <balaji@anl.gov>

Xin Zhao authored Mar 01, 2015 and Pavan Balaji committed Mar 04, 2015

Because we may cut one RMA operation into multiple packets,
and each packet needs a request object to track the completion,
here we use a request array instead of single request in
RMA operation structure.
Signed-off-by: Pavan Balaji <balaji@anl.gov>

Xin Zhao authored Mar 01, 2015 and Pavan Balaji committed Mar 04, 2015

After (1) issuing an op (no LOCK flag), or (2) issuing an op
(with LOCK flag) and receiving an ACK that LOCK is granted or
queued, we should set the user request (ureq) to be completed.
This patch wraps up the work of setting ureq into a function,
and call that function after (1) and (2) happens.
Signed-off-by: Pavan Balaji <balaji@anl.gov>

Xin Zhao authored Mar 01, 2015 and Pavan Balaji committed Mar 04, 2015

After we issue an op, we set the next_op_to_issue to the next op,
and if next op is NULL, we set sync_flag to NONE. When we receive
the lock ACK saying that lock request is discarded, we set the
next_op_to_issue back to the current op, we reset the sync_flag
from NONE to corresponding flag, since we need to re-transmit the
current op.
Signed-off-by: Pavan Balaji <balaji@anl.gov>

Xin Zhao authored Mar 01, 2015 and Pavan Balaji committed Mar 04, 2015

When the lock is not satisfied, we queue up
the lock request and op data in a lock entry
queue. In the struct of lock entry, we use 'data_size'
to remember the size of buffer for storing the
data. Since the size of buffer is not type_size*count
but might be type_extent*extent, here we change
its name from 'data_size' to 'buf_size'.
Signed-off-by: Pavan Balaji <balaji@anl.gov>

Xin Zhao authored Mar 03, 2015 and Pavan Balaji committed Mar 04, 2015

The original implementation of RMA does not consider pair basic
types (e.g. MPI_FLOAT_INT, MPI_DOUBLE_INT). It only
works correctly with builtin datatypes (e.g. MPI_INT, MPI_FLOAT).
This patch makes the RMA work correctly with pair basic types.

The bug is that: (1) when performing the ACC computation, the original
implementation uses 'eltype' in the datatype structure, which is set
when all basic elements in this datatype have the same builtin
datatype. When basic elements have different builtin datatypes, like
pair datatypes, the 'eltype' is set to MPI_DATATYPE_NULL. This makes
the ACC computation be unable to work with pair types; (2) for all
basic type of data, the original implementation assumes that
they are all contiguous and issues them in an unpacked manner
with length of data size (count*type_size). This is incorrect for
pair datatypes, because most pair datatypes are non-contiguous
(type_extent != type_size).

In the previous patch, we already made 'eltype' to store basic
type instead of builtin type. In this patch, we fixed this
bug by (1) modify ACC computation to treat 'eltype' as basic
type; (2) For non-contiguous basic type data, we use the noncontig
API so that it will be issued in a packed manner.
Signed-off-by: Pavan Balaji <balaji@anl.gov>

In this patch, we gather window basic attributes of other
processes (base_addr, size, disp_unit, win_handle) using a
struct called "basic_info_table". By doing this, we can use
one contiguous memory region to store them.
Signed-off-by: Pavan Balaji <balaji@anl.gov>

No reviewer.

For GET-like RMA packets and response packets (GACC,
GET, FOP, CAS, GACC_RESP, GET_RESP, FOP_RESP, CAS_RESP),
originally we carry source_win_handle in packet struct
in order to locate window handle on origin side in the
packet handler of response packets. However, this is
not necessary because source_win_handle can be stored
in the request on the origin side. This patch delete
source_win_handle from those packets to reduce the size
of packet union.
Signed-off-by: Pavan Balaji <balaji@anl.gov>

In this patch, we replace one argument of function
finish_op_on_target, "packet(op) type", with "has_response_data".
Since finish_op_on_target does not care what specific
packet(op) type it is processing on, but only cares
about if the current op has response data (like GET/GACC),
changing the argument in this way can simplify the
code by avoiding acquiring packet(op) type everytime
before calling finish_op_on_target.
Signed-off-by: Pavan Balaji <balaji@anl.gov>

Originally we add "immed_data" and "immed_len" areas to RMA packets,
in order to piggyback small amount of data with packet header to
reduce number of packets (Note that "immed_len" is necessary when
the piggybacked data is not the entire data). However, those areas
potentially increase the packet union size and worsen the two-sided
communication. This patch fixes this issue.

In this patch, we remove "immed_data" and "immed_len" from normal
"MPIDI_CH3_Pkt_XXX_t" operation type (e.g. MPIDI_CH3_Pkt_put_t), and
we introduce new "MPIDI_CH3_Pkt_XXX_immed_t" packt type for each
operation (e.g. MPIDI_CH3_Pkt_put_immed_t).

"MPIDI_CH3_Pkt_XXX_immed_t" is used when (1) both origin and target
are basic datatypes, AND, (2) the data to be sent can be entirely fit
into the header. By doing this, "MPIDI_CH3_Pkt_XXX_immed_t" needs
"immed_data" area but can drop "immed_len" area. Also, since it only
works with basic target datatype, it can drop "dataloop_size" area
as well. All operations that do not satisfy (1) or (2) will use
normal "MPIDI_CH3_Pkt_XXX_t" type.

Originally we always piggyback FOP data into the packet header,
which makes the packet size too large. In this patch we split the
FOP operaton into IMMED packets and normal packets.

Because CAS only work with 2 basic datatype and non-complex
elements, the data amount is relatively small, we always piggyback
the data with packet header and only use "MPIDI_CH3_Pkt_XXX_immed_t"
packet type for CAS.
Signed-off-by: Pavan Balaji <balaji@anl.gov>

Originally we added lock_type and origin_rank areas
in RMA packet, in order to piggyback passive lock request
with RMA operations. However, those areas potentially
enlarged the packet union size, and actually they are
not necessary and can be completetly avoided.

"Lock_type" is used to remember what types of lock (shared or
exclusive) the origin wants to acquire on the target. To remove
it from RMA packet, we use flags (already exists in RMA packet)
to remember such information.

"Origin_rank" is used to remember which origin has sent lock
request to the target, so that when the lock is granted to this
origin later, the target can send ack to that origin. Actually
the target does not need to store origin_rank but can only store
origin_vc, which is known from progress engine on target side.
Therefore, we can completely remove origin_rank from RMA packet.
Signed-off-by: Pavan Balaji <balaji@anl.gov>

When data is dropped but lock is queued, we should still store
the lock entry in current request, so that we can try to acquire
the lock when we received and dropped all data.

No reviewer.

No reviewer.

No reviewer.

No reviewer.

The behavior of UNLOCK_ACK flag is exactly the same
with the behavior of FLUSH_ACK, so here we just delete
UNLOCK_ACK flag and use FLUSH_ACK flag for all FLUSH
ACK packets.

No reviewer.

No reviewer.

No reviewer.

Because we will send different kinds of LOCK ACKs (not
just LOCK_GRANTED, but maybe LOCK_DISCARDED, for example),
so naming related packets and function as "LOCK_GRANTED"
is not proper anymore. Here we rename them to "LOCK_ACK".

No reviewer.

No reviewer.

Rewrite progress engine functions as following:

Basic functions:

(1) check_target_state: check to see if we can switch target state,
    issue synchronization messages if needed.
(2) issue_ops_target: issue al pending operations to this target.
(3) check_window_state: check to see if we can switch window state.
(4) issue_ops_win: issue all pending operations on this window.
    Currently it internally calls check_target_state and
    issue_ops_target, it should be optimized in future.

Progress making functions:

(1) Make_progress_target: make progress on one target, which
    internally call check_target_state and issue_ops_target.
(2) Make_progress_win: make progress on all targets on one window,
    which internally call check_window_state and issue_ops_win.
(3) Make_progress_global: make progress on all windows, which
    internally call make_progress_win.

No reviewer.

No reviewer.

No reviewer.

No reviewer.

Use int instead of size_t in RMA pkt header to reduce
packet size.

No reviewer.

Originally we only allows LOCK request to be piggybacked
with small RMA operations (all data can be fit in packet
header). This brings communication overhead for larger
operations since origin side needs to wait for the LOCK
ACK before it can transmit data to the target.

In this patch we add support of piggybacking LOCK with
RMA operations with arbitrary size. Note that (1) this
only works with basic datatypes; (2) if the LOCK cannot
be satisfied, we temporarily buffer this operation on
the target side.

No reviewer.

No reviewer.

Arrange RMA sync functions in src/mpid/ch3/src/ch3u_rma_sync.c
in the following order:

Win_fence
Win_post
Win_start
Win_complete
Win_wait
Win_test
Win_lock
Win_unlock
Win_flush
Win_flush_local
Win_lock_all
Win_unlock_all
Win_flush_all
Win_flush_local_all
Win_sync

No reviewer.

Here we wrap up common action when one RMA op is finished on target
into a function to make code structure cleaner.
Signed-off-by: Pavan Balaji <balaji@anl.gov>

Originally do_accumulate_op() only accepts request pointer as
argument which is too restrict to be reused. Here we modify
it to access buffer address, count, datatype and op, so that
it can be reused in more general cases.
Signed-off-by: Pavan Balaji <balaji@anl.gov>

When operation pending list and request lists are all empty, FLUSH message
needs to be sent by origin only when origin issued PUT/ACC operations since
the last synchronization calls, otherwise origin does not need to issue FLUSH
at all and does not need to wait for FLUSH ACK message.

Similiarly, origin waits for ACK of UNLOCK message only when origin issued
PUT/ACC operations since the last synchronization calls. However, UNLOCK
message always needs to be sent out because origin needs to unlock the
target process. This patch avoids issuing unnecessary
FLUSH / FLUSH ACK / UNLOCK ACK messages.
Signed-off-by: Pavan Balaji <balaji@anl.gov>

Add some original RMA PVARs back to the new
RMA infrastructure, including timing of packet
handlers, op allocation and setting, window
creation, etc.
Signed-off-by: Pavan Balaji <balaji@anl.gov>

We made a huge change to RMA infrastructure and
a lot of old code can be droped, including separate
handlers for lock-op-unlock, ACCUM_IMMED specific
code, O(p) data structure code, code of lazy issuing,
etc.
Signed-off-by: Pavan Balaji <balaji@anl.gov>

1. Piggyback LOCK request with first IMMED operation.

When we see an IMMED operation, we can always piggyback
LOCK request with that operation to reduce one sync
message of single LOCK request. When packet header of
that operation is received on target, we will try to
acquire the lock and perform that operation. The target
either piggybacks LOCK_GRANTED message with the response
packet (if available), or sends a single LOCK_GRANTED
message back to origin.

2. Rewrite code of manage lock queue.

When the lock request cannot be satisfied on target,
we need to buffer that lock request on target. All we
need to do is enqueuing the packet header, which contains
all information we need after lock is granted. When
the current lock is released, the runtime will goes
over the lock queue and grant the lock to the next
available request. After lock is granted, the runtime
just trigger the packet handler for the second time.

3. Release lock on target side if piggybacking with UNLOCK.

If there are active-message operations to be issued,
we piggyback a UNLOCK flag with the last operation.
When the target recieves it, it will release the current
lock and grant the lock to the next process.
Signed-off-by: Pavan Balaji <balaji@anl.gov>

Here we extract the common code of different
issuing functions at origin side and simplify
those issuing functions.
Signed-off-by: Pavan Balaji <balaji@anl.gov>

We add a IMMED data area (16 bytes by default) in
packet header which will contains as much origin
data as possible. If origin can put all data in
packet header, then it no longer needs to send
separate data packet. When target recieves the
packet header, it will first copy data out from
the IMMED data area. If there is still more
data coming, it continues to receive following
packets; if all data is included in header, then
recieving is done.
Signed-off-by: Pavan Balaji <balaji@anl.gov>

During PSCW, when there are active-message operations
to be issued in Win_complete, we piggback a AT_COMPLETE
flag with it so that when target receives it, it can
decrement a counter on target side and detect completion
when target counter reaches zero.
Signed-off-by: Pavan Balaji <balaji@anl.gov>

When the origin wants to do a FLUSH sync, if there are
active-message operations that are going to be issued,
we piggback the FLUSH message with the last operation;
if no such operations, we just send a single FLUSH packet.

If the last operation is a write op (PUT, ACC) or only
a single FLUSH packet is sent, after target recieves it,
target will send back a single FLUSH_ACK packet;
if the last operation contains a read action (GET, GACC, FOP,
CAS), after target receiveds it, target will piggback a
FLUSH_ACK flag with the response packet.

After origin receives the FLUSH_ACK packet or response packet
with FLUSH_ACK flag, it will decrement the counter which
indicates number of outgoing sync messages (FLUSH / UNLOCK).
When that counter reaches zero, origin can know that remote
completion is achieved.
Signed-off-by: Pavan Balaji <balaji@anl.gov>

Xin Zhao authored Nov 02, 2014 and Pavan Balaji committed Nov 03, 2014

We were duplicating information in the operation structure and in the
packet structure when the message is actually issued.  Since most of
the information is the same anyway, this patch just embeds a packet
structure into the operation structure, so that we eliminate unnessary
copy.
Signed-off-by: Pavan Balaji <balaji@anl.gov>