1. 13 Feb, 2015 3 commits
  2. 16 Dec, 2014 12 commits
    • Xin Zhao's avatar
      Support handling different LOCK ACKs · 45afd1fd
      Xin Zhao authored
      No reviewer.
      45afd1fd
    • Xin Zhao's avatar
      Delete MPIDI_CH3_PKT_FLAG_RMA_UNLOCK_ACK flag · 97022653
      Xin Zhao authored
      The behavior of UNLOCK_ACK flag is exactly the same
      with the behavior of FLUSH_ACK, so here we just delete
      UNLOCK_ACK flag and use FLUSH_ACK flag for all FLUSH
      ACK packets.
      
      No reviewer.
      97022653
    • Xin Zhao's avatar
      Modify ACK of op with both LOCK and UNLOCK (FLUSH) flags · 03ebc97b
      Xin Zhao authored
      No reviewer.
      03ebc97b
    • Xin Zhao's avatar
      Modify send_lock_ack_pkt function to contain flags. · 01679120
      Xin Zhao authored
      No reviewer.
      01679120
    • Xin Zhao's avatar
      Add new pkt flags for different LOCK ACKs. · faae55ad
      Xin Zhao authored
      Add new flags for four different kinds of LOCK ACKs:
      
      (1) LOCK_GRANTED: lock is granted on target.
      (2) LOCK_QUEUED_DATA_QUEUED: lock is not granted on target,
          but it is safely queued on target. If this lock request
          is sent with an RMA operation, the operation data is also
          safely queued on target.
      (3) LOCK_QUEUED_DATA_DISCARDED: lock is not granted on target,
          but it is safely queued on target. If this lock request
          is sent with an RMA operation, the operation data is discarded
          on target due to out of resources.
      (4) LOCK_DISCARDED: lock is not granted on target, and it is
          not queued up on target due to out of resources. If this
          lock request is set with an RMA opration, the operation data
          is also discarded on target.
      
      No reviewer.
      faae55ad
    • Xin Zhao's avatar
      Change routine/pkt name from LOCK_GRANTED to LOCK_ACK · e36203c3
      Xin Zhao authored
      Because we will send different kinds of LOCK ACKs (not
      just LOCK_GRANTED, but maybe LOCK_DISCARDED, for example),
      so naming related packets and function as "LOCK_GRANTED"
      is not proper anymore. Here we rename them to "LOCK_ACK".
      
      No reviewer.
      e36203c3
    • Xin Zhao's avatar
      Bug-fix: add pkt type LOCK in GET_TARGET_WIN_HANDLE macro · 385f0aae
      Xin Zhao authored
      No reviewer.
      385f0aae
    • Xin Zhao's avatar
      Use int instead of size_t in RMA pkt header. · 3a05784f
      Xin Zhao authored
      Use int instead of size_t in RMA pkt header to reduce
      packet size.
      
      No reviewer.
      3a05784f
    • Xin Zhao's avatar
      Bug-fix: add IMMED area in GET/GACC response packets · 87acbbbe
      Xin Zhao authored
      In this patch we allow GET/GACC response packets to
      piggyback some IMMED data, just like what we did
      for PUT/GACC/FOP/CAS packets.
      
      No reviewer.
      87acbbbe
    • Xin Zhao's avatar
      Perf-optimize: support piggybacking LOCK on large RMA operations. · 4739df59
      Xin Zhao authored
      Originally we only allows LOCK request to be piggybacked
      with small RMA operations (all data can be fit in packet
      header). This brings communication overhead for larger
      operations since origin side needs to wait for the LOCK
      ACK before it can transmit data to the target.
      
      In this patch we add support of piggybacking LOCK with
      RMA operations with arbitrary size. Note that (1) this
      only works with basic datatypes; (2) if the LOCK cannot
      be satisfied, we temporarily buffer this operation on
      the target side.
      
      No reviewer.
      4739df59
    • Xin Zhao's avatar
      Clean up unused attributes in RMA packet structs. · b155e7e0
      Xin Zhao authored
      No reviewer.
      b155e7e0
    • Xin Zhao's avatar
      Code-refactor: arrange RMA pkt structure. · 389aab16
      Xin Zhao authored
      Arrange RMA packet definition and structures in
      src/mpid/ch3/include/mpidpkt.h in the following
      order:
      
      1. RMA operation packets: PUT, GET, ACC, GACC, CAS, FOP
      2. RMA operation response packets: GET_RESP, GACC_RESP, CAS_RESP, FOP_RESP
      3. RMA control packets: LOCK, UNLOCK, FLUSH, DECR_AT_COUNTER
      4. RMA control response packets: LOCK_ACK, FLUSH_ACK
      
      No reviewer.
      389aab16
  3. 13 Nov, 2014 1 commit
    • Xin Zhao's avatar
      Perf-tuning: issue FLUSH, FLUSH ACK, UNLOCK ACK messages only when needed. · a9d968cc
      Xin Zhao authored
      
      
      When operation pending list and request lists are all empty, FLUSH message
      needs to be sent by origin only when origin issued PUT/ACC operations since
      the last synchronization calls, otherwise origin does not need to issue FLUSH
      at all and does not need to wait for FLUSH ACK message.
      
      Similiarly, origin waits for ACK of UNLOCK message only when origin issued
      PUT/ACC operations since the last synchronization calls. However, UNLOCK
      message always needs to be sent out because origin needs to unlock the
      target process. This patch avoids issuing unnecessary
      FLUSH / FLUSH ACK / UNLOCK ACK messages.
      Signed-off-by: Pavan Balaji's avatarPavan Balaji <balaji@anl.gov>
      a9d968cc
  4. 03 Nov, 2014 12 commits
    • Xin Zhao's avatar
      Delete no longer needed code. · cc63b367
      Xin Zhao authored
      
      
      We made a huge change to RMA infrastructure and
      a lot of old code can be droped, including separate
      handlers for lock-op-unlock, ACCUM_IMMED specific
      code, O(p) data structure code, code of lazy issuing,
      etc.
      Signed-off-by: Pavan Balaji's avatarPavan Balaji <balaji@anl.gov>
      cc63b367
    • Xin Zhao's avatar
      Rewrite code of passive lock control messages. · 0542e304
      Xin Zhao authored
      
      
      1. Piggyback LOCK request with first IMMED operation.
      
      When we see an IMMED operation, we can always piggyback
      LOCK request with that operation to reduce one sync
      message of single LOCK request. When packet header of
      that operation is received on target, we will try to
      acquire the lock and perform that operation. The target
      either piggybacks LOCK_GRANTED message with the response
      packet (if available), or sends a single LOCK_GRANTED
      message back to origin.
      
      2. Rewrite code of manage lock queue.
      
      When the lock request cannot be satisfied on target,
      we need to buffer that lock request on target. All we
      need to do is enqueuing the packet header, which contains
      all information we need after lock is granted. When
      the current lock is released, the runtime will goes
      over the lock queue and grant the lock to the next
      available request. After lock is granted, the runtime
      just trigger the packet handler for the second time.
      
      3. Release lock on target side if piggybacking with UNLOCK.
      
      If there are active-message operations to be issued,
      we piggyback a UNLOCK flag with the last operation.
      When the target recieves it, it will release the current
      lock and grant the lock to the next process.
      Signed-off-by: Pavan Balaji's avatarPavan Balaji <balaji@anl.gov>
      0542e304
    • Xin Zhao's avatar
      Reset the start of the enum to 0. · 7fbe72dd
      Xin Zhao authored
      
      
      We must make the initial value of enum to zero because some places
      check number of packet types by checking ending type value.
      Signed-off-by: Pavan Balaji's avatarPavan Balaji <balaji@anl.gov>
      7fbe72dd
    • Xin Zhao's avatar
      Rearrange enum of pkt types. · be3e5bdd
      Xin Zhao authored
      
      
      Rearrange the ordering of packet types so that all RMA issuing types
      can be placed together. This is convenient when we check if currently
      involved packets are all RMA packets.
      Signed-off-by: Pavan Balaji's avatarPavan Balaji <balaji@anl.gov>
      be3e5bdd
    • Xin Zhao's avatar
      Add IMMED area in packet header. · e8d4c6d5
      Xin Zhao authored
      
      
      We add a IMMED data area (16 bytes by default) in
      packet header which will contains as much origin
      data as possible. If origin can put all data in
      packet header, then it no longer needs to send
      separate data packet. When target recieves the
      packet header, it will first copy data out from
      the IMMED data area. If there is still more
      data coming, it continues to receive following
      packets; if all data is included in header, then
      recieving is done.
      Signed-off-by: Pavan Balaji's avatarPavan Balaji <balaji@anl.gov>
      e8d4c6d5
    • Xin Zhao's avatar
      Add useful pkt wrappers. · 1c638a12
      Xin Zhao authored
      
      Signed-off-by: Pavan Balaji's avatarPavan Balaji <balaji@anl.gov>
      1c638a12
    • Xin Zhao's avatar
      Decrement Active Target counter at target side. · b73778ea
      Xin Zhao authored
      
      
      During PSCW, when there are active-message operations
      to be issued in Win_complete, we piggback a AT_COMPLETE
      flag with it so that when target receives it, it can
      decrement a counter on target side and detect completion
      when target counter reaches zero.
      Signed-off-by: Pavan Balaji's avatarPavan Balaji <balaji@anl.gov>
      b73778ea
    • Xin Zhao's avatar
      Detect remote completion by FLUSH / FLUSH_ACK messages. · 6578785d
      Xin Zhao authored
      
      
      When the origin wants to do a FLUSH sync, if there are
      active-message operations that are going to be issued,
      we piggback the FLUSH message with the last operation;
      if no such operations, we just send a single FLUSH packet.
      
      If the last operation is a write op (PUT, ACC) or only
      a single FLUSH packet is sent, after target recieves it,
      target will send back a single FLUSH_ACK packet;
      if the last operation contains a read action (GET, GACC, FOP,
      CAS), after target receiveds it, target will piggback a
      FLUSH_ACK flag with the response packet.
      
      After origin receives the FLUSH_ACK packet or response packet
      with FLUSH_ACK flag, it will decrement the counter which
      indicates number of outgoing sync messages (FLUSH / UNLOCK).
      When that counter reaches zero, origin can know that remote
      completion is achieved.
      Signed-off-by: Pavan Balaji's avatarPavan Balaji <balaji@anl.gov>
      6578785d
    • Xin Zhao's avatar
      Split shared RMA packet structures. · c0094faa
      Xin Zhao authored
      
      
      Previously several RMA packet types share the same structure,
      which is misleading for coding. Here make different
      RMA packet types use different packet data structures.
      Signed-off-by: Pavan Balaji's avatarPavan Balaji <balaji@anl.gov>
      c0094faa
    • Xin Zhao's avatar
      Embedding packet structure into RMA operation structure. · b1685139
      Xin Zhao authored
      
      
      We were duplicating information in the operation structure and in the
      packet structure when the message is actually issued.  Since most of
      the information is the same anyway, this patch just embeds a packet
      structure into the operation structure, so that we eliminate unnessary
      copy.
      Signed-off-by: Pavan Balaji's avatarPavan Balaji <balaji@anl.gov>
      b1685139
    • Xin Zhao's avatar
      Rename ACK packets in RMA. · ba1a400c
      Xin Zhao authored
      
      
      The packet type MPIDI_CH3_PKT_PT_RMA_DONE is used for ACK
      of FLUSH / UNLOCK packets. Here we rename it to
      MPIDI_CH3_PKT_FLUSH_ACK and modify the related functions
      and data structures.
      Signed-off-by: Pavan Balaji's avatarPavan Balaji <balaji@anl.gov>
      ba1a400c
    • Xin Zhao's avatar
      Avoid using VC in RMA lock queue structure. · 0eaf344b
      Xin Zhao authored
      
      
      We were adding an unnecessary dependency on VC structure
      declarations in the mpidpkt.h file. The required information
      in RMA lock queue is only the rank, but not actual VC.
      Here we replace VC with rank.
      Signed-off-by: Pavan Balaji's avatarPavan Balaji <balaji@anl.gov>
      0eaf344b
  5. 30 Oct, 2014 1 commit
  6. 31 Jul, 2014 1 commit
    • Wesley Bland's avatar
      Add MPI_Comm_revoke · 57f6ee88
      Wesley Bland authored
      
      
      MPI_Comm_revoke is a special function because it does not have a matching call
      on the "receiving side". This is because it has to act as an out-of-band,
      resilient broadcast algorithm. Because of this, in this commit, in addition to
      the usual functions to implement MPI communication calls (MPI/MPID/CH3/etc.),
      we add a new CH3 packet type that will handle revoking a communicator without
      involving a matching call from the MPI layer (similar to how RMA is currently
      implemented).
      
      The thing that must be handled most carefully when revoking a communicator is
      to ensure that a previously used context ID will eventually be returned to the
      pool of available context IDs and that after this occurs, no old messages will
      match the new usage of the context ID (for instance, if some messages are very
      slow and show up late). To accomplish this, revoke is implemented as an
      all-to-all algorithm. When one process calls revoke, it will send a message to
      all other processes in the communicator, which will trigger that process to
      send a message to all other processes, and so on. Once a process has already
      revoked its communicator locally, it won't send out another wave of messages.
      As each process receives the revoke messages from the other processes, it will
      track how many messages have been received. Once it has either received a
      revoke message or a message about a process failure for each other process, it
      will release its refcount on the communicator object. After the application
      has freed all of its references to the communicator (and all requests, files,
      etc. associated with it), the context ID will be returned to the available
      pool.
      Signed-off-by: default avatarJunchao Zhang <jczhang@mcs.anl.gov>
      57f6ee88
  7. 21 Jul, 2014 4 commits
  8. 18 Jul, 2014 1 commit
  9. 17 Jul, 2014 1 commit
    • Pavan Balaji's avatar
      Simplified RMA_Op structure. · 274a5a70
      Pavan Balaji authored
      
      
      We were creating duplicating information in the operation structure
      and in the packet structure when the message is actually issued.
      Since most of the information is the same anyway, this patch just
      embeds a packet structure into the operation structure.
      Signed-off-by: default avatarXin Zhao <xinzhao3@illinois.edu>
      274a5a70
  10. 21 Feb, 2013 4 commits
    • James Dinan's avatar
      Implemented lock op piggybacking for MODE_NOCHECK · 223fce45
      James Dinan authored
      When the MPI_MODE_NOCHECK assertion is given to a passive target lock
      operation, we defer acquisition of the lock and piggyback the request on
      the first RMA op to the target.  This eliminates a round-trip
      lock-request message.
      
      Reviewer: goodell
      223fce45
    • James Dinan's avatar
      Cleanup of FOP packet header · 422006da
      James Dinan authored
      Removed source_win_handle from the packet header, since it's no longer
      needed.
      
      Reviewer: goodell
      422006da
    • James Dinan's avatar
      RMA sync. piggybacking from origin->target · 4e67607f
      James Dinan authored
      This patch uses packet header flags to piggyback the unlock operation on other
      RMA operations.  For most operations, there is no net change.  However, FOP and
      GACC, unlock piggybacking was previously not implemented.
      
      Reviewer: goodell
      4e67607f
    • James Dinan's avatar
      Added flags field to RMA op packet headers · c3f87fe3
      James Dinan authored
      Added flags field to RMA operation packets that are sent from origin to target.
      This will be used to piggyback RMA synchronization operations.
      
      Reviewer: goodell
      c3f87fe3