GitLab

MPID_nem_ofi_data_callback used to check sreq->cc in order to track progress of
the RTS/CTS/DATA protocol. The was an implicit assumption that fi_tsend with RTS
completes first. However this would cause a hang if fi_trecv completed earlier.
The fix is: don't rely on the cc but rather check the tag bits explicitly.
Note, the RTS/CTS/DATA bits are no longer accumulated (i.e., no more
"wc->tag | CTS/DATA").
Signed-off-by: Charles J Archer <charles.j.archer@intel.com>

This patch modifies the OFI netmod to support large tag layouts, while preserving the old
tag layout.  OFI defines a 64 bit tag, but also provides for a 64 bit tag and immediate data.
In some OFI providers, we may want to select different tag layouts.  This patch currently
does not query for the proper tag layout or attempt to make a choice of the optimal layout,
it provides macro/templatized support for different tag formats.  Additional selection
criteria will be added in subsequent patches.

  * Tag layout is moved to a separate file.
    Added init_sendtag_M2, init_recvtag_M2 (M2 stands for MODE #2, i.e. the mode
    that uses fi_tsenddata and does not pack source into tag).

  * Created a template file for ofi_tagged.c
    Moved do_isend into template file which is included twice into ofi_tagged.c thus providing for the two
    versions of do_isend and do_isend_2 corresponding to the two API sets.

  * All send functions are available in two versions.
    Added macro that declares a function for the two API sets. The first set has the namings inherited from
    the previous netmod version. The functions of the second API set have the "_2" suffix.

  * Recv_posted, anysource_posted, recv_callback, ofi_probe  are templatized.

  * ofi_tag_to_vc renamed ofi_wc_to_vc
    Note, for the API_SET_2 the pgid is stored in the imm data while
    psource and port will be packed the same way as in API_SET_1.

  * Adds api_set member in gl_data struct.  Initialize routines based on api_set

  * Added RCD (RtsCtsData) protocol identifiers

  * Added support for OFI MEM_TAG_FORMAT

  * PGID placement modified
Signed-off-by: Charles J Archer <charles.j.archer@intel.com>

When configured with --enable-thread-cs=per-object, MPICH build will go
wrong because MPID_cc_t will be defined as OPA_int_t, which can not be
directly compared with or assigned with int.

This patch adds one macro,  MPID_cc_get, for correctly reading
MPID_cc_t. Also, change writing to MPID_cc_t to use MPID_cc_set.

Fixes #2250

Signed-off-by: Charles J Archer <charles.j.archer@intel.com>

Charles J Archer authored Nov 17, 2014 and Pavan Balaji committed Nov 21, 2014

 * Implements a tag matching interface netmod over the OFIWG Scalable Fabric Interfaces (SFI)