Reviewed by Mike Blocksome @ IBM.

We now provide two types of function pointers: _sched and _req.  The
_sched functions just add instructions to the schedule, but no request
is created.  The _req functions return an MPID request.

Reviewed by Mike Blocksome @ IBM.

Michael Blocksome authored Nov 01, 2012 and Pavan Balaji committed May 15, 2013

The following collectives are updated:

  MPIX_Ibcast
  MPIX_Iallgather
  MPIX_Iallgatherv
  MPIX_Iallreduce
  MPIX_Ialltoall
  MPIX_Ialltoallv
  MPIX_Ialltoallw
  MPIX_Iexscan
  MPIX_Igather
  MPIX_Igatherv
  MPIX_Ireduce_scatter_block
  MPIX_Ireduce_scatter
  MPIX_Ireduce
  MPIX_Iscan
  MPIX_Iscatter
  MPIX_Iscatterv

These collectives are updated similar to how the optimized MPIX_Ibarrier
is implemented:

| The previous MPIR_Ibarrier_impl() function forced all adi implementations
| to create a MPID_Sched_t opaque object which was then passed in to the
| specific ibarrier implementation via a function pointer table.
|
| The MPID_Sched_t object represents a completely new state machine that
| must be advanced whenever mpi progress is made.
|
| The required construction of the MPID_Sched_t object and the required
| advance of the schedule state machine would be extremely detrimental to
| pamid performance.

(ibm) 47b497e15cd87df666c6031a28f5f8a5ae53cece
(ibm) aee872eb91f4300066957349e5a3d8103da33f40

Signed-off-by: Charles Archer <archerc@us.ibm.com>

moved some error messages which should have been mpix to mpi.

Reviewed by dinan.

MPI_.

A slightly older version of this patch was reviewed by goodell.

In several places, after checking for a parameter (e.g., comm) we were
directly using it assuming that the parameter is valid.  Since the
previous ERRTEST macros did not jump to fn_fail on an error, this
could result in undefined behavior if the parameter was invalid.  Now,
since we jump on errors within the macros themselves, once the check
is done, we know that the parameter values are valid.

Reviewed by buntinas.

Pulled bit manipulation arithmetic into macros for converting handles to
indices in MPIR_Op and MPIR_Op_check_dtype function pointer tables.

Reviewer: buntinas

since we don't need to maintain backward compatibility here.
Hardcoded them to const, so they are better tested.

Reviewed by goodell.

disabled for the public interfaces, but all internal functions use
const.

No reviewer.

This avoids segfaults in some cases.  Found by the clang static
analyzer.

No reviewer.

This is a direct port of the recursive-doubling algorithm used for MPI_Exscan.

Reviewed by buntinas@.

This also allocates tags values per-comm, since NBC post order is only
required to be consistent on a per-communicator basis.

Reviewed by buntinas@.

This change consists of four main parts:

 1. Updating all of the MPIR_Ifoo_impl routines to assume that all
    nonblocking collective functions are set in the coll_fns field.

 2. Defining the MPID_Sched_ portion of the ADI in mpir_nbc.h.

 3. Implementing that interface in src/mpid/common/sched

 4. Modifications to ch3:nemesis and ch3:sock in order to call the
    progress engine hook for the scheduling mechanism, as well as
    correctly including the various scheduling headers in CH3 headers.

Reviewed by balaji@.

Reviewed by balaji@.

Reviewed by balaji@.

These functions build but will yield an error at runtime if invoked.
They will correctly call any collective function overrides specified in
comm_ptr->coll_fns, permitting devices to experiment with NBC
implementation before a complete implementation is available above the
ADI.

Reviewed by buntinas@.