This commit adds functionality to the dragonfly plus model. The
functionality is to make whether source group rerouting and
intermediate router choice a configurable option that is set
in the config file.

To use said configurables, use these config variables in your
config file:

	source_group_rerouting= {"off" | "on"} default="on";
	intermediate_choice= {"leaf" | "spine" | "both"} default="leaf";

If source group rerouting is on, then if a packet is currently being
routed by a spine router in the source group, but the spine router
does not have a connection to the intermediate group, then the spine
will route the packet to another spine router within the source
group who does have said connection to the intermediate group.

If source group rerouting if off, then if a packet being currently routed
by a spine router in the source group cannot be directly routed to the
pre specified intermediate group in the message, then a new intermediate
router/group will be chosen from the list of possible intermediate
groups/routers that the source spine has connections to. It is not
rerouted within the source group.

Intermediate choice determines what type of routers can be chosen
as an intermediate router. If "leaf" is set, then intermediate routers
will only be of type leafs. Similar for "spine". If "both" is selected
then it could be either.

Additional features added in this commit:

New configurable:
	notification_on_hops_greater_than={"<int>"} default = MAX_INT

This new configurable sets the maximum number of hops allowed before a
printed notification of the number of hops that a current packet
is at is output to stdout. This is to help debug and know when routing
decisions are producing unexpected results.

notes:
- this generator will issue MPI-IO events, which cannot yet be handled
  directly by CODES simulations but are honored by the
  codes-workload-mpi-replay utility
- the codes-workload-mpi-replay utility now includes a --prep argument
  that will generate files that need to be read for subsequent replay

- contributed by Jian Peng <jpeng10@hawk.iit.edu>

For simplest case, only 2 functions and premable changes should suffice to add a
new network. Update Express Mesh to serve as an example.

Change-Id: I48de25b4c00360201cb23a4a7089ddc08300137d

-- Multi rail
-- Tapering
-- Within node handling

-- Some commits include fixes in other networks

Fixing ordering issue with waits (should resolve the unmatched sends/receives error with traces like minife)

Fixing ordering issue with waits (should resolve the unmatched sends/receives error with traces like minife)

changed MPI_COMM_WORLD into MPI_COMM_CODES and made model-net-mpi-replay a function instead of a main