Skip to content
Projects
Groups
Snippets
Help
Loading...
Help
Support
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
C
codes
Project overview
Project overview
Details
Activity
Releases
Repository
Repository
Files
Commits
Branches
Tags
Contributors
Graph
Compare
Issues
38
Issues
38
List
Boards
Labels
Milestones
Merge Requests
8
Merge Requests
8
Analytics
Analytics
Repository
Value Stream
Wiki
Wiki
Members
Members
Collapse sidebar
Close sidebar
Activity
Graph
Create a new issue
Commits
Issue Boards
Open sidebar
codes
codes
Commits
ec9569ec
Commit
ec9569ec
authored
Jul 07, 2014
by
mubarak
Browse files
Options
Browse Files
Download
Email Patches
Plain Diff
Adding readme file for the dragonfly model + some minor changes in dragonfly
parent
de35ac6f
Changes
6
Hide whitespace changes
Inline
Side-by-side
Showing
6 changed files
with
118 additions
and
7 deletions
+118
-7
codes/model-net.h
codes/model-net.h
+1
-0
doc/.README.dragonfly.txt.swp
doc/.README.dragonfly.txt.swp
+0
-0
doc/README.dragonfly.txt
doc/README.dragonfly.txt
+100
-0
src/models/networks/model-net/dragonfly.c
src/models/networks/model-net/dragonfly.c
+6
-6
src/models/networks/model-net/model-net.c
src/models/networks/model-net/model-net.c
+10
-1
tests/conf/modelnet-test-dragonfly.conf
tests/conf/modelnet-test-dragonfly.conf
+1
-0
No files found.
codes/model-net.h
View file @
ec9569ec
...
...
@@ -124,6 +124,7 @@ struct dragonfly_param
int
global_vc_size
;
/* buffer size of the global channels */
int
cn_vc_size
;
/* buffer size of the compute node channels */
short
routing
;
/* minimal or non-minimal routing */
short
chunk_size
;
/* full-sized packets are broken into smaller chunks.*/
};
struct
torus_param
...
...
doc/.README.dragonfly.txt.swp
0 → 100644
View file @
ec9569ec
File added
doc/README.dragonfly.txt
0 → 100644
View file @
ec9569ec
*** README file for dragonfly network model ***
This file describes the setup and configuration for the ROSS dragonfly network model.
1- Model of the dragonfly network topology
The dragonfly is a hierarchical topology having several groups connected by
all-to-all links. Each router has “p” number of compute nodes connected to it,
and there are “a” routers in each group. Routers within a group are connected
via local channels. Each router has “h” global channels, which are intergroup
connections through which routers in a group connect to routers in other
groups. Thus, the radix of each router is k = a + p + h − 1. For load balancing
the network traffic, the recommended dragonfly configuration is a = 2p = 2h.
Overall, the total number of groups “g” in the network is g = a ∗ h+1. Since
each group has “p” nodes and “a” routers, the total number of nodes ’N’ in the
network is determined by N = p ∗ a ∗ g.
Our ROSS dragonfly model uses the configuration a=2p=2h for modeling the
dragonfly topology. Full-sized network packets (default size: 512 bytes) are
broken into smaller packet chunks (default size: 32 bytes) for transportation
over the network. ROSS dragonfly model supports three different forms of
routing: minimal: packet is sent directly from the source group to destination
group over the single global channel connecting the source and destination
groups. non-minimal: packet is first sent to an intermediate group and then to
the destination group. This type of routing helps to load balance the network
traffic under some traffic patterns which congest the single global channel
connecting the two groups. adaptive routing: a congestion sensing algorithm is
used to choose the minimal or non-minimal path for the packet.
ROSS models are made up of a collection of logical processes (LPs). Each LP
models a distinct component of the system. LPs interact with one another
through events in the form of timestamped messages.In the dragonfly model, each
LP represents an individual router or node and each time-stamped message
represents a packet sent to/from a node/router.
2- Configuring ROSS dragonfly network model
CODES dragonfly network model can be configured using the dragonfly config file (currently
located in codes-net/tests/conf). To adjust the network size, configure the MODELNET_GRP
section of the config file as well as the 'num_routers' parameter in the PARAMS section in
the config file. For example, for a dragonfly configuration of a=8, p=4, h=4, we have 33 groups
(g=a*h+1=8*4+1), 264 routers (total_routers=a*g=4*33=264), 1056 network nodes (total_nodes=
p*a*g=8*4*33). This configuration can be specified in the config file in the following way
MODELNET_GRP
{
repetitions="264";
server="4";
modelnet_dragonfly="4";
dragonfly_router="1";
}
PARAMS
{
....
num_routers="8";
....
}
The first section, MODELNET_GRP specified the number of LPs and the layout of LPs. In the above
case, there are 264 repetitions of 4 server LPs, 4 dragonfly network node LPs and 1 dragonfly
router LP, which makes a total of 264 routers, 1056 network nodes and 1056 servers in the network.
The second section, PARAMS uses 'num_routers' for the dragonfly topology lay out and setsup the
connections between routers, network nodes and the servers.
Some other dragonfly specific parameters in the PARAMS section are
- packet_size: size of a dragonfly packet in bytes (default set to 512 bytes)
- num_vcs: number of virtual channels connecting a router-router, node-router (default set to 1)
- local_vc_size: Number of packet chunks (default: 32 bytes) that can fit in the channel connecting routers
within the same group.
- chunk_size: A full-sized packet of 'packet_size' is divided into smaller packet chunks for transporation
(default set to 64 bytes).
- global_vc_size: Number of packet chunks (default: 32 bytes) that can fit in the global channel connecting
two groups with each other.
- cn_vc_size: Number of packet chunks (default: 32 bytes) that can fit in the channel connecting the network
node with its router.
- local_bandwidth: bandwidth of the channels connecting the routers within the same group.
- global_bandwidth: bandwidth of the global channels connecting routers of two different groups. Note than
each router has 'h' number of global channels connected to it where a=2p=2h in our configuration.
- cn_bandwidth: bandwidth of the channel connecing the compute node with the router.
** All the above bandwidth parameters are in Gigabytes/sec.
- routing: the routing algorithm can be minimal, nonminimal or adaptive.
3- Running ROSS dragonfly network model
- To run the dragonfly network model with the model-net test program, the following options are available
ROSS serial mode:
./tests/modelnet-test --sync=1 -- tests/conf/modelnet-test-dragonfly.conf
ROSS conservative mode:
mpirun -np 8 tests/modelnet-test --sync=2 -- tests/conf/modelnet-test-dragonfly.conf
ROSS optimistic mode:
mpirun -np 8 tests/modelnet-test --sync=3 -- tests/conf/modelnet-test-dragonfly.conf
4- Performance optimization tips for ROSS dragonfly model
- For large-scale dragonfly runs, the model has significant speedup in optimistic mode than the conservative mode.
- If
src/models/networks/model-net/dragonfly.c
View file @
ec9569ec
...
...
@@ -17,7 +17,6 @@
#include "codes/model-net-lp.h"
#include "codes/net/dragonfly.h"
#define CHUNK_SIZE 32.0
#define CREDIT_SIZE 8
#define MEAN_PROCESS 1.0
...
...
@@ -47,7 +46,7 @@ static int radix=0;
/* number of virtual channels, number of routers comes from the
* config file, number of compute nodes, global channels and group
* is calculated from these configurable parameters */
static
int
num_vcs
,
num_routers
,
num_cn
,
num_global_channels
,
num_groups
;
static
int
num_vcs
,
num_routers
,
num_cn
,
num_global_channels
,
num_groups
,
chunk_size
;
/* adaptive threshold is to bias the adaptive routing */
static
int
total_routers
,
adaptive_threshold
=
10
;
...
...
@@ -212,6 +211,7 @@ static void dragonfly_setup(const void* net_params)
local_vc_size
=
d_param
->
local_vc_size
;
cn_vc_size
=
d_param
->
cn_vc_size
;
routing
=
d_param
->
routing
;
chunk_size
=
d_param
->
chunk_size
;
radix
=
num_vcs
*
(
num_cn
+
num_global_channels
+
num_routers
);
total_routers
=
num_groups
*
num_routers
;
...
...
@@ -460,7 +460,7 @@ void packet_generate(terminal_state * s, tw_bf * bf, terminal_message * msg, tw_
tw_event
*
e
;
terminal_message
*
m
;
int
i
,
total_event_size
;
num_chunks
=
msg
->
packet_size
/
CHUNK_SIZE
;
num_chunks
=
msg
->
packet_size
/
chunk_size
;
msg
->
packet_ID
=
lp
->
gid
+
g_tw_nlp
*
s
->
packet_counter
+
tw_rand_integer
(
lp
->
rng
,
0
,
lp
->
gid
+
g_tw_nlp
*
s
->
packet_counter
);
msg
->
travel_start_time
=
tw_now
(
lp
);
msg
->
my_N_hop
=
0
;
...
...
@@ -541,7 +541,7 @@ void packet_send(terminal_state * s, tw_bf * bf, terminal_message * msg, tw_lp *
// Each packet is broken into chunks and then sent over the channel
msg
->
saved_available_time
=
s
->
terminal_available_time
;
head_delay
=
(
1
/
cn_bandwidth
)
*
CHUNK_SIZE
;
head_delay
=
(
1
/
cn_bandwidth
)
*
chunk_size
;
ts
=
head_delay
+
tw_rand_exponential
(
lp
->
rng
,
(
double
)
head_delay
/
200
);
s
->
terminal_available_time
=
max
(
s
->
terminal_available_time
,
tw_now
(
lp
));
s
->
terminal_available_time
+=
ts
;
...
...
@@ -1258,7 +1258,7 @@ if( msg->packet_ID == TRACK && next_stop != msg->dest_terminal_id && msg->chunk_
#endif
// If source router doesn't have global channel and buffer space is available, then assign to appropriate intra-group virtual channel
msg
->
saved_available_time
=
s
->
next_output_available_time
[
output_port
];
ts
=
g_tw_lookahead
+
0
.
1
+
((
1
/
bandwidth
)
*
CHUNK_SIZE
)
+
tw_rand_exponential
(
lp
->
rng
,
(
double
)
CHUNK_SIZE
/
200
);
ts
=
g_tw_lookahead
+
0
.
1
+
((
1
/
bandwidth
)
*
chunk_size
)
+
tw_rand_exponential
(
lp
->
rng
,
(
double
)
chunk_size
/
200
);
s
->
next_output_available_time
[
output_port
]
=
max
(
s
->
next_output_available_time
[
output_port
],
tw_now
(
lp
));
s
->
next_output_available_time
[
output_port
]
+=
ts
;
...
...
@@ -1333,7 +1333,7 @@ router_packet_receive( router_state * s,
msg
->
my_N_hop
++
;
ts
=
g_tw_lookahead
+
0
.
1
+
tw_rand_exponential
(
lp
->
rng
,
(
double
)
MEAN_INTERVAL
/
200
);
num_chunks
=
msg
->
packet_size
/
CHUNK_SIZE
;
num_chunks
=
msg
->
packet_size
/
chunk_size
;
if
(
msg
->
packet_ID
==
TRACK
&&
msg
->
chunk_id
==
num_chunks
-
1
)
printf
(
"
\n
packet %lld chunk %d received at router %d "
,
msg
->
packet_ID
,
msg
->
chunk_id
,
(
int
)
lp
->
gid
);
...
...
src/models/networks/model-net/model-net.c
View file @
ec9569ec
...
...
@@ -331,7 +331,7 @@ int model_net_set_params()
else
if
(
strcmp
(
model_net_method_names
[
DRAGONFLY
],
mn_name
)
==
0
)
{
dragonfly_param
net_params
;
int
num_routers
=
0
,
num_vcs
=
0
,
local_vc_size
=
0
,
global_vc_size
=
0
,
cn_vc_size
=
0
;
int
num_routers
=
0
,
num_vcs
=
0
,
local_vc_size
=
0
,
global_vc_size
=
0
,
cn_vc_size
=
0
,
chunk_size
=
0
;
double
local_bandwidth
=
0
.
0
,
cn_bandwidth
=
0
.
0
,
global_bandwidth
=
0
.
0
;
configuration_get_value_int
(
&
config
,
"PARAMS"
,
"num_routers"
,
&
num_routers
);
...
...
@@ -374,6 +374,14 @@ int model_net_set_params()
}
net_params
.
cn_vc_size
=
cn_vc_size
;
configuration_get_value_int
(
&
config
,
"PARAMS"
,
"chunk_size"
,
&
chunk_size
);
if
(
!
chunk_size
)
{
chunk_size
=
64
;
printf
(
"
\n
Chunk size for packets is specified, setting to %d "
,
chunk_size
);
}
net_params
.
chunk_size
=
chunk_size
;
configuration_get_value_double
(
&
config
,
"PARAMS"
,
"local_bandwidth"
,
&
local_bandwidth
);
if
(
!
local_bandwidth
)
{
...
...
@@ -398,6 +406,7 @@ int model_net_set_params()
}
net_params
.
cn_bandwidth
=
cn_bandwidth
;
char
routing
[
MAX_NAME_LENGTH
];
configuration_get_value
(
&
config
,
"PARAMS"
,
"routing"
,
routing
,
MAX_NAME_LENGTH
);
if
(
strcmp
(
routing
,
"minimal"
)
==
0
)
...
...
tests/conf/modelnet-test-dragonfly.conf
View file @
ec9569ec
...
...
@@ -14,6 +14,7 @@ PARAMS
modelnet
=
"dragonfly"
;
# scheduler options
modelnet_scheduler
=
"fcfs"
;
chunk_size
=
"32"
;
# modelnet_scheduler="round-robin";
num_vcs
=
"1"
;
num_routers
=
"4"
;
...
...
Write
Preview
Markdown
is supported
0%
Try again
or
attach a new file
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment