codes-vis-readme.md 6.4 KB
Newer Older
Caitlin Ross's avatar
Caitlin Ross committed
1
## README for using ROSS instrumentation with CODES
2

3
For details about the ROSS instrumentation, see the [ROSS Instrumentation blog post](http://carothersc.github.io/ROSS/instrumentation/instrumentation.html) 
Caitlin Ross's avatar
Caitlin Ross committed
4 5
on the ROSS webpage.
 
6

7 8 9 10 11
There are currently 4 types of instrumentation: GVT-based, real time sampling, virtual time sampling, and event tracing.
See the ROSS documentation for more info on the specific options or use `--help` with your model.
To collect data about the simulation engine, no changes are needed to model code for any of the instrumentation modes.
Some additions to the model code is needed in order to turn on any model-level data collection.
See the "Model-level data sampling" section on [ROSS Instrumentation blog post](http://carothersc.github.io/ROSS/instrumentation/instrumentation.html).
Caitlin Ross's avatar
Caitlin Ross committed
12
Here we describe CODES specific details.
13

Caitlin Ross's avatar
Caitlin Ross committed
14
### Register Instrumentation Callback Functions
15

Caitlin Ross's avatar
Caitlin Ross committed
16
The examples here are based on the dragonfly router and terminal LPs (`src/networks/model-net/dragonfly.c`).
17

Caitlin Ross's avatar
Caitlin Ross committed
18
As described in the ROSS Vis documentation, we need to create a `st_model_types` struct with the pointer and size information.
19
```C
Caitlin Ross's avatar
Caitlin Ross committed
20
st_model_types dragonfly_model_types[] = {
21
    {(ev_trace_f) dragonfly_event_collect,
Caitlin Ross's avatar
Caitlin Ross committed
22 23 24
    sizeof(int),
    (model_stat_f) dragonfly_model_stat_collect,
    sizeof(tw_lpid) + sizeof(long) * 2 + sizeof(double) + sizeof(tw_stime) * 2},
25
    {(ev_trace_f) dragonfly_event_collect,
Caitlin Ross's avatar
Caitlin Ross committed
26 27 28
    sizeof(int),
    (model_stat_f) dfly_router_model_stat_collect,
    0}, // updated in router_setup()
29 30 31
    {0}
}
```
Caitlin Ross's avatar
Caitlin Ross committed
32
`dragonfly_model_types[0]` is the function pointers for the terminal LP and `dragonfly_model_types[1]` is for the router LP.
33 34 35 36
For the first function pointer for each LP type, we use the same `dragonfly_event_collect()` because right now we just collect the event type, so it's the same for both of these LP types.
You can change these if you want to use different functions for different LP types.
The number following that function pointer is the size of the data that will be saved when the function is called.
The second pointer is for the data to be sampled at the GVT or real time sampling points.
Caitlin Ross's avatar
Caitlin Ross committed
37
In this case the LPs have different function pointers since we want to collect different types of data for the two LP types.
38
For the terminal, I set the appropriate size of the data to be collected, but for the router, the size of the data is dependent on the radix for the dragonfly configuration being used, which isn't known until runtime.
Caitlin Ross's avatar
Caitlin Ross committed
39 40 41

*Note*: You can only reuse the function for event tracing for LPs that use the same type of message struct.  
For example, the dragonfly terminal and router LPs both use the `terminal_message` struct, so they can
42 43
use the same functions for event tracing.  
However the model net base LP uses the `model_net_wrap_msg` struct, so it gets its own event collection function and `st_trace_type` struct, in order to read the event type correctly from the model. 
44

Caitlin Ross's avatar
Caitlin Ross committed
45 46 47
In the ROSS instrumentation documentation, there are two methods provided for letting ROSS know about these `st_model_types` structs.  
In CODES, this step is a little different, as `codes_mapping_setup()` calls `tw_lp_settype()`.  
Instead, you add a function to return this struct for each of your LP types:
48
```C
Caitlin Ross's avatar
Caitlin Ross committed
49 50 51 52 53
static const st_model_types *dragonfly_get_model_types(void)
{
    return(&dragonfly_model_types[0]);
}
static const st_model_types *dfly_router_get_model_types(void)
54
{
Caitlin Ross's avatar
Caitlin Ross committed
55
    return(&dragonfly_model_types[1]);
56 57 58
}
```

Caitlin Ross's avatar
Caitlin Ross committed
59
Now you need to add register functions for CODES:
60
```C
Caitlin Ross's avatar
Caitlin Ross committed
61
static void dragonfly_register_model_types(st_model_types *base_type)
62
{
Caitlin Ross's avatar
Caitlin Ross committed
63
    st_model_type_register(LP_CONFIG_NM_TERM, base_type);
64 65
}

Caitlin Ross's avatar
Caitlin Ross committed
66 67 68 69
static void router_register_model_types(st_model_types *base_type)
{
    st_model_type_register(LP_CONFIG_NM_ROUT, base_type);
}
70
```
Caitlin Ross's avatar
Caitlin Ross committed
71
`st_model_type_register(const char* name, const st_trace_type* type)` is part of the CODES API and lets CODES know the pointers for LP initialization.
72

Caitlin Ross's avatar
Caitlin Ross committed
73
At this point, there are two different steps to follow depending on whether the model is one of the model-net models or not.
74

Caitlin Ross's avatar
Caitlin Ross committed
75 76 77
##### Model-net Models
In the `model_net_method` struct, two fields have been added: `mn_model_stat_register` and `mn_get_model_stat_types`.  
You need to set these to the functions described above.  For example:
78 79

```C
Caitlin Ross's avatar
Caitlin Ross committed
80 81 82 83 84 85 86
struct model_net_method dragonfly_method =
{
    .mn_configure = dragonfly_configure,
    // ... all the usual model net stuff
    .mn_model_stat_register = dragonfly_register_model_types,
    .mn_get_model_stat_types = dragonfly_get_model_types,
};
87

Caitlin Ross's avatar
Caitlin Ross committed
88
struct model_net_method dragonfly_router_method =
89
{
Caitlin Ross's avatar
Caitlin Ross committed
90 91 92 93 94
    .mn_configure = NULL,
    // ... all the usual model net stuff
    .mn_model_stat_register = router_register_model_types,
    .mn_get_model_stat_types = dfly_router_get_model_types,
};
95 96
```

Caitlin Ross's avatar
Caitlin Ross committed
97 98 99 100
##### All other CODES models

Using the synthetic workload LP for dragonfly as an example (`src/network-workloads/model-net-synthetic.c`).
In the main function, you call the register function *before* calling `codes_mapping_setup()`.
101
```C
Caitlin Ross's avatar
Caitlin Ross committed
102
st_model_types svr_model_types[] = {
103
    {(ev_trace_f) svr_event_collect,
Caitlin Ross's avatar
Caitlin Ross committed
104 105 106 107
    sizeof(int),
    (model_stat_f) svr_model_stat_collect,
    0},  // at the moment, we're not actually collecting any data about this LP
    {0}
108 109
}

Caitlin Ross's avatar
Caitlin Ross committed
110
static void svr_register_model_types()
111
{
Caitlin Ross's avatar
Caitlin Ross committed
112
    st_model_type_register("server", &svr_model_types[0]);
113 114
}

Caitlin Ross's avatar
Caitlin Ross committed
115
int main(int argc, char **argv)
116
{
Caitlin Ross's avatar
Caitlin Ross committed
117 118 119 120 121 122 123 124 125 126 127 128
    // ... some set up removed for brevity
    
    model_net_register();
    svr_add_lp_type();
    
    if (g_st_ev_trace || g_st_model_stats)
        svr_register_model_types();
        
    codes_mapping_setup();
    
    //...
}
129 130
```

Caitlin Ross's avatar
Caitlin Ross committed
131 132 133
`g_st_ev_trace` is a ROSS flag for determining if event tracing is turned on and `g_st_model_stats` determines if the GVT-based or real time instrumentation
modes are collecting model-level data as well.  

134

Caitlin Ross's avatar
Caitlin Ross committed
135
### CODES LPs that currently have event type collection implemented:
136
If you're using any of the following CODES models, you don't have to add anything, unless you want to change the data that's being collected.
137 138
- nw-lp (model-net-mpi-replay.c)
- dfly server LP (model-net-synthetic.c)
139
- custom dfly server LP (model-net-synthetic-custom-dfly.c)
140
- fat tree server LP (model-net-synthetic-fattree.c)
141 142 143 144 145 146
- slimfly server LP (model-net-synthetic-slimfly.c)
- original dragonfly router and terminal LPs (dragonfly.c)
- dragonfly custom router and terminal LPs (dragonfly-custom.C)
- slimfly router and terminal LPs (slimfly.c)
- fat tree switch and terminal LPs (fat-tree.c)
- model-net-base-lp (model-net-lp.c)
Caitlin Ross's avatar
Caitlin Ross committed
147