GitLab maintenance scheduled for Today, 2019-12-05, from 17:00 to 18:00 CT - Services will be unavailable during this time.

codes-vis-readme.md 6.72 KB
Newer Older
Caitlin Ross's avatar
Caitlin Ross committed
1
## README for using ROSS instrumentation with CODES
2

Caitlin Ross's avatar
Caitlin Ross committed
3 4 5
For details about the ROSS instrumentation, see the [ROSS Instrumentation blog post](http://carothersc.github.io/ROSS/feature/instrumentation.html) 
on the ROSS webpage.
 
6 7

There are currently 3 types of instrumentation: GVT-based, real time, and event tracing.  See the ROSS documentation for more info on
Caitlin Ross's avatar
Caitlin Ross committed
8 9 10 11
the specific options or use `--help` with your model. To collect data about the simulation engine, no changes are needed to model code
for any of the instrumentation modes.  Some additions to the model code is needed in order to turn on any model-level data collection.
See the "Model-level data sampling" section on [ROSS Instrumentation blog post](http://carothersc.github.io/ROSS/feature/instrumentation.html).
Here we describe CODES specific details.
12

Caitlin Ross's avatar
Caitlin Ross committed
13
### Register Instrumentation Callback Functions
14

Caitlin Ross's avatar
Caitlin Ross committed
15
The examples here are based on the dragonfly router and terminal LPs (`src/networks/model-net/dragonfly.c`).
16

Caitlin Ross's avatar
Caitlin Ross committed
17
As described in the ROSS Vis documentation, we need to create a `st_model_types` struct with the pointer and size information.
18
```C
Caitlin Ross's avatar
Caitlin Ross committed
19 20
st_model_types dragonfly_model_types[] = {
    {(rbev_trace_f) dragonfly_event_collect,
21
    sizeof(int),
Caitlin Ross's avatar
Caitlin Ross committed
22 23 24 25 26 27 28 29 30 31
    (ev_trace_f) dragonfly_event_collect,
    sizeof(int),
    (model_stat_f) dragonfly_model_stat_collect,
    sizeof(tw_lpid) + sizeof(long) * 2 + sizeof(double) + sizeof(tw_stime) * 2},
    {(rbev_trace_f) dragonfly_event_collect,
    sizeof(int),
    (ev_trace_f) dragonfly_event_collect,
    sizeof(int),
    (model_stat_f) dfly_router_model_stat_collect,
    0}, // updated in router_setup()
32 33 34
    {0}
}
```
Caitlin Ross's avatar
Caitlin Ross committed
35 36 37 38 39 40 41 42 43 44 45 46 47 48 49
`dragonfly_model_types[0]` is the function pointers for the terminal LP and `dragonfly_model_types[1]` is for the router LP.
For the first two function pointers for each LP, we use the same `dragonfly_event_collec()` because right now we just collect the event type, so
it's the same for both of these LPs.  You can change these if you want to use different functions for different LP types or if you want a different
function for the full event tracing than that used for the rollback event trace (`rbev_trace_f` is for the event tracing of rollback triggering events only,
while `ev_trace_f` is for the full event tracing).
The number following each function pointer is the size of the data that will be saved when the function is called.
The third pointer is for the data to be sampled at the GVT or real time sampling points.
In this case the LPs have different function pointers since we want to collect different types of data for the two LP types.
For the terminal, I set the appropriate size of the data to be collected, but for the router, the size of the data is dependent on the radix for the
dragonfly configuration being used, which isn't known until runtime.

*Note*: You can only reuse the function for event tracing for LPs that use the same type of message struct.  
For example, the dragonfly terminal and router LPs both use the `terminal_message` struct, so they can
use the same functions for event tracing.  However the model net base LP uses the `model_net_wrap_msg` struct, so it gets its own event collection function and 
`st_trace_type` struct, in order to read the event type correctly from the model. 
50

Caitlin Ross's avatar
Caitlin Ross committed
51 52 53
In the ROSS instrumentation documentation, there are two methods provided for letting ROSS know about these `st_model_types` structs.  
In CODES, this step is a little different, as `codes_mapping_setup()` calls `tw_lp_settype()`.  
Instead, you add a function to return this struct for each of your LP types:
54
```C
Caitlin Ross's avatar
Caitlin Ross committed
55 56 57 58 59
static const st_model_types *dragonfly_get_model_types(void)
{
    return(&dragonfly_model_types[0]);
}
static const st_model_types *dfly_router_get_model_types(void)
60
{
Caitlin Ross's avatar
Caitlin Ross committed
61
    return(&dragonfly_model_types[1]);
62 63 64
}
```

Caitlin Ross's avatar
Caitlin Ross committed
65
Now you need to add register functions for CODES:
66
```C
Caitlin Ross's avatar
Caitlin Ross committed
67
static void dragonfly_register_model_types(st_model_types *base_type)
68
{
Caitlin Ross's avatar
Caitlin Ross committed
69
    st_model_type_register(LP_CONFIG_NM_TERM, base_type);
70 71
}

Caitlin Ross's avatar
Caitlin Ross committed
72 73 74 75
static void router_register_model_types(st_model_types *base_type)
{
    st_model_type_register(LP_CONFIG_NM_ROUT, base_type);
}
76
```
Caitlin Ross's avatar
Caitlin Ross committed
77
`st_model_type_register(const char* name, const st_trace_type* type)` is part of the CODES API and lets CODES know the pointers for LP initialization.
78

Caitlin Ross's avatar
Caitlin Ross committed
79
At this point, there are two different steps to follow depending on whether the model is one of the model-net models or not.
80

Caitlin Ross's avatar
Caitlin Ross committed
81 82 83
##### Model-net Models
In the `model_net_method` struct, two fields have been added: `mn_model_stat_register` and `mn_get_model_stat_types`.  
You need to set these to the functions described above.  For example:
84 85

```C
Caitlin Ross's avatar
Caitlin Ross committed
86 87 88 89 90 91 92
struct model_net_method dragonfly_method =
{
    .mn_configure = dragonfly_configure,
    // ... all the usual model net stuff
    .mn_model_stat_register = dragonfly_register_model_types,
    .mn_get_model_stat_types = dragonfly_get_model_types,
};
93

Caitlin Ross's avatar
Caitlin Ross committed
94
struct model_net_method dragonfly_router_method =
95
{
Caitlin Ross's avatar
Caitlin Ross committed
96 97 98 99 100
    .mn_configure = NULL,
    // ... all the usual model net stuff
    .mn_model_stat_register = router_register_model_types,
    .mn_get_model_stat_types = dfly_router_get_model_types,
};
101 102
```

Caitlin Ross's avatar
Caitlin Ross committed
103 104 105 106
##### All other CODES models

Using the synthetic workload LP for dragonfly as an example (`src/network-workloads/model-net-synthetic.c`).
In the main function, you call the register function *before* calling `codes_mapping_setup()`.
107
```C
Caitlin Ross's avatar
Caitlin Ross committed
108 109 110 111 112 113 114 115
st_model_types svr_model_types[] = {
    {(rbev_trace_f) svr_event_collect,
    sizeof(int),
    (ev_trace_f) svr_event_collect,
    sizeof(int),
    (model_stat_f) svr_model_stat_collect,
    0},  // at the moment, we're not actually collecting any data about this LP
    {0}
116 117
}

Caitlin Ross's avatar
Caitlin Ross committed
118
static void svr_register_model_types()
119
{
Caitlin Ross's avatar
Caitlin Ross committed
120
    st_model_type_register("server", &svr_model_types[0]);
121 122
}

Caitlin Ross's avatar
Caitlin Ross committed
123
int main(int argc, char **argv)
124
{
Caitlin Ross's avatar
Caitlin Ross committed
125 126 127 128 129 130 131 132 133 134 135 136
    // ... some set up removed for brevity
    
    model_net_register();
    svr_add_lp_type();
    
    if (g_st_ev_trace || g_st_model_stats)
        svr_register_model_types();
        
    codes_mapping_setup();
    
    //...
}
137 138
```

Caitlin Ross's avatar
Caitlin Ross committed
139 140 141
`g_st_ev_trace` is a ROSS flag for determining if event tracing is turned on and `g_st_model_stats` determines if the GVT-based or real time instrumentation
modes are collecting model-level data as well.  

142

Caitlin Ross's avatar
Caitlin Ross committed
143
### CODES LPs that currently have event type collection implemented:
144
If you're using any of the following CODES models, you don't have to add anything, unless you want to change the data that's being collected.
145 146 147 148
- nw-lp (model-net-mpi-replay.c)
- original dragonfly router and terminal LPs (dragonfly.c)
- dfly server LP (model-net-synthetic.c)
- model-net-base-lp (model-net-lp.c)
149
- fat tree server LP (model-net-synthetic-fattree.c)
Caitlin Ross's avatar
Caitlin Ross committed
150 151 152
 
The fat-tree terminal and switch LPs (fattree.c) are only partially implemented at the moment.  It needs two `model_net_method` structs to fully implement, 
but currently both terminal and switch LPs use the same `fattree_method` struct.