add support for DataWarp accounting API
Recent versions of Cray Linux include a DataWarp telemetry API that provides information analogous to what the Lustre module provides. Specifically, dw_get_stripe_configuration(3)
returns
int stripe_size
int stripe_width
int starting_index
And the DataWarp servers to which starting_index
refers can be divined from /proc/mounts
or other kernel interfaces.
Furthermore, CLE 6.0UP04 now includes dw_get_accounting_data_json(3)
which returns aggregate server-side telemetry. While not broken out into a per-file basis, the data types returned may be of interest at some point. An example output is:
{
"realm": {
"namespaces": [
{
"stage_bytes_written": 0,
"namespace_id": 378,
"stripe_width": 2,
"file_size_limit": 0,
"stage_bytes_read": 0,
"stripe_size": 8388608,
"files_create_threshold": 0,
"substripe_width": 12,
"substripe_size": 8388608,
"files_created": 0,
"num_data_created": 0,
"bytes_read": 0,
"max_offset_written": 0,
"bytes_written": 0,
"max_offset_read": 0
}
],
"fragments": [
{
"fs_capacity": 0,
"files_created": 0,
"capacity_used": 0,
"capacity_max": 0,
"max_window_write": 86400,
"write_high_water": 0,
"server_hostname": "c0-0c0s6n2",
"write_moving_avg": 0,
"write_limit": 1730066513920,
"bytes_written": 0,
"bytes_read": 0
},
{
"fs_capacity": 172920012800,
"files_created": 0,
"capacity_used": 34369536,
"capacity_max": 34369536,
"max_window_write": 86400,
"write_high_water": 0,
"server_hostname": "c0-0c0s6n1",
"write_moving_avg": 0,
"write_limit": 1730066513920,
"bytes_written": 0,
"bytes_read": 0
}
],
"realm_id": 378,
"namespace_count": 1,
"server_count": 2
}
}
These sorts of data may be best collected by some other tool (just as we decided client-side file system counters should be).