Commit 9f12dd54 authored by Jakob Luettgau's avatar Jakob Luettgau
Browse files

Add comments, note about extra dependencies.

parent cd732a8f
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
# DarshanUtils for Python # DarshanUtils for Python
This notebook gives an overwiew of features provided by the Python bindings for DarshanUtils. This notebook gives an overwiew of features provided by the Python bindings for DarshanUtils.
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
By default all records, metadata, available modules and the name records are loaded: By default all records, metadata, available modules and the name records are loaded:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
import darshan import darshan
report = darshan.DarshanReport("example-logs/example.darshan", read_all=True) # Default behavior report = darshan.DarshanReport("example-logs/example.darshan", read_all=True) # Default behavior
report.info() report.info()
``` ```
%%%% Output: stream %%%% Output: stream
Filename: example-logs/example.darshan Filename: example-logs/example.darshan
Times: 2017-03-20 10:07:47 to 2017-03-20 10:09:43 (Duration 0:01:56) Times: 2017-03-20 10:07:47 to 2017-03-20 10:09:43 (Duration 0:01:56)
Executeable: /global/project/projectdirs/m888/glock/tokio-abc-results/bin.edison/vpicio_uni /scratch2/scratchdirs/glock/tokioabc-s.4478544/vpicio/vpicio.hdf5 32 Executeable: /global/project/projectdirs/m888/glock/tokio-abc-results/bin.edison/vpicio_uni /scratch2/scratchdirs/glock/tokioabc-s.4478544/vpicio/vpicio.hdf5 32
Processes: 2048 Processes: 2048
JobID: 4478544 JobID: 4478544
UID: 69615 UID: 69615
Modules in Log: ['POSIX', 'MPI-IO', 'LUSTRE', 'STDIO'] Modules in Log: ['POSIX', 'MPI-IO', 'LUSTRE', 'STDIO']
Loaded Records: {'POSIX': 1, 'MPI-IO': 1, 'STDIO': 129} Loaded Records: {'POSIX': 1, 'MPI-IO': 1, 'STDIO': 129}
Name Records: 4 Name Records: 4
Darshan/Hints: {'lib_ver': '3.1.3', 'h': 'romio_no_indep_rw=true;cb_nodes=4'} Darshan/Hints: {'lib_ver': '3.1.3', 'h': 'romio_no_indep_rw=true;cb_nodes=4'}
DarshanReport: id(140346659969064) (tmp) DarshanReport: id(140346659969064) (tmp)
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
report.modules report.modules
``` ```
%%%% Output: execute_result %%%% Output: execute_result
{'POSIX': {'len': 186, 'ver': 3, 'idx': 1, 'num_records': 1}, {'POSIX': {'len': 186, 'ver': 3, 'idx': 1, 'num_records': 1},
'MPI-IO': {'len': 154, 'ver': 2, 'idx': 2, 'num_records': 1}, 'MPI-IO': {'len': 154, 'ver': 2, 'idx': 2, 'num_records': 1},
'LUSTRE': {'len': 87, 'ver': 1, 'idx': 7}, 'LUSTRE': {'len': 87, 'ver': 1, 'idx': 7},
'STDIO': {'len': 3234, 'ver': 1, 'idx': 8, 'num_records': 129}} 'STDIO': {'len': 3234, 'ver': 1, 'idx': 8, 'num_records': 129}}
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
A few of the internal data structures explained: A few of the internal data structures explained:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
# report.metadata # dictionary with raw metadata from darshan log # report.metadata # dictionary with raw metadata from darshan log
# report.modules # dictionary with raw module info from darshan log (need: technical, module idx) # report.modules # dictionary with raw module info from darshan log (need: technical, module idx)
# report.name_records # dictionary for resovling name records: id -> path/name # report.name_records # dictionary for resovling name records: id -> path/name
# report.records # per module "dataframes"/dictionaries holding loaded records # report.records # per module "dataframes"/dictionaries holding loaded records
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
The darshan report holds a variety of namespaces for report related data. All of them are also referenced in `report.data` at the moment, but reliance on this internal organization of the report object is discouraged once the API stabilized. Currently, `report.data` references the following information: The darshan report holds a variety of namespaces for report related data. All of them are also referenced in `report.data` at the moment, but reliance on this internal organization of the report object is discouraged once the API stabilized. Currently, `report.data` references the following information:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
report.data.keys() report.data.keys()
``` ```
%%%% Output: execute_result %%%% Output: execute_result
dict_keys(['version', 'metadata', 'records', 'summary', 'modules', 'counters', 'name_records', 'mounts']) dict_keys(['version', 'metadata', 'records', 'summary', 'modules', 'counters', 'name_records', 'mounts'])
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
report.mod_read_all_records('POSIX') report.mod_read_all_records('POSIX')
``` ```
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
report.mod_read_all_records('STDIO') report.mod_read_all_records('STDIO')
``` ```
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
report.update_name_records() report.update_name_records()
report.info() report.info()
``` ```
%%%% Output: stream %%%% Output: stream
POSIX POSIX
MPI-IO MPI-IO
STDIO STDIO
Filename: example-logs/example.darshan Filename: example-logs/example.darshan
Times: 2017-03-20 10:07:47 to 2017-03-20 10:09:43 (Duration 0:01:56) Times: 2017-03-20 10:07:47 to 2017-03-20 10:09:43 (Duration 0:01:56)
Executeable: /global/project/projectdirs/m888/glock/tokio-abc-results/bin.edison/vpicio_uni /scratch2/scratchdirs/glock/tokioabc-s.4478544/vpicio/vpicio.hdf5 32 Executeable: /global/project/projectdirs/m888/glock/tokio-abc-results/bin.edison/vpicio_uni /scratch2/scratchdirs/glock/tokioabc-s.4478544/vpicio/vpicio.hdf5 32
Processes: 2048 Processes: 2048
JobID: 4478544 JobID: 4478544
UID: 69615 UID: 69615
Modules in Log: ['POSIX', 'MPI-IO', 'LUSTRE', 'STDIO'] Modules in Log: ['POSIX', 'MPI-IO', 'LUSTRE', 'STDIO']
Loaded Records: {'POSIX': 1, 'MPI-IO': 1, 'STDIO': 129} Loaded Records: {'POSIX': 1, 'MPI-IO': 1, 'STDIO': 129}
Name Records: 3 Name Records: 3
Darshan/Hints: {'lib_ver': '3.1.3', 'h': 'romio_no_indep_rw=true;cb_nodes=4'} Darshan/Hints: {'lib_ver': '3.1.3', 'h': 'romio_no_indep_rw=true;cb_nodes=4'}
DarshanReport: id(140346659969064) (tmp) DarshanReport: id(140346659969064) (tmp)
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
# visualization helper used by different examples in the remainder of this notebook # visualization helper used by different examples in the remainder of this notebook
from IPython.display import display, HTML from IPython.display import display, HTML
# usage: display(obj) # usage: display(obj)
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
### Record Formats and Selectively Loading Records ### Record Formats and Selectively Loading Records
For memory efficiant analysis, it is possible to supress records from being loaded automatically. This is useful, for example, when analysis considers only records of a particular layer/module. For memory efficiant analysis, it is possible to supress records from being loaded automatically. This is useful, for example, when analysis considers only records of a particular layer/module.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
import darshan import darshan
report = darshan.DarshanReport("example-logs/example.darshan", read_all=False, lookup_name_records=True) # Loads no records! report = darshan.DarshanReport("example-logs/example.darshan", read_all=False, lookup_name_records=True) # Loads no records!
``` ```
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
# expected to fail, as no records were loaded # expected to fail, as no records were loaded
try: try:
print(len(report.records['STDIO']), "records loaded for STDIO.") print(len(report.records['STDIO']), "records loaded for STDIO.")
except: except:
print("No STDIO records loaded for this report yet.") print("No STDIO records loaded for this report yet.")
``` ```
%%%% Output: stream %%%% Output: stream
No STDIO records loaded for this report yet. No STDIO records loaded for this report yet.
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Additional records then can be loaded selectively, for example, on a per module basis: Additional records then can be loaded selectively, for example, on a per module basis:
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
#### dtype: pandas #### dtype: pandas
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
report.mod_read_all_records("STDIO", dtype="pandas") report.mod_read_all_records("STDIO", dtype="pandas")
``` ```
%%%% Output: stream %%%% Output: stream
STDIO STDIO
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
print('id', report.records['STDIO'][0]['id']) print('id', report.records['STDIO'][0]['id'])
print('rank', report.records['STDIO'][0]['rank']) print('rank', report.records['STDIO'][0]['rank'])
display(report.records['STDIO'][0]['counters']) display(report.records['STDIO'][0]['counters'])
display(report.records['STDIO'][0]['fcounters']) display(report.records['STDIO'][0]['fcounters'])
``` ```
%%%% Output: stream %%%% Output: stream
id -1 id -1
rank -1 rank -1
%%%% Output: display_data %%%% Output: display_data
%%%% Output: display_data %%%% Output: display_data
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
#### dtype: dict #### dtype: dict
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
report.mod_read_all_records("STDIO", dtype='dict') report.mod_read_all_records("STDIO", dtype='dict')
report.records['STDIO'][0] report.records['STDIO'][0]
``` ```
%%%% Output: stream %%%% Output: stream
STDIO STDIO
%%%% Output: execute_result %%%% Output: execute_result
{'id': 15920181672442173319, {'id': 15920181672442173319,
'rank': 0, 'rank': 0,
'counters': {'STDIO_OPENS': 1, 'counters': {'STDIO_OPENS': 1,
'STDIO_FDOPENS': -1, 'STDIO_FDOPENS': -1,
'STDIO_READS': 0, 'STDIO_READS': 0,
'STDIO_WRITES': 6, 'STDIO_WRITES': 6,
'STDIO_SEEKS': 0, 'STDIO_SEEKS': 0,
'STDIO_FLUSHES': 0, 'STDIO_FLUSHES': 0,
'STDIO_BYTES_WRITTEN': 280, 'STDIO_BYTES_WRITTEN': 280,
'STDIO_BYTES_READ': 0, 'STDIO_BYTES_READ': 0,
'STDIO_MAX_BYTE_READ': 0, 'STDIO_MAX_BYTE_READ': 0,
'STDIO_MAX_BYTE_WRITTEN': 279, 'STDIO_MAX_BYTE_WRITTEN': 279,
'STDIO_FASTEST_RANK': 0, 'STDIO_FASTEST_RANK': 0,
'STDIO_FASTEST_RANK_BYTES': 0, 'STDIO_FASTEST_RANK_BYTES': 0,
'STDIO_SLOWEST_RANK': 0, 'STDIO_SLOWEST_RANK': 0,
'STDIO_SLOWEST_RANK_BYTES': 0}, 'STDIO_SLOWEST_RANK_BYTES': 0},
'fcounters': {'STDIO_F_META_TIME': 0.0, 'fcounters': {'STDIO_F_META_TIME': 0.0,
'STDIO_F_WRITE_TIME': 6.794929504394531e-05, 'STDIO_F_WRITE_TIME': 6.794929504394531e-05,
'STDIO_F_READ_TIME': 0.0, 'STDIO_F_READ_TIME': 0.0,
'STDIO_F_OPEN_START_TIMESTAMP': 0.0, 'STDIO_F_OPEN_START_TIMESTAMP': 0.0,
'STDIO_F_CLOSE_START_TIMESTAMP': 0.0, 'STDIO_F_CLOSE_START_TIMESTAMP': 0.0,
'STDIO_F_WRITE_START_TIMESTAMP': 0.07752799987792969, 'STDIO_F_WRITE_START_TIMESTAMP': 0.07752799987792969,
'STDIO_F_READ_START_TIMESTAMP': 0.0, 'STDIO_F_READ_START_TIMESTAMP': 0.0,
'STDIO_F_OPEN_END_TIMESTAMP': 0.0, 'STDIO_F_OPEN_END_TIMESTAMP': 0.0,
'STDIO_F_CLOSE_END_TIMESTAMP': 0.0, 'STDIO_F_CLOSE_END_TIMESTAMP': 0.0,
'STDIO_F_WRITE_END_TIMESTAMP': 116.28358292579651, 'STDIO_F_WRITE_END_TIMESTAMP': 116.28358292579651,
'STDIO_F_READ_END_TIMESTAMP': 0.0, 'STDIO_F_READ_END_TIMESTAMP': 0.0,
'STDIO_F_FASTEST_RANK_TIME': 0.0, 'STDIO_F_FASTEST_RANK_TIME': 0.0,
'STDIO_F_SLOWEST_RANK_TIME': 0.0, 'STDIO_F_SLOWEST_RANK_TIME': 0.0,
'STDIO_F_VARIANCE_RANK_TIME': 0.0, 'STDIO_F_VARIANCE_RANK_TIME': 0.0,
'STDIO_F_VARIANCE_RANK_BYTES': 0.0}} 'STDIO_F_VARIANCE_RANK_BYTES': 0.0}}
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
``` ```
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
#### dtype: numpy #### dtype: numpy
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
report.mod_read_all_records("STDIO") report.mod_read_all_records("STDIO")
report.records['STDIO'][0] report.records['STDIO'][0]
``` ```
%%%% Output: execute_result %%%% Output: execute_result
{'id': 15920181672442173319, {'id': 15920181672442173319,
'rank': 0, 'rank': 0,
'counters': array([ 1, -1, 0, 6, 0, 0, 280, 0, 0, 279, 0, 0, 0, 'counters': array([ 1, -1, 0, 6, 0, 0, 280, 0, 0, 279, 0, 0, 0,
0]), 0]),
'fcounters': array([0.00000000e+00, 6.79492950e-05, 0.00000000e+00, 0.00000000e+00, 'fcounters': array([0.00000000e+00, 6.79492950e-05, 0.00000000e+00, 0.00000000e+00,
0.00000000e+00, 7.75279999e-02, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 7.75279999e-02, 0.00000000e+00, 0.00000000e+00,
0.00000000e+00, 1.16283583e+02, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 1.16283583e+02, 0.00000000e+00, 0.00000000e+00,
0.00000000e+00, 0.00000000e+00, 0.00000000e+00])} 0.00000000e+00, 0.00000000e+00, 0.00000000e+00])}
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
#### The Log in Memory #### The Log in Memory
Let's have a look at how calling `report.mod_read_all_records("STDIO")` changed the state of the log in memory. Let's have a look at how calling `report.mod_read_all_records("STDIO")` changed the state of the log in memory.
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
# Compare to info line: "Loaded Records: {...}" # Compare to info line: "Loaded Records: {...}"
report.info() report.info()
``` ```
%%%% Output: stream %%%% Output: stream
Filename: example-logs/example.darshan Filename: example-logs/example.darshan
Times: 2017-03-20 10:07:47 to 2017-03-20 10:09:43 (Duration 0:01:56) Times: 2017-03-20 10:07:47 to 2017-03-20 10:09:43 (Duration 0:01:56)
Executeable: /global/project/projectdirs/m888/glock/tokio-abc-results/bin.edison/vpicio_uni /scratch2/scratchdirs/glock/tokioabc-s.4478544/vpicio/vpicio.hdf5 32 Executeable: /global/project/projectdirs/m888/glock/tokio-abc-results/bin.edison/vpicio_uni /scratch2/scratchdirs/glock/tokioabc-s.4478544/vpicio/vpicio.hdf5 32
Processes: 2048 Processes: 2048
JobID: 4478544 JobID: 4478544
UID: 69615 UID: 69615
Modules in Log: ['POSIX', 'MPI-IO', 'LUSTRE', 'STDIO'] Modules in Log: ['POSIX', 'MPI-IO', 'LUSTRE', 'STDIO']
Loaded Records: {'STDIO': 129} Loaded Records: {'STDIO': 129}
Name Records: 2 Name Records: 2
Darshan/Hints: {'lib_ver': '3.1.3', 'h': 'romio_no_indep_rw=true;cb_nodes=4'} Darshan/Hints: {'lib_ver': '3.1.3', 'h': 'romio_no_indep_rw=true;cb_nodes=4'}
DarshanReport: id(139926751139712) (tmp) DarshanReport: id(139926751139712) (tmp)
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
When interacting on individual log data for example in a for loop you would most likely care about the following instead: When interacting on individual log data for example in a for loop you would most likely care about the following instead:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
print("Num records:", len(report.records['STDIO'])) print("Num records:", len(report.records['STDIO']))
# show first 10 records # show first 10 records
for rec in report.records['STDIO'][0:10]: for rec in report.records['STDIO'][0:10]:
print(rec) print(rec)
``` ```
%%%% Output: stream %%%% Output: stream
Num records: 129 Num records: 129
{'id': 15920181672442173319, 'rank': 0, 'counters': array([ 1, -1, 0, 6, 0, 0, 280, 0, 0, 279, 0, 0, 0, {'id': 15920181672442173319, 'rank': 0, 'counters': array([ 1, -1, 0, 6, 0, 0, 280, 0, 0, 279, 0, 0, 0,
0]), 'fcounters': array([0.00000000e+00, 6.79492950e-05, 0.00000000e+00, 0.00000000e+00, 0]), 'fcounters': array([0.00000000e+00, 6.79492950e-05, 0.00000000e+00, 0.00000000e+00,
0.00000000e+00, 7.75279999e-02, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 7.75279999e-02, 0.00000000e+00, 0.00000000e+00,
0.00000000e+00, 1.16283583e+02, 0.00000000e+00, 0.00000000e+00, 0.00000000e+00, 1.16283583e+02, 0.00000000e+00, 0.00000000e+00,
0.00000000e+00, 0.00000000e+00, 0.00000000e+00])} 0.00000000e+00, 0.00000000e+00, 0.00000000e+00])}
{'id': 7238257241479193519, 'rank': 0, 'counters': array([ 1, -1, 0, 68, 0, 0, 3029, 0, 0, 3028, 0, {'id': 7238257241479193519, 'rank': 0, 'counters': array([ 1, -1, 0, 68, 0, 0, 3029, 0, 0, 3028, 0,
0, 0, 0]), 'fcounters': array([ 0. , -2662.74663377, 0. , 0. , 0, 0, 0]), 'fcounters': array([ 0. , -2662.74663377, 0. , 0. ,
0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. ,
0. , 0. , 0. , 0. , 0. , 0. , 0. , 0. ,
0. , 0. , 0. ])} 0. , 0. , 0. ])}
{'id': 7238257241479193519, 'rank': 16, 'counters': array([ 1, -1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]), 'fcounters': array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.])} {'id': 7238257241479193519, 'rank': 16, 'counters': array([ 1, -1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]), 'fcounters': array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.])}
{'id': 7238257241479193519, 'rank': 32, 'counters': array([ 1, -1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]), 'fcounters': array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.])} {'id': 7238257241479193519, 'rank': 32, 'counters': array([ 1, -1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]), 'fcounters': array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.])}
{'id': 7238257241479193519, 'rank': 48, 'counters': array([ 1, -1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]), 'fcounters': array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.])} {'id': 7238257241479193519, 'rank': 48, 'counters': array([ 1, -1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]), 'fcounters': array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.])}
{'id': 7238257241479193519, 'rank': 64, 'counters': array([ 1, -1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]), 'fcounters': array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.])} {'id': 7238257241479193519, 'rank': 64, 'counters': array([ 1, -1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]), 'fcounters': array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.])}
{'id': 7238257241479193519, 'rank': 80, 'counters': array([ 1, -1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]), 'fcounters': array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.])} {'id': 7238257241479193519, 'rank': 80, 'counters': array([ 1, -1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]), 'fcounters': array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.])}
{'id': 7238257241479193519, 'rank': 96, 'counters': array([ 1, -1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]), 'fcounters': array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.])} {'id': 7238257241479193519, 'rank': 96, 'counters': array([ 1, -1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]), 'fcounters': array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.])}
{'id': 7238257241479193519, 'rank': 112, 'counters': array([ 1, -1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]), 'fcounters': array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.])} {'id': 7238257241479193519, 'rank': 112, 'counters': array([ 1, -1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]), 'fcounters': array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.])}
{'id': 7238257241479193519, 'rank': 128, 'counters': array([ 1, -1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]), 'fcounters': array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.])} {'id': 7238257241479193519, 'rank': 128, 'counters': array([ 1, -1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0]), 'fcounters': array([0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0., 0.])}
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
### Aggregation and Filtering (Experimental) ### Aggregation and Filtering (Experimental)
Darshan log data is routinely aggregated for quick overview. The report object offers a few methods to perform common aggregations: Darshan log data is routinely aggregated for quick overview. The report object offers a few methods to perform common aggregations:
%% Cell type:markdown id: tags: %% Cell type:markdown id: tags:
Report aggregations and summarization remains **experimental** for now, mostly to allow interfaces to stabilize. But experimental features can be switched on easily by invoking `darshan.enable_experimental()`: Report aggregations and summarization remains **experimental** for now, mostly to allow interfaces to stabilize. But experimental features can be switched on easily by invoking `darshan.enable_experimental()`:
%% Cell type:code id: tags: %% Cell type:code id: tags:
``` python ``` python
import darshan import darshan
darshan.enable_experimental(verbose=True) # Enable verbosity, listing new functionality darshan.enable_experimental(verbose=True) # Enable verbosity, listing new functionality
``` ```
%%%% Output: stream %%%% Output: stream
Added method create_time_summary to DarshanReport. Added method create_time_summary to DarshanReport.