GitLab

some configure cleanup as well as modification of darshan
linker flags when lustre module is enabled

Fixes #270

Philip Carns authored Mar 16, 2021 and Shane Snyder committed Mar 16, 2021

- exit with error by default in parsers
- show more verbose error message with tips to help
- add --show-incomplete option to enable previous behavior
- properly set incomplete flag for modules that don't store any records

Fixes #243

Previously, modules would explicitly disable instrumentation when
ever Darshan core called into the module to shut it down. The
issue is that Darshan core starts shutting down well before modules
do, so there is a time interval where modules could be trying to
instrument data while its not safe to do so.

Now, we expose a function (darshan_core_disabled_instrumentation)
that modules can call to determine if its safe to instrument
data.

Signed-off-by: Cong Xu <cong.xu@intel.com>

2. Refactor DXLT to be a generic IO tracing tool for files on any file system.
3. Add file type information in DXLT log parser.

Next steps:
1. Based on the file type information in DXLT log parser, add information for specific file types. (For instance, add OST information for files on Lustre)
2. In the process of parsing DXLT logs, figure out how to obtain Lustre striping information from Lustre module.
3. Rename DXLT to be "Darshan eXtended Tracing (DXT)" or "Darshan eXtended IO Tracing (DXIOT)".
Signed-off-by: Cong Xu <cong.xu@intel.com>

Signed-off-by: Cong Xu <cong.xu@intel.com>

1. Use Lustre Module in Darshan
2. Integrate DXLT IO tracing code into one module
3. Add hooks in POSIX and MPI Modules
4. Add Intel copyright
Signed-off-by: Cong Xu <cong.xu@intel.com>

1. Use Lustre Module in Darshan
2. Integrate DXLT IO tracing code into one module
3. Add hooks in POSIX and MPI Modules
4. Add Intel copyright
Signed-off-by: Cong Xu <cong.xu@intel.com>

There are cases where lustre stripe info is not available the
first time a given file is opened (e.g., when using the
llapi_file_create call). So, now we call into the lustre module
every time a file is opened to give the module a chance to gather
data.

fixed bugs that prevented record names from mapping to file systems in darshan-util; also fixed thread safety and deadlock issues

implemented a simple get_output_data that doesn't use MPI; all ranks except rank0 just drop their shared records

partly working lustre module supporting ost mapping; shared file agg still doesn't work, and log parsing doesn't either

added filename hashing so multiple opens/closes don't create multiple lustre-mod records; added proper reducer so that shared files are flattened into a single set of counters whose values are dictated by rank 0

the POSIX module will call into the Lustre module for each file
record created that corresponds to a file from a Lustre FS. No
counters are currently stored but most of the module strucutre
has been stubbed.