Commit 6dd6e0ab authored by Shane Snyder's avatar Shane Snyder

update darshan-util docs to describe DxT

parent 36e039ff
......@@ -587,6 +587,106 @@ The `--file-list-detailed` is the same as --file-list except that it
produces many columns of output containing statistics broken down by file.
This option is mainly useful for more detailed automated analysis.
=== darshan-dxt-parser
The `darshan-dxt-parser` utility can be used to parse DxT traces out of Darshan
log files, assuming the corresponding application was executed with the DxT
modules enabled. The following example parses all DxT trace information out
of a Darshan log file and stores it in a text file:
----
darshan-dxt-parser shane_ior_id25016_1-31-38066-13864742673678115131_1.darshan > ~/ior-trace.txt
----
=== Guide to darshan-dxt-parser output
The preamble to `darshan-dxt-parser` output is identical to that of the traditional
`darshan-parser` utility, which is described above.
`darshan-dxt-parser` displays detailed trace information contained within a Darshan log
that was generated with DxT instrumentation enabled. Trace data is captured from both
POSIX and MPI-IO interfaces. Example output is given below:
.Example output
----
# ***************************************************
# DXT_POSIX module data
# ***************************************************
# DXT, file_id: 16457598720760448348, file_name: /tmp/test/testFile
# DXT, rank: 0, hostname: shane-thinkpad
# DXT, write_count: 4, read_count: 4
# DXT, mnt_pt: /, fs_type: ext4
# Module Rank Wt/Rd Segment Offset Length Start(s) End(s)
X_POSIX 0 write 0 0 262144 0.0029 0.0032
X_POSIX 0 write 1 262144 262144 0.0032 0.0035
X_POSIX 0 write 2 524288 262144 0.0035 0.0038
X_POSIX 0 write 3 786432 262144 0.0038 0.0040
X_POSIX 0 read 0 0 262144 0.0048 0.0048
X_POSIX 0 read 1 262144 262144 0.0049 0.0049
X_POSIX 0 read 2 524288 262144 0.0049 0.0050
X_POSIX 0 read 3 786432 262144 0.0050 0.0051
# ***************************************************
# DXT_MPIIO module data
# ***************************************************
# DXT, file_id: 16457598720760448348, file_name: /tmp/test/testFile
# DXT, rank: 0, hostname: shane-thinkpad
# DXT, write_count: 4, read_count: 4
# DXT, mnt_pt: /, fs_type: ext4
# Module Rank Wt/Rd Segment Length Start(s) End(s)
X_MPIIO 0 write 0 262144 0.0029 0.0032
X_MPIIO 0 write 1 262144 0.0032 0.0035
X_MPIIO 0 write 2 262144 0.0035 0.0038
X_MPIIO 0 write 3 262144 0.0038 0.0040
X_MPIIO 0 read 0 262144 0.0048 0.0049
X_MPIIO 0 read 1 262144 0.0049 0.0049
X_MPIIO 0 read 2 262144 0.0049 0.0050
X_MPIIO 0 read 3 262144 0.0050 0.0051
----
===== DxT POSIX module
This module provides details on each read or write access at the POSIX layer.
The trace output is organized first by file then by process rank. So, for each
file accessed by the application, DxT will provide each process's I/O trace
segments in separate blocks, ordered by increasing process rank. Within each
file/rank block, I/O trace segments are ordered chronologically.
Before providing details on each I/O operation, DxT provides a short preamble
for each file/rank trace block with the following bits of information: the Darshan
identifier for the file (which is equivalent to the identifers used by Darshan in its
traditional modules), the full file path, the corresponding MPI rank the current
block of trace data belongs to, the hostname associated with this process rank, the
number of individual POSIX read and write operations by this process, and the mount
point and file system type corresponding to the traced file.
The output format for each indvidual I/O operation segment is:
----
# Module Rank Wt/Rd Segment Offset Length Start(s) End(s)
----
* Module: corresponding DxT module (DXT_POSIX or DXT_MPIIO)
* Rank: process rank responsible for I/O operation
* Wt/Rd: whether the operation was a write or read
* Segment: The operation number for this segment (first operation is segment 0)
* Offset: file offset the I/O operation occured at
* Length: length of the I/O operation in bytes
* Start: timestamp of the start of the operation (w.r.t. application start time)
* End: timestamp of the end of the operation (w.r.t. application start time)
===== DxT MPI-IO module
If the MPI-IO interface is used by an application, this module provides details on
each read or write access at the MPI-IO layer. This data is often useful in
understanding how MPI-IO read or write operations map to underlying POSIX read
or write operations issued to the traced file.
The output format for the DxT MPI-IO module is essentially identical to the DxT
POSIX module, except that the offset of file operations is not tracked.
=== Other darshan-util utilities
The darshan-util package includes a number of other utilies that can be
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment