GitLab maintenance scheduled for Today, 2019-09-24, from 12:00 to 13:00 CT - Services will be unavailable during this time.

Commit b4a0aed4 authored by Shane Snyder's avatar Shane Snyder

first cut at complete dev-modular documentation

parent 657e1b1a
all: darshan-util-lib darshan-posix-parser
#all: darshan-parser darshan-convert darshan-diff darshan-analyzer darshan-log-params darshan-util-lib
DESTDIR =
srcdir = @srcdir@
......
......@@ -17,6 +17,8 @@
/* default to a compression buffer size of 4 MiB */
#define DARSHAN_DEF_DECOMP_BUF_SZ (4*1024*1024)
/* TODO: we need to refactor s.t. utilities don't know implementation
of this, but module-specific functions do */
struct darshan_fd_s
{
gzFile gzf;
......@@ -35,12 +37,12 @@ struct darshan_record_ref
};
darshan_fd darshan_log_open(const char *name, const char* mode);
int darshan_log_getheader(darshan_fd file, struct darshan_header *header);
int darshan_log_getjob(darshan_fd file, struct darshan_job *job);
int darshan_log_getheader(darshan_fd fd, struct darshan_header *header);
int darshan_log_getjob(darshan_fd fd, struct darshan_job *job);
int darshan_log_getexe(darshan_fd fd, char *buf);
int darshan_log_getmounts(darshan_fd fd, char*** mnt_pts,
char*** fs_types, int* count);
int darshan_log_gethash(darshan_fd file, struct darshan_record_ref **hash);
int darshan_log_gethash(darshan_fd fd, struct darshan_record_ref **hash);
int darshan_log_get_moddat(darshan_fd fd, darshan_module_id mod_id,
void *moddat_buf, int moddat_buf_sz);
void darshan_log_close(darshan_fd file);
......
......@@ -15,6 +15,7 @@
#include <getopt.h>
#include <assert.h>
#include "darshan-logutils.h"
#include "darshan-posix-logutils.h"
#include "uthash-1.9.2/src/uthash.h"
......
......@@ -287,7 +287,80 @@ simplifying maintenance.
=== Darshan-util
Text.
The darshan-util component is composed of a log parsing library (libdarshan-util) and a
corresponding set of utility programs that can parse and analyze Darshan I/O characterization
logs using this library. The log parsing library includes a generic interface (see
`darshan-logutils.h`) for retrieving specific portions of a given log file. Specifically,
this interface allows utilities to retrieve a log's header metadata, job details, record
identifier mapping, and any module-specific data contained within the log.
Module developers may wish to define additional interfaces for parsing module-specific data
that can then be integrated into the log parsing library. This extended functionality can be
implemented in terms of the generic functions offered by darshan-logutils and by module-specific
formatting information.
==== darshan-logutils
Here we define each function of the `darshan-logutils` interface, which can be used to implement
new log analysis utilities and to define module-specific functionality.
[source,c]
darshan_fd darshan_log_open(const char *name, const char* mode);
Opens Darshan log file stored at path `name`. `mode` can only be `r` for reading or `w` for
writing. Returns a Darshan file descriptor on success, or `NULL` on error.
[source,c]
int darshan_log_getheader(darshan_fd fd, struct darshan_header *header);
Fills in `header` structure from the log file referenced by descriptor `fd`. The `darshan_header`
structure is defined in `darshan-log-format.h`. Returns `0` on success, `-1` on failure.
[source,c]
int darshan_log_getjob(darshan_fd fd, struct darshan_job *job);
Fills in `job` structure from the log file referenced by descriptor `fd`. The `darshan_job`
structure is defined in `darshan-log-format.h`. Returns `0` on success, `-1` on failure.
[source,c]
int darshan_log_getexe(darshan_fd fd, char *buf);
Fills in `buf` with the corresponding executable string (exe name and command line arguments)
for the Darshan log referenced by `fd`. Returns `0` on success, `-1` on failure.
[source,c]
int darshan_log_getmounts(darshan_fd fd, char*** mnt_pts, char*** fs_types, int* count);
Returns mounted file system information for the Darshan log referenced by `fd`. `mnt_pnts` points
to an array of strings storing mount points, `fs_types` points to an array of strings storing file
system types (e.g., ext4, nfs, etc.), and `count` points to an integer storing the total number
of mounted file systems recorded by Darshan. Returns `0` on success, `-1` on failure.
[source,c]
int darshan_log_gethash(darshan_fd fd, struct darshan_record_ref **hash);
Builds hash table of Darshan record identifiers to full names for all records contained
in the Darshan log referenced by `fd`. `hash` is a pointer to the hash table (of type
struct darshan_record_ref *, which should be initialized to `NULL`). This hash table is
defined by the `uthash` hash table implementation and includes corresponding macros for
searching, iterating, and deleting records from the hash. For detailed documentation on using this
hash table, consult `uthash` documentation in `darshan-util/uthash-1.9.2/doc/txt/userguide.txt`.
The `darshan-posix-parser` utility (for parsing POSIX module information out of a Darshan log)
provides an example of how this hash table may be used. Returns `0` on success, `-1` on failure.
[source,c]
int darshan_log_get_moddat(darshan_fd fd, darshan_module_id mod_id, void *moddat_buf, int moddat_buf_sz);
Retrieves a chunk of (uncompressed) data for the module identified by `mod_id` from the Darshan log
referenced by `fd`. `moddat_buf_sz` specifies the number of uncompressed bytes to read from the file
and store in `moddat_buf`. This function may be repeatedly called to retrieve sequential chunks of data
from a given Darshan file descriptor. This function returns `1` if `moddat_buf_sz` bytes were read
successfully, `0` if no more data is available for this module, and `-1` otherwise.
[source,c]
void darshan_log_close(darshan_fd fd);
Close Darshan file descriptor `fd`. Returns `0` on success, `-1` on failure.
== Adding new instrumentation modules
......@@ -318,9 +391,6 @@ of `CP_WRAPPERS` in `darshan-config.in`.
source files, which will be stored in the `lib/` directory. The prerequisites to building
static and dynamic versions of `lib-darshan` must be updated to include these objects, as well.
It is necessary to rerun the `prepare` script and reconfigure darshan-runtime for these changes
to take effect.
==== Instrumentation module implementation
An exemplar instrumentation module for POSIX I/O functions is given in `lib/darshan-posix.c` as
......@@ -347,7 +417,22 @@ to store the record structure in a hash table or some other structure.
==== Build modifications
Text.
The following modifications to the darshan-util build system are necessary to integrate new
instrumentation modules:
* Update `Makefile.in` with new targets necessary for building new utilities and module-specific
log parsing source.
- Make sure to update `all`, `clean`, and `install` targets to reference updates.
- If adding new log parsing functionality, make sure to add it as a prerequisite source for
building libdarshan-util.
==== Module-specific logutils and utilities
For a straightforward reference implementation of module-specific log parsing functionality for
the POSIX module, consider files `darshan-posix-logutils.c` and `darshan-posix-logutils.h`.
Also, the `darshan-posix-parser` source provides a simple example of a utility which can leverage
libdarshan-util for analyzing the contents of a given Darshan I/O characterization log.
== Other resources
......
Markdown is supported
0% or
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment