Skip to content
GitLab
Projects
Groups
Snippets
/
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
darshan
darshan
Commits
6806ff0c
Commit
6806ff0c
authored
Dec 09, 2016
by
Cong Xu
Browse files
Merge branch 'master' of
https://xgitlab.cels.anl.gov/darshan/darshan
parents
6d33b9e9
55960ccf
Changes
20
Hide whitespace changes
Inline
Side-by-side
ChangeLog
View file @
6806ff0c
...
...
@@ -2,13 +2,27 @@
Darshan Release Change Log
--------------------------
Darshan-3.1.2
=============
* bug fix in resolving underlying call to fopen64 when using the LD_PRELOAD
instrumentation mechanism
* bug fix to make sure Lustre module headers are installed when installing
darshan-util
- reported by Matthieu Dorrier
Darshan-3.1.1
=============
* bug fix to MPIIO_F_WRITE_START_TIMESTAMP, which may have produced incorrect
timestamps in some cases
- reported by Wucherl (William) Yoo
Darshan-3.1.0
=============
* add stdio I/O library instrumentation module (Philip Carns)
- this handles instrumentation of file stream I/O functions
like fopen(), fprintf(), fscanf(), etc.
- this module also captures stats on the standard streams (stdin,
stdout, & std
in
)
stdout, & std
err
)
* add Lustre instrumentation module (Glenn Lockwood)
- this module provides Lustre striping details (e.g., stripe
width, stripe size, list of OSTs a file is striped over)
...
...
darshan-mpiio-log-format.h
View file @
6806ff0c
...
...
@@ -8,7 +8,7 @@
#define __DARSHAN_MPIIO_LOG_FORMAT_H
/* current MPI-IO log format version */
#define DARSHAN_MPIIO_VER
1
#define DARSHAN_MPIIO_VER
2
/* TODO: maybe use a counter to track cases in which a derived datatype is used? */
...
...
darshan-runtime/configure
View file @
6806ff0c
#! /bin/sh
# Guess values for system-dependent variables and create Makefiles.
# Generated by GNU Autoconf 2.69 for darshan-runtime 3.1.
0
.
# Generated by GNU Autoconf 2.69 for darshan-runtime 3.1.
2
.
#
#
# Copyright (C) 1992-1996, 1998-2012 Free Software Foundation, Inc.
...
...
@@ -577,8 +577,8 @@ MAKEFLAGS=
# Identity of this package.
PACKAGE_NAME
=
'darshan-runtime'
PACKAGE_TARNAME
=
'darshan-runtime'
PACKAGE_VERSION
=
'3.1.
0
'
PACKAGE_STRING
=
'darshan-runtime 3.1.
0
'
PACKAGE_VERSION
=
'3.1.
2
'
PACKAGE_STRING
=
'darshan-runtime 3.1.
2
'
PACKAGE_BUGREPORT
=
''
PACKAGE_URL
=
''
...
...
@@ -1259,7 +1259,7 @@ if test "$ac_init_help" = "long"; then
# Omit some internal or obsolete options to make the list less imposing.
# This message is too long to be a string in the A/UX 3.1 sh.
cat
<<
_ACEOF
\`
configure' configures darshan-runtime 3.1.
0
to adapt to many kinds of systems.
\`
configure' configures darshan-runtime 3.1.
2
to adapt to many kinds of systems.
Usage:
$0
[OPTION]... [VAR=VALUE]...
...
...
@@ -1321,7 +1321,7 @@ fi
if
test
-n
"
$ac_init_help
"
;
then
case
$ac_init_help
in
short
|
recursive
)
echo
"Configuration of darshan-runtime 3.1.
0
:"
;;
short
|
recursive
)
echo
"Configuration of darshan-runtime 3.1.
2
:"
;;
esac
cat
<<
\
_ACEOF
...
...
@@ -1335,7 +1335,7 @@ Optional Features:
Set log files to be group readable
--disable-bgq-mod Disables compilation and use of BG/Q module (for BG/Q systems)
--disable-lustre-mod Disables compilation and use of the Lustre module
--enable-mm
m
ap-logs Enables ability to mmap I/O data to log file
--enable-mmap-logs
Enables ability to mmap I/O data to log file
Optional Packages:
--with-PACKAGE[=ARG] use PACKAGE [ARG=yes]
...
...
@@ -1431,7 +1431,7 @@ fi
test
-n
"
$ac_init_help
"
&&
exit
$ac_status
if
$ac_init_version
;
then
cat
<<
\
_ACEOF
darshan-runtime configure 3.1.
0
darshan-runtime configure 3.1.
2
generated by GNU Autoconf 2.69
Copyright (C) 2012 Free Software Foundation, Inc.
...
...
@@ -1783,7 +1783,7 @@ cat >config.log <<_ACEOF
This file contains any messages produced by compilers while
running configure, to aid debugging if configure makes a mistake.
It was created by darshan-runtime
$as_me
3.1.
0
, which was
It was created by darshan-runtime
$as_me
3.1.
2
, which was
generated by GNU Autoconf 2.69. Invocation command line was
$ $0
$@
...
...
@@ -4360,7 +4360,7 @@ else
MPICH_LIB_OLD
=
0
fi
DARSHAN_VERSION
=
"3.1.
0
"
DARSHAN_VERSION
=
"3.1.
2
"
...
...
@@ -4880,7 +4880,7 @@ cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1
# report actual input values of CONFIG_FILES etc. instead of their
# values after options handling.
ac_log="
This file was extended by darshan-runtime
$as_me
3.1.
0
, which was
This file was extended by darshan-runtime
$as_me
3.1.
2
, which was
generated by GNU Autoconf 2.69. Invocation command line was
CONFIG_FILES =
$CONFIG_FILES
...
...
@@ -4942,7 +4942,7 @@ _ACEOF
cat
>>
$CONFIG_STATUS
<<
_ACEOF
|| ac_write_fail=1
ac_cs_config="`
$as_echo
"
$ac_configure_args
" | sed 's/^ //; s/[
\\
""
\`\$
]/
\\\\
&/g'`"
ac_cs_version="
\\
darshan-runtime config.status 3.1.
0
darshan-runtime config.status 3.1.
2
configured by
$0
, generated by GNU Autoconf 2.69,
with options
\\
"
\$
ac_cs_config
\\
"
...
...
darshan-runtime/configure.in
View file @
6806ff0c
...
...
@@ -5,7 +5,7 @@ dnl Process this file with autoconf to produce a configure script.
dnl You may need to use autoheader as well if changing any DEFINEs
dnl sanity checks, output header, location of scripts used here
AC_INIT([darshan-runtime], [3.1.
0
])
AC_INIT([darshan-runtime], [3.1.
2
])
AC_CONFIG_SRCDIR([darshan.h])
AC_CONFIG_AUX_DIR(../maint/config)
AC_CONFIG_HEADER(darshan-runtime-config.h)
...
...
@@ -61,7 +61,7 @@ AC_ARG_ENABLE(
)
AC_ARG_ENABLE(mmap-logs,
[ --enable-mm
m
ap-logs Enables ability to mmap I/O data to log file],
[ --enable-mmap-logs
Enables ability to mmap I/O data to log file],
[if test "x$enableval" = "xyes" ; then
AC_DEFINE(__DARSHAN_ENABLE_MMAP_LOGS, 1, Define if Darshan should mmap data structures to log file)
ENABLE_MMAP_LOGS=1
...
...
darshan-runtime/doc/darshan-runtime.txt
View file @
6806ff0c
...
...
@@ -219,7 +219,7 @@ should be compiled using the normal, unmodified MPI compiler.
To use this mechanism, set the `LD_PRELOAD` environment variable to the full
path to the Darshan shared library. The preferred method of inserting Darshan
instrumentation in this case is to set the LD_PRELOAD variable specifically
instrumentation in this case is to set the
`
LD_PRELOAD
`
variable specifically
for the application of interest. Typically this is possible using
command line arguments offered by the `mpirun` or `mpiexec` scripts or by
the job scheduler:
...
...
@@ -249,6 +249,11 @@ unrelated binaries:
export LD_PRELOAD=/home/carns/darshan-install/lib/libdarshan.so
----
[NOTE]
For SGI systems running the MPT environment, it may be necessary to set the `MPI_SHEPHERD`
environment variable equal to `true` to avoid deadlock when preloading the Darshan shared
library.
=== Instrumenting dynamically-linked Fortran applications
Please follow the general steps outlined in the previous section. For
...
...
@@ -444,8 +449,9 @@ method has been briefly tested using both GNU and Intel compilers.
[NOTE]
====
Darshan is only known to work with C and C++ executables generated by the
Intel MPI suite. Darshan will not produce instrumentation for Fortran
executables. For more details please check this Intel forum discussion:
Intel MPI suite in versions prior to the 2017 version -- Darshan will not
produce instrumentation for Fortran executables in these earlier versions (pre-2017).
For more details on this issue please check this Intel forum discussion:
http://software.intel.com/en-us/forums/showthread.php?t=103447&o=a&s=lr
====
...
...
darshan-runtime/lib/darshan-mpiio.c
View file @
6806ff0c
...
...
@@ -210,7 +210,7 @@ static int enable_dxt_io_trace = 0;
rec_ref->file_rec->counters[MPIIO_RW_SWITCHES] += 1; \
rec_ref->last_io_type = DARSHAN_IO_WRITE; \
if(rec_ref->file_rec->fcounters[MPIIO_F_WRITE_START_TIMESTAMP] == 0 || \
rec_ref->file_rec->fcounters[MPIIO_F_
READ
_START_TIMESTAMP] > __tm1) \
rec_ref->file_rec->fcounters[MPIIO_F_
WRITE
_START_TIMESTAMP] > __tm1) \
rec_ref->file_rec->fcounters[MPIIO_F_WRITE_START_TIMESTAMP] = __tm1; \
rec_ref->file_rec->fcounters[MPIIO_F_WRITE_END_TIMESTAMP] = __tm2; \
if(rec_ref->file_rec->fcounters[MPIIO_F_MAX_WRITE_TIME] < __elapsed) { \
...
...
darshan-test/example-output/mpi-io-test-ppc64-3.1.1.darshan
0 → 100644
View file @
6806ff0c
File added
darshan-test/example-output/mpi-io-test-ppc64-3.1.2.darshan
0 → 100644
View file @
6806ff0c
File added
darshan-test/example-output/mpi-io-test-x86_64-3.1.1.darshan
0 → 100644
View file @
6806ff0c
File added
darshan-test/example-output/mpi-io-test-x86_64-3.1.2.darshan
0 → 100644
View file @
6806ff0c
File added
darshan-test/regression/run-all.sh
View file @
6806ff0c
...
...
@@ -45,17 +45,6 @@ fi
# set up environment for tests according to platform
source
$DARSHAN_TESTDIR
/
$DARSHAN_PLATFORM
/env.sh
# in case of using LD_PRELOAD, it is possible errors with Darshan symbols
# (e..g, forgetting to resolve a symbol Darshan wraps using dlsym) can
# cause arbitrary binaries to crash. We check the output of the true
# command to successfully identify failures like this and exit
true_out
=
`
/bin/true 2>&1
`
if
[
$?
-ne
0
-o
-n
"
$true_out
"
]
;
then
echo
-n
$true_out
>
tmp
echo
"environment setup failed"
exit
1
fi
failure_count
=
0
for
i
in
`
ls
$DARSHAN_TESTDIR
/test-cases/
*
.sh
`
;
do
...
...
darshan-test/regression/workstation-dynamic/env.sh
View file @
6806ff0c
...
...
@@ -38,3 +38,20 @@ FULL_MPICC_PATH=`which mpicc`
export
LD_PRELOAD
=
`
dirname
$FULL_MPICC_PATH
`
/../lib/libfmpich.so:
$DARSHAN_PATH
/lib/libdarshan.so:
$LD_PRELOAD
export
DARSHAN_RUNJOB
=
"mpiexec -n
$DARSHAN_DEFAULT_NPROCS
"
# in case of using LD_PRELOAD, it is possible errors with Darshan symbols
# (e..g, forgetting to resolve a symbol Darshan wraps using dlsym) can
# cause arbitrary binaries to crash. We check the output of the true
# command and ls to try to identify failures like this and exit
true_out
=
`
true
`
if
[
$?
-ne
0
-o
-n
"
$true_out
"
]
;
then
echo
-n
"
$true_out
"
echo
"true command test failed"
exit
1
fi
ls_out
=
`
ls
1>/dev/null
`
if
[
$?
-ne
0
-o
-n
"
$ls_out
"
]
;
then
echo
-n
"
$ls_out
"
echo
"ls command test failed"
exit
1
fi
darshan-util/Makefile.in
View file @
6806ff0c
...
...
@@ -155,6 +155,7 @@ endif
install
-m
644
$(srcdir)
/darshan-pnetcdf-logutils.h
$(includedir)
install
-m
644
$(srcdir)
/darshan-bgq-logutils.h
$(includedir)
install
-m
644
$(srcdir)
/darshan-null-logutils.h
$(includedir)
install
-m
644
$(srcdir)
/darshan-lustre-logutils.h
$(includedir)
install
-m
644
$(srcdir)
/darshan-stdio-logutils.h
$(includedir)
install
-m
644
$(srcdir)
/darshan-dxt-logutils.h
$(includedir)
install
-m
644
$(srcdir)
/../darshan-null-log-format.h
$(includedir)
...
...
@@ -163,6 +164,7 @@ endif
install
-m
644
$(srcdir)
/../darshan-hdf5-log-format.h
$(includedir)
install
-m
644
$(srcdir)
/../darshan-pnetcdf-log-format.h
$(includedir)
install
-m
644
$(srcdir)
/../darshan-bgq-log-format.h
$(includedir)
install
-m
644
$(srcdir)
/../darshan-lustre-log-format.h
$(includedir)
install
-m
644
$(srcdir)
/../darshan-stdio-log-format.h
$(includedir)
install
-m
644
$(srcdir)
/../darshan-dxt-log-format.h
$(includedir)
install
-d
$(includedir)
/uthash-1.9.2
...
...
darshan-util/configure
View file @
6806ff0c
#! /bin/sh
# Guess values for system-dependent variables and create Makefiles.
# Generated by GNU Autoconf 2.69 for darshan-util 3.1.
0
.
# Generated by GNU Autoconf 2.69 for darshan-util 3.1.
2
.
#
#
# Copyright (C) 1992-1996, 1998-2012 Free Software Foundation, Inc.
...
...
@@ -577,8 +577,8 @@ MAKEFLAGS=
# Identity of this package.
PACKAGE_NAME
=
'darshan-util'
PACKAGE_TARNAME
=
'darshan-util'
PACKAGE_VERSION
=
'3.1.
0
'
PACKAGE_STRING
=
'darshan-util 3.1.
0
'
PACKAGE_VERSION
=
'3.1.
2
'
PACKAGE_STRING
=
'darshan-util 3.1.
2
'
PACKAGE_BUGREPORT
=
''
PACKAGE_URL
=
''
...
...
@@ -1247,7 +1247,7 @@ if test "$ac_init_help" = "long"; then
# Omit some internal or obsolete options to make the list less imposing.
# This message is too long to be a string in the A/UX 3.1 sh.
cat
<<
_ACEOF
\`
configure' configures darshan-util 3.1.
0
to adapt to many kinds of systems.
\`
configure' configures darshan-util 3.1.
2
to adapt to many kinds of systems.
Usage:
$0
[OPTION]... [VAR=VALUE]...
...
...
@@ -1309,7 +1309,7 @@ fi
if
test
-n
"
$ac_init_help
"
;
then
case
$ac_init_help
in
short
|
recursive
)
echo
"Configuration of darshan-util 3.1.
0
:"
;;
short
|
recursive
)
echo
"Configuration of darshan-util 3.1.
2
:"
;;
esac
cat
<<
\
_ACEOF
...
...
@@ -1405,7 +1405,7 @@ fi
test
-n
"
$ac_init_help
"
&&
exit
$ac_status
if
$ac_init_version
;
then
cat
<<
\
_ACEOF
darshan-util configure 3.1.
0
darshan-util configure 3.1.
2
generated by GNU Autoconf 2.69
Copyright (C) 2012 Free Software Foundation, Inc.
...
...
@@ -1770,7 +1770,7 @@ cat >config.log <<_ACEOF
This file contains any messages produced by compilers while
running configure, to aid debugging if configure makes a mistake.
It was created by darshan-util
$as_me
3.1.
0
, which was
It was created by darshan-util
$as_me
3.1.
2
, which was
generated by GNU Autoconf 2.69. Invocation command line was
$ $0
$@
...
...
@@ -4117,7 +4117,7 @@ fi
done
DARSHAN_UTIL_VERSION
=
"3.1.
0
"
DARSHAN_UTIL_VERSION
=
"3.1.
2
"
...
...
@@ -4633,7 +4633,7 @@ cat >>$CONFIG_STATUS <<\_ACEOF || ac_write_fail=1
# report actual input values of CONFIG_FILES etc. instead of their
# values after options handling.
ac_log="
This file was extended by darshan-util
$as_me
3.1.
0
, which was
This file was extended by darshan-util
$as_me
3.1.
2
, which was
generated by GNU Autoconf 2.69. Invocation command line was
CONFIG_FILES =
$CONFIG_FILES
...
...
@@ -4695,7 +4695,7 @@ _ACEOF
cat
>>
$CONFIG_STATUS
<<
_ACEOF
|| ac_write_fail=1
ac_cs_config="`
$as_echo
"
$ac_configure_args
" | sed 's/^ //; s/[
\\
""
\`\$
]/
\\\\
&/g'`"
ac_cs_version="
\\
darshan-util config.status 3.1.
0
darshan-util config.status 3.1.
2
configured by
$0
, generated by GNU Autoconf 2.69,
with options
\\
"
\$
ac_cs_config
\\
"
...
...
darshan-util/configure.in
View file @
6806ff0c
...
...
@@ -5,7 +5,7 @@ dnl Process this file with autoconf to produce a configure script.
dnl You may need to use autoheader as well if changing any DEFINEs
dnl sanity checks, output header, location of scripts used here
AC_INIT([darshan-util], [3.1.
0
])
AC_INIT([darshan-util], [3.1.
2
])
AC_CONFIG_SRCDIR([darshan-logutils.h])
AC_CONFIG_AUX_DIR(../maint/config)
AC_CONFIG_HEADER(darshan-util-config.h)
...
...
darshan-util/darshan-convert.c
View file @
6806ff0c
...
...
@@ -360,6 +360,7 @@ int main(int argc, char **argv)
{
darshan_log_close
(
infile
);
darshan_log_close
(
outfile
);
unlink
(
outfile_name
);
return
(
-
1
);
}
...
...
@@ -368,6 +369,7 @@ int main(int argc, char **argv)
{
darshan_log_close
(
infile
);
darshan_log_close
(
outfile
);
unlink
(
outfile_name
);
return
(
-
1
);
}
...
...
@@ -389,19 +391,8 @@ int main(int argc, char **argv)
/* we have module data to convert */
memset
(
mod_buf
,
0
,
DEF_MOD_BUF_SIZE
);
ret
=
mod_logutils
[
i
]
->
log_get_record
(
infile
,
(
void
**
)
&
mod_buf
);
if
(
ret
!=
1
)
{
fprintf
(
stderr
,
"Error: failed to parse the first %s module record.
\n
"
,
darshan_module_names
[
i
]);
darshan_log_close
(
infile
);
darshan_log_close
(
outfile
);
unlink
(
outfile_name
);
return
(
-
1
);
}
/* loop over each of the module's records and convert */
do
while
((
ret
=
mod_logutils
[
i
]
->
log_get_record
(
infile
,
(
void
**
)
&
mod_buf
))
==
1
)
{
base_rec
=
(
struct
darshan_base_record
*
)
mod_buf
;
...
...
@@ -412,12 +403,22 @@ int main(int argc, char **argv)
{
darshan_log_close
(
infile
);
darshan_log_close
(
outfile
);
unlink
(
outfile_name
);
return
(
-
1
);
}
memset
(
mod_buf
,
0
,
DEF_MOD_BUF_SIZE
);
}
}
while
((
ret
=
mod_logutils
[
i
]
->
log_get_record
(
infile
,
(
void
**
)
&
mod_buf
))
==
1
);
}
if
(
ret
<
0
)
{
fprintf
(
stderr
,
"Error: failed to parse %s module record.
\n
"
,
darshan_module_names
[
i
]);
darshan_log_close
(
infile
);
darshan_log_close
(
outfile
);
unlink
(
outfile_name
);
return
(
-
1
);
}
}
darshan_log_close
(
infile
);
...
...
darshan-util/darshan-diff.c
View file @
6806ff0c
...
...
@@ -384,13 +384,11 @@ static int darshan_build_global_record_hash(
{
fprintf
(
stderr
,
"Error: unable to read module %s data from log file.
\n
"
,
darshan_module_names
[
i
]);
free
(
mod_rec
->
mod_dat
);
free
(
mod_rec
);
return
(
-
1
);
}
else
if
(
ret
==
0
)
{
free
(
mod_rec
->
mod_dat
);
free
(
mod_rec
);
break
;
}
...
...
darshan-util/darshan-logutils.c
View file @
6806ff0c
...
...
@@ -643,6 +643,17 @@ int darshan_log_get_mod(darshan_fd fd, darshan_module_id mod_id,
if
(
fd
->
mod_map
[
mod_id
].
len
==
0
)
return
(
0
);
/* no data corresponding to this mod_id */
/* assume module will support backwards compatibility, but we obviously
* can't provide any sort of "forwards" compatibility
*/
if
(
fd
->
mod_ver
[
mod_id
]
>
darshan_module_versions
[
mod_id
])
{
fprintf
(
stderr
,
"Error: invalid %s module log format version "
"(expected %d, got %d)
\n
"
,
darshan_module_names
[
mod_id
],
darshan_module_versions
[
mod_id
],
fd
->
mod_ver
[
mod_id
]);
return
(
-
1
);
}
/* read this module's data from the log file */
ret
=
darshan_log_dzread
(
fd
,
mod_id
,
mod_buf
,
mod_buf_sz
);
if
(
ret
<
0
)
...
...
darshan-util/darshan-mpiio-logutils.c
View file @
6806ff0c
...
...
@@ -64,7 +64,7 @@ static int darshan_log_get_mpiio_file(darshan_fd fd, void** mpiio_buf_p)
if
(
!
file
)
return
(
-
1
);
}
ret
=
darshan_log_get_mod
(
fd
,
DARSHAN_MPIIO_MOD
,
file
,
sizeof
(
struct
darshan_mpiio_file
));
...
...
@@ -165,6 +165,12 @@ static void darshan_log_print_mpiio_description(int ver)
printf
(
"# MPIIO_F_*_RANK_TIME: fastest and slowest I/O time for a single rank (for shared files).
\n
"
);
printf
(
"# MPIIO_F_VARIANCE_RANK_*: variance of total I/O time and bytes moved for all ranks (for shared files).
\n
"
);
if
(
ver
<
2
)
{
printf
(
"
\n
# WARNING: MPIIO module log format version 1 has the following limitations:
\n
"
);
printf
(
"# - MPIIO_F_WRITE_START_TIMESTAMP may not be accurate.
\n
"
);
}
return
;
}
...
...
darshan-util/doc/darshan-util.txt
View file @
6806ff0c
...
...
@@ -39,7 +39,6 @@ make
make install
----
[NOTE]
The darshan-util package is intended to be used on a login node or
workstation. For most use cases this means that you should
either leave CC to its default setting or specify a local compiler. This is
...
...
@@ -97,6 +96,12 @@ You can also manually specify the name of the output file using the
An example of the output produced by darshan-job-summary.pl can be found
link:http://www.mcs.anl.gov/research/projects/darshan/docs/ssnyder_ior-hdf5_id3655016_9-23-29011-12333993518351519212_1.darshan.pdf[HERE].
*NOTE*: The darshan-job-summary tool depends on a few LaTeX packages that may not
be availalble by default on all systems, including: lastpage, subfigure, and
threeparttable. These packages can be found and installed using your system's package
manager. For instance, the packages can be installed on Debian or Ubuntu systems as
follows: `apt-get install texlive-latex-extra`
=== darshan-summary-per-file.sh
This utility is similar to darshan-job-summary.pl, except that it produces a
...
...
Write
Preview
Supports
Markdown
0%
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment