ChangeLog 14.6 KB
Newer Older
Philip Carns's avatar
Philip Carns committed
1
2
3
--------------------------
Darshan Release Change Log
--------------------------
4

Philip Carns's avatar
Philip Carns committed
5
6
darshan-2.2.9-pre1
=============
Philip Carns's avatar
Philip Carns committed
7
8
9
10
11
12
* Bug fixes:
  - Fix mnt table overflow if a large number of file systems are mounted, 
    reported by David Shrader.
  - Fix argument parsing for darshan-convert, reported by Mouhamed Gueye.
  - Fix metadata annotation overflow in darshan-convert, reported by 
    Mouhamed Gueye.
13
14
  - Fix const-correctness in dynamic library when built against MPI 3.x
    libraries.
15
16
  - Fix "undefined symbol: dlsym" error when using preloaded dynamic library
    on some platforms, reported by Florin Isaila.
Philip Carns's avatar
Philip Carns committed
17

Philip Carns's avatar
Philip Carns committed
18
darshan-2.2.8
19
=============
20
21
22
23
* WARNING: please note that the Darshan module file for Cray environments has
  been updated, especially in the DARSHAN_POST_LINK_OPTS variable.  Please
  update your module file accordingly when upgrading from 2.2.7 or earlier 
  on Cray platforms.
Philip Carns's avatar
Philip Carns committed
24
25
* Improved ability to analyze I/O activity related to particular files 
  opened by an application
Philip Carns's avatar
Philip Carns committed
26
  - darshan-summary-per-file.sh script to generate a separate pdf summary for
Philip Carns's avatar
Philip Carns committed
27
    each file opened by an application. Developed by Rob Latham.
Philip Carns's avatar
Philip Carns committed
28
  - Added --file-list and --file-list-detailed options to darshan-parser to 
Philip Carns's avatar
Philip Carns committed
29
    list files opened by an application along with brief statistics
Philip Carns's avatar
Philip Carns committed
30
31
  - Added --file option to darshan-convert to filter out activity for a
    specific file from a Darshan log
32
33
34
* Add wrappers for POSIX AIO operations (fixes tracking of underlying POSIX
  operations resulting from nonblocking MPI-IO operations.  Bug reported by
  David Shrader.)
35
36
37
38
39
40
* Fix compile-time errors when Darshan is configured for use with MPICH 3.x 
  installations with const correct MPI function prototypes. Also affects 
  MPICH 1.5+ installations with optional const support.  Reported by Yushu Yao.
* Fix segmentation fault when using LD_PRELOAD instrumentation on programs
  that use MPI_Init_thread() rather than MPI_Init().  Reported by Myriam
  Botalla.
41

42
darshan-2.2.7
Philip Carns's avatar
Philip Carns committed
43
44
45
=============
* Updated Cray installation documentation for cleaner integration in Cray
  environments
Philip Carns's avatar
Philip Carns committed
46
47
48
49
* Fix bug that recorded incorrect device ID (and therefore incorrect mount
  point mapping) if stat() was called before open() on a file
* Store version number of the darshan runtime library in the log file 
  metadata (see lib_ver in darshan-parser output)
Philip Carns's avatar
Philip Carns committed
50

Philip Carns's avatar
Philip Carns committed
51
darshan-2.2.6
52
=============
53
* Bug fixes:
54
  - make sure to honor user-specified hints passed in at runtime
55
56
    via the DARSHAN_LOGHINTS env variable.
  - include fread and fwrite in darshan-job-summary.pl read and write counts
57
  - fix segmentation fault on invalid arguments to darshan-parser
Philip Carns's avatar
Philip Carns committed
58
59
60
61
62
63
64
65
66
67
68
69
70
* Performance enhancements:
  - collect mount point information at rank 0 and broadcast to all 
    processes to avoid excess file system traffic on startup
  - change default MPI-IO hints for writing log file to
    romio_no_indep_rw=true and cb_nodes=4 to improve log creation performance
* Install libdarshan-util and headers during installation process for
  darshan-util; patch contributed by Matthieu Dorier
* Detect PMPI support at link time when using compiler scripts produced
  by the darshan-gen-* utilities.  This avoids link problems when Darshan
  compiler wrappers are used with the ADIOS dummy MPI library; reported
  by Jingqing Mu
* Support MPICH_{CC/CXX/F77} environment variables in compiler scripts
  produced by the darshan-gen-* utilities
Philip Carns's avatar
Philip Carns committed
71
72
* Add instrumentation of startup time, enabled via DARSHAN_INTERNAL_TIMING
  environment variable
Philip Carns's avatar
Philip Carns committed
73
74
* Rename cp-shutdown-bench test utility to darshan-shutdown-bench and enable
  benchmarking hooks in library by default so that darshan-shutdown-bench 
75
  can be used with any Darshan installation
76
* Remove deprecated --enable-st-dev-workaround configure option
77
* Misc. documentation updates
78

79
darshan-2.2.5
80
81
82
83
=============
* Fix bug in mount point identification when --enable-stat-at-open option is
  not used.  In Darshan 2.2.4, some file entries were recorded as using the "/"
  file system regardless of their location.
84
85
* Update patches and documentation for Cray xt-asyncpe environment 5.12 or 
  higher; contributed by Yushu Yao.
86
87
* Update documentation to no longer recommend --enable-stat-at-open option on 
  any systems
88
89
* Fix unresolved symbols error when linking non-MPI programs with Cray
  compiler.  Reported by Yushu Yao.
90
91
92
93
94
95
96
97
* Use fstat() in place of stat() for detecting file information when
  --enable-stat-at-open option is used
* Use statfs() at startup to detect default block size for supported file
  systems
* Added --enable-group-readable-logs configure option, which will cause
  Darshan to to generate log files with the group read permission bit set.
  This option is useful in conjunction with deployments that set the setgid
  bit on log directories.
98

Philip Carns's avatar
Philip Carns committed
99
darshan-2.2.4
Philip Carns's avatar
Philip Carns committed
100
=============
Philip Carns's avatar
Philip Carns committed
101
102
* Disable extra stat() of newly opened files by default.  This improves 
  performance on shared files for some platforms.  Reported by Yushu Yao.
Philip Carns's avatar
Philip Carns committed
103
104
105
  - prevents Darshan from reporting SIZE_AT_OPEN, FILE_ALIGNENT, and 
    DARSHAN_FILE_NOT_ALIGNED values
  - features can still be enabled via --enable-stat-at-open option
106
107
* Fix missing -lz in post ld flags reported by Yushu Yao.  Fixes a link-time
  error for some corner-case applications.
108
109
* Fix bug in Cray compiler script patches that was setting compiler flags 
  incorrectly.  Reported by Yushu Yao.
110
111
  WARNING: if you are using a previous Darshan release (2.2.3 through
  2.2.4-pre4) on a Cray platform, please re-patch your compiler scripts.
112
113
114
115
* Update darshan-gen-* scripts to support the potential for additonal LDFLAGS
  or CFLAGS that appear before the object or source file in the compile or
  link commands.  This fixes compatibility with some mvapich2 installations,
  reported by Dragos Constantin.
Philip Carns's avatar
Philip Carns committed
116

Philip Carns's avatar
Philip Carns committed
117
darshan-2.2.3
Philip Carns's avatar
Philip Carns committed
118
119
120
121
=============
* improved Cray XE6 support
  - support for GNU, PGI, Cray, Pathscale, and Intel compilers
  - patch adding Darshan capability to system compiler scripts
Philip Carns's avatar
Philip Carns committed
122
123
124
  - software module, including testing and features contributed 
    by Yushu Yao and Katie Antypas of The National Energy Research 
    Scientific Computing Center (NERSC)
Philip Carns's avatar
Philip Carns committed
125
  - improved documentation
Philip Carns's avatar
Philip Carns committed
126
127
128
129
* minor bug fixes:
  - properly detect cxx library name when generating BG/Q compiler wrappers
  - improve timing mechanism for cp-shutdown-bench benchmark
  - improve hashing to avoid log file name collisions
Philip Carns's avatar
Philip Carns committed
130

131
132
133
134
135
136
137
138
darshan-2.2.2
=============
* bug fixes:
  - remove debugging message that was inadvertently included in
    MPI_File_sync() wrapper
  - fix potential hang if the --with-log-path-by-env argument was used at 
    configure time but the environment variable was not set at run time

Philip Carns's avatar
Philip Carns committed
139
darshan-2.2.1
Philip Carns's avatar
Philip Carns committed
140
141
142
=============
* significant improvements to how counters are handled in multi-threaded
  applications
Philip Carns's avatar
Philip Carns committed
143
144
* initial (rough) documentation for using Darshan in Cray
  programming environments with static linking
Philip Carns's avatar
Philip Carns committed
145
* bug fixes:
Philip Carns's avatar
Philip Carns committed
146
147
  - escape special characters in mount point paths in darshan-job-summary.pl
    (reported by Mouhamed Gueye)
Philip Carns's avatar
Philip Carns committed
148
149
150
151
152
  - workarounds for various runtime problems with cuserid() and stat() in  
    Cray environments
  - build problems with darshan-utils on some versions of OSX
  - accurate shared file statistics for libraries that use deferred opens

153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
darshan-2.2.0
=============
* split darshan into separate packages:
  - darshan-runtime: for runtime instrumentation
  - darshan-utils: for processing darshan log files
* changed default output file name for darshan-job-summary.pl to be based on
  input file name rather than summary.pdf
* reorganized init and finalize routines so that can be linked separately (to
  allow for easier integration with other instrumentation tools)
* add -cc, -cxx, -f77, -f90, and -fc support to compiler scripts generated by
  the darshan-gen-*.pl scripts
* bug fixes:
  - potential MAX_BYTE overflow on 32 bit systems
  - incorrect pread and pwrite offset tracking
  - corrections to darshan-job-summary variance table
  - better runtime error handling if bzip or gnuplot tools are insufficient
  - improvements to time range in darshan-job-summary graphs
170
171
172
173
* documentation:
  - improved documentation for both the darshan-runtime and darshan-util
  portions of Darshan can be found in the respective doc/ subdirectory for
  each
174

175
176
177
178
179
darshan-2.1.2
=============
* improved error handling when writing log files.  If a write fails then the
  log file will be deleted and a warning will be printed to stderr.

180
darshan-2.1.1
181
=============
Philip Carns's avatar
Philip Carns committed
182
183
184
185
186
187
188
189
190
* new darshan-convert command line utility for converting existing log files,
  with optional anonymization and optional bzip2 compression
* bzip2 support in command line utilities (but not in the darshan library
  itself)
* updated log file format that allows for string key/value pairs to be stored
  in the header
* added ability to set MPI-IO hints when writing darshan log
  at configure time: --with-log-hints
  at run time: DARSHAN_LOGHINTS environment variable
191
192
* bug fix contributed by Sandra Schröder: use case-insensitive search for MPI
  symbols in Fortran wrapper script
Philip Carns's avatar
Philip Carns committed
193
194
* performance bug fix: remove unecessary call to MPI_File_set_size when
  writing log
195
196
197
* added --with-logpath-by-env configure option to allow absolute 
  log path to be specified via environment variable

198
199
200
201
202
203
204
205
206
207
208
darshan-2.1.0
=============
* additinoal environment variables to control log, jobid and
  alignment parameters
* additional configure tests to improve portability
* bug fixes for darshan-parser --perf calucations
* support for MPI1.x
* support for OpenMPI
* support for PGI, Intel compilers
* new libdarshan.so dynamic library for use with LD_PRELOAD

209
210
211
212
213
214
215
216
217
218
219
220
221
darshan-2.0.2
=============
* added a random identifier to job logs (to avoid collisions from multiple
  application instances within a single scheduler job)
* improved installation and library path management for darshan-job-summary.pl
* improved error handling in darshan-job-summary.pl
* additional derived statistics categories for darshan-parser output:
    --all   : all sub-options are enabled
    --base  : darshan log field data [default]
    --file  : total file counts
    --perf  : derived perf data
    --total : aggregated darshan field data

222
223
darshan-2.0.1
=============
Philip Carns's avatar
Philip Carns committed
224
225
226
* bug fix to variance/minimum calculations on shared files
* switch to automatic generation of all MPI compiler scripts using
  darshan-gen-* tools
227
228
229
230
* new run time environment variable: DARSHAN_INTERNAL_TIMING.  If set at job
  execution time, it will cause Darshan to time its own internal data 
  aggregation routines and print the results to stdout at rank 0.

231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
darshan-2.0.0
=============
* new output file format that is portable across architectures
  NOTE: Darshan 1.x output files are incompatible with the tools in this 
  release unless they were generated on a ppc32 architecture (Blue Gene)
* 8 new counters that record the rank of the fastest and slowest process that
  opened each shared file, along with the number of seconds and number of
  bytes consumed by those processes.  It also reports the variance in both
  time and amount of data.
* new --with-jobid-env configure argument to support recording job identifiers
  from different schedulers
* job ID is now recorded within the Darshan log in addition to in the file
  name
* darshan-job-summary.pl:
  * opens output files directly without using intermediate darshan-parser output
  * table showing data usage per file system
Philip Carns's avatar
Philip Carns committed
247
  * table showing I/O variance in shared files
248
249
250
* Fixes for bugs reported by Noah Watkins: 
  * avoid name collision in hashing function
  * divide by zero error in darshan-job-summary.pl
251

252
253
254
255
darshan-1.1.14
==============
* fixed erroneous incompatibility warning when opening old logs in darshan-parser

256
257
258
259
darshan-1.1.13
==============
* updated compiler scripts to fix link order in some cases with -lhdf5

260
261
262
263
264
265
266
darshan-1.1.12
==============
* improved mapping of file records to mount points
* new page in darshan-job-summary.pl output showing timelines of file access
* bug fix to "fast" mpif90 script
* checkpoint/restart ability in parallel fsstats script

Philip Carns's avatar
Philip Carns committed
267
268
269
270
271
darshan-1.1.11
==============
* Bug fix for lseek, pread, and pwrite when used in 32 applications without large file support
* Improved experimental darshan-gen-cc.pl script
* Added new experimental darshan-gen-cxx.pl script
Philip Carns's avatar
Philip Carns committed
272

273
274
275
276
277
darshan-1.1.10
==============
* Bug fix for files that are accessed with stat() but never opened
* Workaround zlib problems with 64 bit offsets on 32 bit architectures

Philip Carns's avatar
Philip Carns committed
278
279
280
281
282
283
284
darshan-1.1.9
=============
* Added "fast" version of each BG/P compiler
* Added experimental scripts in test directory to run fsstats in parallel
* Kevin Harms: Added experimental utilities for loading darshan results into SQL
* Use exclusive flag when opening output file (to protect against file name collision)
* Rob Ross: updates to allow command line tools to build on Darwin
285
* Bug fix for pnetcdf configure problem reported by Rob Latham; darshan now always pulls in MPI_Wtime() symbol at link time
Philip Carns's avatar
Philip Carns committed
286

287
288
289
darshan-1.1.8
=============
* Added tracking of file system type and mount point for each file
Philip Carns's avatar
Philip Carns committed
290
291
292
* Added tracking of file size at open time (CP_SIZE_AT_OPEN)
* Moved sync cost to be counted in cumulative write time rather than cumulative metadata time
* Added sync as a separate category in darshan-job-summar.pl counters
293
294
295
296
297
298
* Bug fix to most frequent access size table in darshan-job-summary.pl
* Converted all utilities to use darshan-logutils api for reading output files
* Added backwards compatibility to darshan-logutils routines
* Kevin Harms: Added darshan-analyzer utility to summarize usage of MPI-IO, pNetCDF, HDF5, and shared files across a set of output files
* Fixed bug field listing for darshan-diff utility

299
300
301
302
darshan-1.1.7
=============
* Minor fix for a compile warning

303
304
305
306
darshan-1.1.6
=============
* Kevin Harms: bug fix for segfault in apps that use MPI_Init_thread()

307
308
309
310
311
darshan-1.1.5
=============
* Limit PMPI usage in library to fewer functions
* Update PMPI detection in compiler scripts to ignore functions unused by Darshan

Philip Carns's avatar
Philip Carns committed
312
313
314
315
316
317
318
319
320
321
darshan-1.1.4
=============
* Track files opened via Parallel NetCDF
* Track files opened via HDF5
* Record slowest individual POSIX read and write times along with access size for those operations
* Inspect symbols at compile time to determine whether to enable Darshan or not based on the presence of MPI and PMPI symbols
* Use GNU and IBM compilers from path rather than hard coded location
* Simplify warning message if unable to open log file
* Remove unused internal benchmark routines

Philip Carns's avatar
Philip Carns committed
322
323
324
325
darshan-1.1.3
=============
* Updated compiler scripts for V1R4 driver on BlueGene/P

326
327
328
329
darshan-1.1.2
=============
* Added *_r versions of each IBM compiler script on BlueGene/P

Philip Carns's avatar
Philip Carns committed
330
331
332
333
334
335
336
337
338
darshan-1.1.1
=============
* Set default permissions to 0400 (user read only) for output files
* Automatically disable Darshan at link time if common PMPI libraries are detected in the command line
* Experimental tool (darshan-gen-cc.pl) to automatically generate Darshan-enabled mpicc scripts

darshan-1.1.0
=============
* Initial public release