README 1.8 KB
Newer Older
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19
Test Cases to evaluate darshan-parser --perf performance calculations.

Logs
----
350665
  * IOR single shared file with multiple iterations
  * 350665.output
  * harms_IOR_id350665_2-8-78651_12.darshan.gz

350690
  * IOR file-per-process with multiple iterations
  * 350690
  * harms_IOR_id350690_2-8-83458_1.darshan.gz

350892
  * IOR with both single file and file-per-process modes
  * 350892.output
  * harms_IOR_id1350892_2-9-74258_1.darshan.gz

20 21 22 23 24
354969
  * partshared run that creates shared files that are used by a subset of ranks
  * 354969.output
  * harms_partshared_id354969_2-25-56979_1.darshan.gz

25 26
Check Performance
------------------
27
Add all the performance test runs (both read and write) and the compute the average for IOR test cases.
28 29 30

awk 'BEGIN {sum=0;count=0;} /^(write|read)/ { sum+=$2; count+=1; } END { print "avg: " sum/count; }' xxxx.output

31
darshan-parser --perf <log.gz>
32 33 34

Compare the four methods to the calculated average.

35 36 37 38 39
For partshared:
grep "rate" 354969.output

Then compare with darshan-parser --perf <log.gz>

40 41 42 43 44 45 46 47 48 49 50
IOR Commands
------------
# File Per Proc with Multiple Files
qsub -q prod -A Operations -t 60 -n 512 --mode vn --env BG_COREDUMPDISABLED=1 IOR/src/C/IOR -a POSIX -F -b 256M -g -m -i 4 -K -o <path>/ior.out -r -w -t 4m -v

# Single Shared File with Multiple Files
qsub -q prod -A Operations -t 60 -n 512 --mode vn --env BG_COREDUMPDISABLED=1 IOR/src/C/IOR -a POSIX -b 256M -g -m -i 4 -K -o <path>/ior.out -r -w -t 4m -v

# Both I/O methods in one run
qsub -q prod -A Operations -t 60 -n 512 --mode vn --env BG_COREDUMPDISABLED=1 IOR/src/C/IOR -f single_and_multi_file.ior

51 52 53 54 55 56 57 58
Partial Shared Files
--------------------
# a subset of ranks access the same file
qsub -q prod -A Operations -t 60 -n 512 --mode vn --env BG_COREDUMPDISABLED=1 ./partshared

partshared
----------
mpicc -o partshared partshared.c