Set of test cases for checking performance estimation of darshan-parser

--perf.


git-svn-id: https://svn.mcs.anl.gov/repos/darshan/trunk@385 3b7491f3-a168-0410-bf4b-c445ed680a29

test/perf/350665.output

+IOR-2.10.0.1: MPI Coordinated Test of Parallel I/O
+
+Run began: Tue Feb  8 21:50:51 2011
+Command line used: /home/harms/bgp/IOR/src/C/IOR -a POSIX -F -b 256M -g -m -i 4 -K -o /intrepid-fs1/users/harms/scratch/ior.out -r -w -t 4m -v
+Machine: CNK ion-R42-1
+Start time skew across all tasks: 0.20 sec
+Path: /intrepid-fs1/users/harms/scratch
+FS: 2194.4 TiB   Used FS: 22.8%   Inodes: 4096.0 Mi   Used Inodes: 33.7%
+Participating tasks: 2048
+
+Summary:
+	api                = POSIX
+	test filename      = /intrepid-fs1/users/harms/scratch/ior.out
+	access             = file-per-process
+	pattern            = segmented (1 segment)
+	ordering           = sequential offsets
+	clients            = 2048 (1 per node)
+	repetitions        = 4
+	xfersize           = 4 MiB
+	blocksize          = 256 MiB
+	aggregate filesize = 512 GiB
+
+Commencing write performance test.
+Tue Feb  8 21:51:00 2011
+
+access    bw(MiB/s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   iter
+------    ---------  ---------- ---------  --------   --------   --------   ----
+write     1853.28    262144     4096       6.46       276.33     0.104290   0   
+Commencing read performance test.
+Tue Feb  8 21:55:54 2011
+
+read      1568.01    262144     4096       6.39       327.85     0.128561   0   
+Commencing write performance test.
+Tue Feb  8 22:02:12 2011
+
+write     1860.19    262144     4096       6.68       275.05     0.108472   1   
+Commencing read performance test.
+Tue Feb  8 22:07:05 2011
+
+read      1497.59    262144     4096       6.39       343.58     0.120167   1   
+Commencing write performance test.
+Tue Feb  8 22:13:36 2011
+
+write     1905.81    262144     4096       4.29       270.71     0.103336   2   
+Commencing read performance test.
+Tue Feb  8 22:18:25 2011
+
+read      1558.55    262144     4096       6.36       329.91     0.126614   2   
+Commencing write performance test.
+Tue Feb  8 22:24:44 2011
+
+write     1952.53    262144     4096       5.20       263.21     0.101522   3   
+Commencing read performance test.
+Tue Feb  8 22:29:25 2011
+
+read      1566.06    262144     4096       6.42       328.24     0.120201   3   
+
+Max Write: 1952.53 MiB/sec (2047.38 MB/sec)
+Max Read:  1568.01 MiB/sec (1644.17 MB/sec)
+
+Run finished: Tue Feb  8 22:35:34 2011

test/perf/350690.output

+stdout[0] IOR-2.10.0.1: MPI Coordinated Test of Parallel I/O
+
+stdout[0] Run began: Tue Feb  8 23:10:58 2011
+stdout[0] Command line used: /home/harms/bgp/IOR/src/C/IOR -a POSIX -b 256M -g -m -i 4 -K -o /intrepid-fs0/users/harms/scratch/ior.out -r -w -t 4m -v
+stdout[0] Machine: CNK ion-R44-9
+stdout[0] Start time skew across all tasks: 0.12 sec
+stdout[0] Path: /intrepid-fs0/users/harms/scratch
+stdout[0] FS: 3072.0 TiB   Used FS: 90.3%   Inodes: 512.0 Mi   Used Inodes: 76.2%
+stdout[0] Participating tasks: 2048
+stdout[0] 
+Summary:
+stdout[0] 	api                = POSIX
+stdout[0] 	test filename      = /intrepid-fs0/users/harms/scratch/ior.out
+stdout[0] 	access             = single-shared-file
+stdout[0] 	pattern            = segmented (1 segment)
+stdout[0] 	ordering           = sequential offsets
+stdout[0] 	clients            = 2048 (1 per node)
+stdout[0] 	repetitions        = 4
+stdout[0] 	xfersize           = 4 MiB
+stdout[0] 	blocksize          = 256 MiB
+stdout[0] 	aggregate filesize = 512 GiB
+stdout[0] 
+stdout[0] Commencing write performance test.
+stdout[0] Tue Feb  8 23:10:59 2011
+stdout[0] 
+stdout[0] access    bw(MiB/s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   iter
+stdout[0] ------    ---------  ---------- ---------  --------   --------   --------   ----
+stdout[0] write     2222.96    262144     4096       0.427417   235.34     0.079745   0   
+stdout[0] Commencing read performance test.
+stdout[0] Tue Feb  8 23:14:54 2011
+stdout[0] 
+stdout[0] read      2878.55    262144     4096       0.017130   182.00     0.123749   0   
+stdout[0] Commencing write performance test.
+stdout[0] Tue Feb  8 23:17:57 2011
+stdout[0] 
+stdout[0] write     2321.09    262144     4096       0.179439   225.64     0.063335   1   
+stdout[0] Commencing read performance test.
+stdout[0] Tue Feb  8 23:21:42 2011
+stdout[0] 
+stdout[0] read      2939.11    262144     4096       0.018604   178.19     0.170030   1   
+stdout[0] Commencing write performance test.
+stdout[0] Tue Feb  8 23:24:41 2011
+stdout[0] 
+stdout[0] write     2314.76    262144     4096       0.183715   226.12     0.194725   2   
+stdout[0] Commencing read performance test.
+stdout[0] Tue Feb  8 23:28:28 2011
+stdout[0] 
+stdout[0] read      2910.44    262144     4096       0.017381   179.94     0.181669   2   
+stdout[0] Commencing write performance test.
+stdout[0] Tue Feb  8 23:31:28 2011
+stdout[0] 
+stdout[0] write     2302.51    262144     4096       0.166151   227.30     0.233268   3   
+stdout[0] Commencing read performance test.
+stdout[0] Tue Feb  8 23:35:16 2011
+stdout[0] 
+stdout[0] read      2903.94    262144     4096       0.017946   180.34     0.189010   3   
+stdout[0] 
+stdout[0] Max Write: 2321.09 MiB/sec (2433.84 MB/sec)
+stdout[0] Max Read:  2939.11 MiB/sec (3081.88 MB/sec)
+stdout[0] 
+stdout[0] Run finished: Tue Feb  8 23:38:16 2011

test/perf/350892.output

+IOR-2.10.0.1: MPI Coordinated Test of Parallel I/O
+
+Run began: Wed Feb  9 20:37:39 2011
+Command line used: /home/harms/bgp/IOR/src/C/IOR -f single_and_multi_file.ior
+Machine: CNK ion-R06-9
+
+Summary:
+	api                = POSIX
+	test filename      = /intrepid-fs0/users/harms/scratch/ior.out
+	access             = single-shared-file
+	ordering           = sequential offsets
+	clients            = 2048 (1 per node)
+	repetitions        = 4
+	xfersize           = 4 MiB
+	blocksize          = 256 MiB
+	aggregate filesize = 512 GiB
+
+access    bw(MiB/s)  block(KiB) xfer(KiB)  open(s)    wr/rd(s)   close(s)   iter
+------    ---------  ---------- ---------  --------   --------   --------   ----
+write     2263.37    262144     4096       0.318107   231.21     0.107741   0   
+read      2819.49    262144     4096       0.016437   185.79     0.145695   0   
+write     2363.48    262144     4096       0.186718   221.42     0.216908   1   
+read      2861.37    262144     4096       0.017310   182.96     0.251082   1   
+write     2261.83    262144     4096       0.167811   231.40     0.234639   2   
+read      2867.13    262144     4096       0.016909   182.67     0.179075   2   
+write     2304.19    262144     4096       0.218270   227.20     0.118000   3   
+read      2908.42    262144     4096       0.018176   180.11     0.134019   3   
+
+Max Write: 2363.48 MiB/sec (2478.29 MB/sec)
+Max Read:  2908.42 MiB/sec (3049.70 MB/sec)
+
+IOR-2.10.0.1: MPI Coordinated Test of Parallel I/O
+
+Run began: Wed Feb  9 21:05:05 2011
+Command line used: /home/harms/bgp/IOR/src/C/IOR -f single_and_multi_file.ior
+Machine: CNK ion-R06-9
+
+Summary:
+	api                = POSIX
+	test filename      = /intrepid-fs0/users/harms/scratch/ior.out
+	access             = file-per-process
+	ordering           = sequential offsets
+	clients            = 2048 (1 per node)
+	repetitions        = 4
+	xfersize           = 4 MiB
+	blocksize          = 256 MiB
+	aggregate filesize = 512 GiB
+
+write     2341.54    262144     4096       0.622082   223.12     0.162660   0   
+read      2862.17    262144     4096       0.017030   183.13     0.030806   0   
+write     2354.37    262144     4096       0.409148   222.11     0.171205   1   
+read      2850.33    262144     4096       0.016691   183.89     0.031446   1   
+write     2356.01    262144     4096       0.425325   221.91     0.200420   2   
+read      2867.77    262144     4096       0.020235   182.77     0.032219   2   
+write     2341.94    262144     4096       0.982192   222.72     0.163251   3   
+read      2884.45    262144     4096       0.054666   181.67     0.035870   3   
+
+Max Write: 2356.01 MiB/sec (2470.45 MB/sec)
+Max Read:  2884.45 MiB/sec (3024.56 MB/sec)
+
+Run finished: Wed Feb  9 21:32:21 2011

test/perf/README

+Test Cases to evaluate darshan-parser --perf performance calculations.
+
+Logs
+----
+350665
+  * IOR single shared file with multiple iterations
+  * 350665.output
+  * harms_IOR_id350665_2-8-78651_12.darshan.gz
+
+350690
+  * IOR file-per-process with multiple iterations
+  * 350690
+  * harms_IOR_id350690_2-8-83458_1.darshan.gz
+
+350892
+  * IOR with both single file and file-per-process modes
+  * 350892.output
+  * harms_IOR_id1350892_2-9-74258_1.darshan.gz
+
+Check Performance
+------------------
+Add all the performance test runs (both read and write) and the compute the average. 
+
+awk 'BEGIN {sum=0;count=0;} /^(write|read)/ { sum+=$2; count+=1; } END { print "avg: " sum/count; }' xxxx.output
+
+darshan-parser --perf log.gz
+
+Compare the four methods to the calculated average.
+
+IOR Commands
+------------
+# File Per Proc with Multiple Files
+qsub -q prod -A Operations -t 60 -n 512 --mode vn --env BG_COREDUMPDISABLED=1 IOR/src/C/IOR -a POSIX -F -b 256M -g -m -i 4 -K -o <path>/ior.out -r -w -t 4m -v
+
+# Single Shared File with Multiple Files
+qsub -q prod -A Operations -t 60 -n 512 --mode vn --env BG_COREDUMPDISABLED=1 IOR/src/C/IOR -a POSIX -b 256M -g -m -i 4 -K -o <path>/ior.out -r -w -t 4m -v
+
+# Both I/O methods in one run
+qsub -q prod -A Operations -t 60 -n 512 --mode vn --env BG_COREDUMPDISABLED=1 IOR/src/C/IOR -f single_and_multi_file.ior
+

test/perf/single_and_multi_file.ior

+# IOR Script to use both single-shared-file and file-per-proc IO
+#
+#qsub -q prod -A Operations -t 60 -n 512 --mode vn --env BG_COREDUMPDISABLED=1 /
+home/harms/bgp/IOR/src/C/IOR -f file.ior
+#
+IOR START
+	# Base Config
+	testFile = /intrepid-fs0/users/harms/scratch/ior.out
+        api = POSIX
+	blockSize = 256M
+	transferSize = 4M
+	intraTestBarriers = 1
+	keepFileWithError = 1
+	multiFile = 1
+	repetitions = 4
+	writeFile = 1
+	readFile = 1
+
+	# Single Shared File 
+	filePerProc = 0
+	RUN
+
+	# File Per Process
+	filePerProc = 1
+	uniqueDir = 1
+	RUN
+IOR STOP