[molpro-user] Benchmark timing on linux raid
Jacek Klos
jklos at umd.edu
Wed Nov 24 23:24:29 GMT 2010
Thanks Gerald,
I ran 30 jobs with 12 CPUs after fresh reboot on linux Dell machine
and the first job n12_1 is the fastest, subsequent jobs
are more or less 160 seconds longer. Looks indeed that bigger jobs
fills some disk cache, smaller jobs like small_normal_ccsd
are more or less fast until some bigger job was running that used
disk more extensively , after that these small benchmarks
suffer slightly longer execution time too.
mpp_verybig_normal_mrci.n12_10.out: TOTAL 0.0000000
1438.67 260.21 1698.88 1794.16 80.19 94.69
mpp_verybig_normal_mrci.n12_11.out: TOTAL 0.0000000
1425.08 265.75 1690.83 1793.75 79.45 94.26
mpp_verybig_normal_mrci.n12_12.out: TOTAL 0.0000000
1426.57 266.03 1692.60 1781.57 80.07 95.01
mpp_verybig_normal_mrci.n12_13.out: TOTAL 0.0000000
1426.60 267.82 1694.42 1802.77 79.13 93.99
mpp_verybig_normal_mrci.n12_14.out: TOTAL 0.0000000
1435.97 266.42 1702.39 1807.21 79.46 94.20
mpp_verybig_normal_mrci.n12_15.out: TOTAL 0.0000000
1434.03 259.73 1693.76 1801.96 79.58 94.00
mpp_verybig_normal_mrci.n12_16.out: TOTAL 0.0000000
1416.84 278.66 1695.50 1794.82 78.94 94.47
mpp_verybig_normal_mrci.n12_17.out: TOTAL 0.0000000
1435.78 281.07 1716.85 1807.26 79.45 95.00
mpp_verybig_normal_mrci.n12_18.out: TOTAL 0.0000000
1426.61 257.95 1684.56 1787.10 79.83 94.26
mpp_verybig_normal_mrci.n12_19.out: TOTAL 0.0000000
1439.22 275.68 1714.90 1803.59 79.80 95.08
FIRST->mpp_verybig_normal_mrci.n12_1.out: TOTAL 0.0000000
1398.61 160.33 1558.94 1639.61 85.30 95.08
mpp_verybig_normal_mrci.n12_20.out: TOTAL 0.0000000
1435.07 262.59 1697.66 1800.39 79.71 94.29
mpp_verybig_normal_mrci.n12_21.out: TOTAL 0.0000000
1441.47 267.42 1708.89 1800.25 80.07 94.93
mpp_verybig_normal_mrci.n12_22.out: TOTAL 0.0000000
1440.82 263.13 1703.95 1802.00 79.96 94.56
mpp_verybig_normal_mrci.n12_23.out: TOTAL 0.0000000
1442.43 270.11 1712.54 1803.42 79.98 94.96
mpp_verybig_normal_mrci.n12_24.out: TOTAL 0.0000000
1427.74 280.92 1708.66 1813.78 78.72 94.20
mpp_verybig_normal_mrci.n12_25.out: TOTAL 0.0000000
1435.17 277.26 1712.43 1818.69 78.91 94.16
mpp_verybig_normal_mrci.n12_26.out: TOTAL 0.0000000
1431.74 271.63 1703.37 1801.21 79.49 94.57
mpp_verybig_normal_mrci.n12_27.out: TOTAL 0.0000000
1425.27 284.95 1710.22 1803.83 79.01 94.81
mpp_verybig_normal_mrci.n12_28.out: TOTAL 0.0000000
1425.87 285.76 1711.63 1819.90 78.35 94.05
mpp_verybig_normal_mrci.n12_29.out: TOTAL 0.0000000
1440.25 272.53 1712.78 1810.78 79.54 94.59
mpp_verybig_normal_mrci.n12_2.out: TOTAL 0.0000000
1433.19 271.42 1704.61 1845.38 77.66 92.37
mpp_verybig_normal_mrci.n12_30.out: TOTAL 0.0000000
1427.30 275.63 1702.93 1819.76 78.43 93.58
mpp_verybig_normal_mrci.n12_3.out: TOTAL 0.0000000
1432.39 269.17 1701.56 1847.98 77.51 92.08
mpp_verybig_normal_mrci.n12_4.out: TOTAL 0.0000000
1440.61 270.70 1711.31 1853.01 77.74 92.35
mpp_verybig_normal_mrci.n12_5.out: TOTAL 0.0000000
1434.48 271.24 1705.72 1844.74 77.76 92.46
mpp_verybig_normal_mrci.n12_6.out: TOTAL 0.0000000
1424.29 273.93 1698.22 1829.11 77.87 92.84
mpp_verybig_normal_mrci.n12_7.out: TOTAL 0.0000000
1429.79 270.34 1700.13 1811.49 78.93 93.85
mpp_verybig_normal_mrci.n12_8.out: TOTAL 0.0000000
1433.15 262.41 1695.56 1804.51 79.42 93.96
mpp_verybig_normal_mrci.n12_9.out: TOTAL 0.0000000
1429.19 271.79 1700.98 1819.42 78.55 93.49
Best
Jacek Klos
On Nov 24, 2010, at 3:16 AM, Gerald Knizia wrote:
> On Wednesday 24 November 2010 00:49, Jacek Klos wrote:
>> I have noticed some strange behavior on our linux Dell machine.
>> Machine is dual-6core 3.33GHz 24GB RAM, and RAID disk composed
>> of 3 SAS 15k RPM drives making logical volume of about 1TB.
>>
>> When I run small_normal_ccsd benchmarks with 12 cpus using parallel
>> molpro (GA4.2 Open-MPI) and I repeat it let's say 20 times after
>> the fresh
>> reboot of the machine the fastest elapsed time is 76 seconds.
>> The range of elapsed times is more or less +/- 5 sek.
>>
>> But when I run again the same series of 12cpus jobs after machine is
>> on for a day or so I get elapsed timings like 30 seconds longer:
>
> It may be possible that this is related to the 'system cache', i.e.,
> the
> amount of free physical RAM not committed to any other running
> processes.
> Operating systems tend to use all of that memory for disk caching,
> and since
> these CCSD jobs are still rather small and you have lots of memory,
> maybe
> after a fresh reboot the OS never actually writes the integral data
> to disk
> in the first place (or never reads it back). AOINT, AOSORT, HF and
> TRANSFORM
> all depend heavily on the IO performance usually, so if for some
> reason there
> is lots of spare RAM for disk caching, these would become much faster.
>
> Why this stops being the case after the system has run for a longer
> time is
> another question. Maybe in the meantime more files have been opened
> and the
> OS determined these as more cache-worthy than the integral files.
>
> There may be tools in linux to tell you about the system cache
> usage, but I'm
> not an expert and could not tell which. One thing you could try is
> to test of
> this difference of execution speed also occurs if (a) you run
> smaller jobs,
> which always fit completely into memory or (b) you run larger jobs
> (with
> integral files >> 24 GB), which never fit completely into memory.
> --
> Gerald Knizia
More information about the Molpro-user
mailing list