Worker Node for HEP

Benchmark and market analysis of worker node for HEP farms

Recommended

HS06

The new benchmark has been called HS06 ... is composed by seven benchmark, 3 Integer and 4 Floating Point Detailed Instruction are available in this page and in the CERN WIKI page Of course you need a licensed version of the SPEC CPU 2006 suite

SPEC CPU 2006

In August 2006 SPEC has released SPEC CPU2006, the new version of their CPU suite. This releases means that CINT, CFP, CINT Rates and CFP Rates of the CPU2000 suite will be replaced by their new 2006 equivalents.

Computer producers are invited to submit CPU2006 results from the date of release. After 3 months SPEC requires that all CPU2000 results submitted for publication will be accompanied by  CPU2006. Six months after publication SPEC did not accept CPU2000 result for publication.

CPU for Worker Nodes

This site has been set up to collect and organize in a single place, technical and practical information that it is felt can be useful for selecting the most appropriate CPUs for our HEP farms

How do physicists measure the performance of an HEP worker node? In the past several unit of measure have been adopted. I remember in 80's the MIPS, the VUPS (Vax Unit per second), the CERN UNIT. Then physicists started using the SPEC CPU suite. First called Spec from the suite Spec 89, then Spec int 92, Spec int 95, Spec int 2000 nicknamed SI2K.

All the benchmark where used to measure the performances of a single processor machine. The farm computing power was computed summing the computing power of each node, since HEP processing needs high Throughput in "batch" mode.

Since 2006 several participants to the HEPIX workshop noticed that the SPECINT 2000 published by the SPEC site was not scaling anymore with the real throughput of the Worker Nodes when they were running HEP Application Code

The main cause of the discrepancy was identified in the use of the gcc compiler but other factors were found to affect the performances: compilation switches, kernel version, usage of 32 or 64 bit

As temporary solution, WLCG recommended to use a benchmark the Specint 2000 measured with gcc and a defined set of compilation switches. In order to compare the old unit of measures (SPECINT 2000 published by SPEC) with the new SPECINT 2000 measured with gcc WLCG decided that the latter should be increased by 50%.

In the Hepix Fall meeting at JLAB a Working group was setup to study the problem and propose a new benchmark since in 2006 SPEC had declared CPU 2000 as obsolete and had presented the new CPU 2006 suite.

The working group made an accurate analysis of the CPU 2006 suite, with particular attention to the integer and floating point suites, to the difference between the speed (single core) and the rate (multi cores) versions. The four LHC experiments were involved in order to compare the performances of the application codes for different type of workload (Generation, Simulation, Digitization, Reconstruction) and at least for CMS for different physics channel

The four experiments code were found in good correlation with both CPU 2006 INT and FP if the gcc compiler is used

A low level analysis using the perfmon tool on the CERN lxbatch farm showed that the HEP codes has a percentage of HEP code greater than CPU INT. The working group proposed to use an feature of the SPEC suite that permits to create "ad hoc" workload to increase the percentage of FP tests in the final suite. The all_cpp bset (benchmark set) is a set of seven benchmark, three ( 471.omnetpp, 473.astar, 483.xalancbmk) from the integer suite and four( 444.namd, 447.dealII, 450.soplex, 453.povray) from the floating point suite, that gives a good match of the integer and FP instruction set.

The new benchmark proposed by WLCG as successor of SI2000 has been called HS06.

On each core runs an instance of the all_cpp set of benchmarks. The geometric average of the seven tests is computed. Then the score of each core is summed to have the performance of a single Worker Node. The score of each WN is summed to get the performance of a Site

Download table

the table above shows the HS06 value for several dual processor WN representative of popular CPU. The value for SI2000 and Spec CPU INT 2006 are shown for comparison

How to run HS06

If you want to make HS06 on your own own, detailed instructions are available in a Wiki site at CERN:

In Short you need the following:

1) A machine with any version of Linux compatible with Scientific Linux (RHEL, SL, SLC, CentOS)

2) The gcc compiler should be installed

3) Configuration files and run script (available as a gzipped tar archive from the CERN Wiki or the HEPiX wiki or from here). The archive's md5sum is 9fed92b8d515b88904705f76809c4028.

4) A tar ball of the SPECcpu2006 DVD called SPEC2006_v11.tar.bz2 that should be in the same directory as the run script

Notice for INFN users. CPU 2006 is sold by SPEC with a site Licence. I have a licence for CPU 2006 and I can distribute the tar ball to all the INFN site who will contact me to the following address: michelotto at pd.infn.it

2009 Outlook

The year 2008 has seen the definitive transition to the quad core cpu.

The newest processor are Harpertown (Intel Xeon 54xx) and the Shanghai (AMD Opteron) 23xx built in 45nm technology and four core per chip.

The typical worker node needs about 2GB of main memory per core to run HS06 and HEP application with loosing performances because of memory paging. More memory could be needed for some HEP Application running in 64 bit mode

© 2009 Michele Michelotto, Some Rights Reserved.
Valid XHTML 1.0 Transitional