Electronics densification is continuing at an unrelenting pace at the server, rack, and facility level. With increasing facility density levels, airflow management has become a major challenge and concern. In an effort to deal with the resulting thermal management challenges, manufacturers are increasingly turning to liquid cooling as a practical solution. The majority of manufacturers have turned to liquid-cooled enclosed racks or rear-door heat exchangers, in which chilled water is delivered to the racks. Some manufacturers are now looking to cold plate cooling solutions that take the heat directly off problem components, such as the CPUs, to get it directly out of the facility.

This paper describes work done at the Pacific Northwest National Labs (PNNL) under a Department of Energy-funded program titled “Energy Smart Data Center.” An 8.2 kW (27,980 Btu/h) rack of HP rx2600 2U servers has been converted from air-cooling to liquid spray cooling (CPUs only). The rack has been integrated into PNNL’s main cluster and subjected to a suite of acceptance tests. Under the testing, the spray-cooled CPUs ran an average of 10°C (18°F) cooler than the air-cooled CPUs. Other peripheral devices, such as the memory DIMMs, ran an average of 8°C (14.4°F) cooler, and the power pod board was measured at 15°C (27°F) cooler. Since installation in July 2005, the rack has been undergoing a one-year uptime and reliability investigation. As part of the investigation, the rack has been subjected to monthly robustness testing and ongoing performance evaluation while running applications such as High Performance Linpack, parts of the NASA NPB-2 Benchmark Suite, and NWChem. The rack has undergone three months’ worth of robustness testing with no major events. Including the robustness testing, the rack uptime is at 95.54% over 299 days. While undergoing application testing, no computational performance differences have been observed between the liquid-cooled and standard aircooled racks. A small-scale (8–10 racks) spray-cooled Energy Smart Data Center is now being designed as a final step to demonstrate the feasibility of scaling liquid cooling at the single rack up to an entire facility.

Units: Dual