Direct use of waste heat to minimize carbon-dioxide emissions
Making computing systems and data centers energy-efficient is a staggering undertaking. Today, the IT industry accounts for 2% of the world’s total carbon emissions—more than the amount generated by global air traffic.
“Hot-water cooling has compelling advantages.”
—IBM scientist Bruno Michel
Up to 50% of an average air-cooled data center's energy consumption and carbon footprint is not caused by computing but by powering the necessary cooling systems to keep the processors from overheating. This situation is far from optimal if we look at energy efficiency from a holistic perspective.
The key to reducing a data center's energy consumption is to cool it with hot water. This eliminates the need for today’s energy-hungry chillers in data centers. Moreover, high-grade heat at the output can be used for such needs as heating building spaces.
Schematic concept of a zero-emission data center. Hot water collects heat from electronic components and is then transferred to a district heating system to heat buildings.
In 2010, IBM built a hot-water-cooled supercomputer for the Swiss Federal Institute of Technology (ETH) in Zurich as part of IBM’s First-Of-A-Kind (FOAK) program. Marking a new era in energy-aware computing, this innovative system, dubbed Aquasar, consumed up to 40% less energy than a comparable air-cooled machine. Direct use of waste heating in the hot-water grid of ETH decreased the carbon footprint of the system by up to 85%. The Aquasar supercomputer consisted of special water-cooled IBM BladeCenter Servers, which were designed and manufactured by IBM scientists in Zurich and Böblingen, Germany.
The servers comprised microchannel coolers attached directly to the processors. Thanks to this chip-level cooling, the thermal resistance between the processor and the water was reduced to the extent that even cooling water temperatures of up to 60°C ensured that the processors did not overheat. This high input temperature of the water resulted in high-grade heat at the output, which in the case of Aquasar is up to 65°C.
Aquasar demonstrated that it is possible to reduce the energy consumption of data centers while restraining costs and curtailing carbon emissions. Liquid cooling and deploying waste heat are becoming instrumental in the drive to improve the energy efficiency of data centers.
Building on the experience gained from the Aquasar system, the IBM iDataPlex DWC dx360 M4 and IBM / Lenovo NeXtScale DWC nx360 M5 servers were designed. These servers are deployed in warm-water-cooled, high-performance computing (HPC) systems at Leibniz Rechenzentrum (LRZ), Garching, Germany: SuperMUC Phase 1 (2012) and Phase 2 (2015).
Lenovo NeXtScale DWC nx360 M5 hardware was additionally implemented in CoolMUC-2, an HPC cluster from which waste heat is used to drive adsorption chillers, which in turn provide cooling for storage servers.
Aquasar system and direct water-cooled IBM BladeCenter QS22 and HS22.
Processors and numerous other components are water-cooled, and the server heat is utilized in the hot-water grid.
Warm-water cooled HPC Systems SuperMUC Phase 1 and Phase 2 at Leibniz Rechenzentrum (LRZ), Garching, Germany.
IBM iDataPlex DWC dx360 M4.
IBM / Lenovo NeXtScale DWC nx360 M5.
Publications
[1] M. Cossale, S. Paredes, R.P. Luijten, and B. Michel,
“Combined power delivery and cooling for high density, high efficiency microservers,” 21st Intl. Workshop on Thermal Investigations of ICs and Systems “THERMINIC” (2015).
[2] S. Zimmermann, I. Meijer, M.K. Tiwari, S. Paredes, B. Michel, D. Poulikakos,
“Aquasar: A hot water cooled data center with direct energy reuse,” Energy 43(1), 237–245 (2012).
[3] S. Zimmermann, M.K. Tiwari, I. Meijer, S. Paredes, B. Michel, D. Poulikakos,
“Hot water cooled electronics: Exergy analysis and waste heat reuse feasibility,” Int’l J. Heat and Mass Transf. 55(23), 6391–6399 (2012).
[4] A. Renfer, M.K. Tiwari, R. Tiwari, F. Alfieri, T. Brunschwiler, B. Michel,
“Microvortex-enhanced heat transfer in 3D-integrated liquid cooling of electronic chip stacks,” Int’l J. Heat and Mass Transf. 65, 33–43 (2012).
[5] P. Ruch, T. Brunschwiler, W. Escher, S. Paredes, B. Michel,
“Toward five-dimensional scaling: How density improves efficiency in future computers,” IBM J. Res. Develop. 55(5), 15:1–15:13 (2011).
[6] A. Kubilay, S. Zimmermann, I. Zinovik, B. Michel, D. Poulikakos,
“Compact thermal model for the transient temperature prediction of a water-cooled microchip module in low carbon emission computing,” Numerical Heat Transfer, Part A: Applications 59(11), 815–835 (2011).
[7] G. I. Meijer,
“Cooling Energy-Hungry Data Centers,” Science 328, 316 (2010).
[8] T. Brunschwiler, G.I. Meijer, S. Paredes, W. Escher, and B. Michel,
“Direct Waste Heat Utilization from Liquid-Cooled Supercomputers,” Proc. 14th Int’l Heat Transfer Conf., IHTC14–23352 (2010).
[9] W. Escher, B. Michel, and D. Poulikakos,
“A Novel High Performance, Ultra Thin Heat Sink for Electronics,” Int’l J. Heat and Fluid Flow 31(4), 586–598 (2010).
[10] W. Escher, T. Brunschwiler, B. Michel, and D. Poulikakos,
“Experimental Investigation of an Ultra-thin Manifold Micro-channel Heat Sink for Liquid-Cooled Chips,” Int’l J. Heat Transf. 132(8), 081402–11 (2010).
[11] P. Kasten, S. Zimmermann, M. Tiwari, B. Michel, and D. Poulikakos,
“Hot Water Cooled Heat Sinks for Efficient Data Center Cooling: Towards Electronic Cooling With High Exergetic Utility,” Frontiers in Heat and Mass Transfer 1, 023006 (2010).
[12] T. Brunschwiler, B. Smith, E. Ruetsche and B. Michel,
“Toward Zero-emission Data Centers through Direct Reuse of Thermal Energy,"
IBM J. Res. Develop. 53(3), Paper 11 (2009).
[13] G.I. Meijer, T. Brunschwiler, S. Paredes, and B. Michel,
“Using Waste Heat from Data Centres to Minimize Carbon Dioxide Emission,” ERCIM News 79, 23–24 (2009).
[14] W. Escher, B. Michel, and D. Poulikakos,
“Efficiency of optimized bifurcating tree-like and parallel microchannel networks in the cooling of electronics,” Int’l J. Heat and Mass Transf. 52, 1421–1430 (2009).