Multipurpose Reconfigurable Supercomputer with Immersion Cooling
DOI:
https://doi.org/10.14529/jsfi230204Keywords:
supercomputers, reconfigurable computing systems, computing performance, immersion cooling systems, computing energy efficiency, computational density, highly connected problemsAbstract
In the paper we consider a promising universal reconfigurable supercomputer, the computing nodes of which are reconfigurable computing device Arcturus. It was developed at the Supercomputers and Neurocomputers Research Center (Taganrog) and based on modern Xilinx FPGAs of UltraScale+ family of HBM-series. The purpose is to achieve the highest computational layout density, to ensure balanced power supply and cooling, as well as the implementation of powerful data exchange configuration. The supercomputer can have up to 1.5 thousand FPGAs of a single computing field. It has extensive information exchange capabilities between FPGAs within the device and between devices to solve tightly coupled problems. Differential lines with multi-gigabit transceivers connected to them are used as the main connections between FPGAs. It provides exchange at the velocity up to 25 Gbit/s. Information interaction between the RCD is performed through optical channels with the capacity up to 4.5 Tbit/s. Immersion technology is used for cooling components of computing system. It provides the removal of the total heat output up to 20 kW. The developed power supply configuration is based on the input constant voltage of 380 V and provides stable power supply to the components. Owning to the implementation of time-consuming algorithms for various scientific and technical problems with the high real performance, it is possible to widespread use of the Arcturus supercomputer. Scaling of computing nodes will allow designing an entire computing circuit of a supercomputer with the performance up to several tens of Petaflops.
References
Kalyaev, A.V., Levin, I.I.: Modular-stackable multiprocessor systems with structural and procedural organization of calculations. Janus-K Publishing house, Moscow (2003). 380 p.
Kalyaev, I.A., Levin, I.I.: Reconfigurable computing systems based on FPGAs. SSC RAS Publishing House, Rostov-on-Don (2022). 475 p.
Kalyaev, I.A., Levin, I.I.: Reconfigurable multiconveyor computing systems for solving streaming problems. Information technologies and computing systems 2, 12–22 (2011)
Levin, I.I, Dordopulo, A.I., Fedorov, A.M., Doronchenko, Yu.I.: Development of technology for constructing reconfigurable computing systems with liquid cooling. Supercomputer Technologies. 5th All-Russian Scientific and Technical Conference. Materials, vol. 1, pp. 184–187. SFedU Publishing House, Taganrog (2018)
Kalyaev, I.A., Levin, I.I.: Reconfigurable computing systems with high real performance. In: International Scientific Conference, Parallel Computational Technologies, PaCT-2009. Proceedings, 186 p. SUSU Publishing House, Chelyabinsk (2009)
BiXBiT immersion cooling (2018). https://bixbit.io/assets/docs/ru/investment_project.pdf, accessed: 2023-04-19
Abramov, S., Amelkin, S., Klyuev, L., Chichkovsky, A.: Liquid cooling of computing complexes. Radioelectronic technologies 5, 79–82 (2017)
Klyuev, L., Hrebtovsky, I.: Experience in building HPC clusters of IMMERS using immersion liquid cooling (2015). http://www2.sscc.ru/Seminars/paper/2015/2015-05-14_IMMERS.pdf, accessed: 2023-04-19
Besedin, I.V., Dmitrenko, N.N., Kalyaev, I.A., et al.: A family of basic modules for building reconfigurable computing systems with a structural and procedural organization of computing. In: Scientific service on the Internet. Proceedings of the All-Russian Conference. pp. 47–49. MSU, RSU, IVT RAS (2006).
Levin, I.I., Pelipets, A.V.: Effective implementation of the parallelization on reconfigurable computer systems. Vestnik komp’iuternykh i informatsionnykh tekhnologii (Herald of computer and information technologies), 11–16 (2018). https://doi.org/10.14489/vkit.2018.08.pp.011-016
Levin, I.I., Dordopulo, A.I., Fedorov, A.M., Gulenok, A.A.: Reconfigurable computer based on Virtex UltraScale+ FPGAs with immersion cooling system. In: International Scientific Conference, Parallel Computational Technologies, PaCT-2017. Proceedings. pp. 27–41. SUSU Publishing Center, Chelyabinsk (2017)
Kalyaev, I.A., Dordopulo, A.I., Levin, I.I., Fedorov, A.M.: Evolution domestic of multichip reconfigurable computer systems: from air to liquid cooling: from air to liquid cooling. Trudy SPIIRAN 1(50), 5–31 (2017). https://doi.org/10.15622/sp.50.1
Levin, I.I., Doronchenko, Yu.I., Dordopulo, A.I., Levina, M.G.: High-performance reconfigurable computing system based on FPGA XILINX ULTRASCALE+. In: International Conference, XXII Kharitonov thematic scientific readings. Collection of scientific papers. pp. 284–296. FSUE “RFNC-VNIIEF”, Sarov (2022).
Levin, I., Dordopulo, A., Fedorov, A., Doronchenko, Yu.: High-Performance Reconfigurable Computer Systems with Immersion Cooling. In: Sokolinsky, L., Zymbler, M. (eds.) Parallel Computational Technologies. PCT 2018. Communications in Computer and Information Science, vol. 910, pp. 62–76. Springer, Cham (2018). https://doi.org/10.1007/978-3-319-99673-8_5
Cerebras. https://www.cerebras.net/product-system/, accessed: 2023-04-20
NVIDIA Data Center Deep Learning Product Performance (2023). https://developer.nvidia.com/deep-learning-performance-training-inference, accessed: 2023-04-20
Dell Precision Rack 7910 Owner’s Manual. https://www.karma-group.ru/upload/iblock/d16/precision-r7910-manual.62BF24D6F1B748C9BB6DA43755132D49.pdf, accessed: 2023-04-21
SPEC CPU2017 Floating Point Rate Results (2017). https://spec.org/cpu2017/results/res2017q3/cpu2017-20170828-00069.pdf, accessed: 2023-04-21
Downloads
Published
How to Cite
License
Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution-Non Commercial 3.0 License that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.