In situ, steerable, hardware-independent and data-structure agnostic visualization with ISAAC

Alexander Matthes; Axel Huebl; René Widera; Sebastian Grottel; Stefan Gumhold; Michael Bussmann

doi:10.14529/jsfi160403

Authors

Alexander Matthes Helmholtz-Zentrum Dresden - Rossendorf, Dresden
Axel Huebl Helmholtz-Zentrum Dresden - Rossendorf, Dresden
René Widera Helmholtz-Zentrum Dresden - Rossendorf, Dresden
Sebastian Grottel Technische Universit ̈at Dresden, Dresden
Stefan Gumhold Technische Universit ̈at Dresden, Dresden
Michael Bussmann Helmholtz-Zentrum Dresden - Rossendorf, Dresden

DOI:

https://doi.org/10.14529/jsfi160403

Abstract

The computation power of supercomputers grows faster than the bandwidth of their storage and network. Especially applications using hardware accelerators like Nvidia GPUs cannot save enough data to be analyzed in a later step. There is a high risk of loosing important scientific information. We introduce the in situ template library ISAAC which enables arbitrary applications like scientific simulations to live visualize their data without the need of deep copy operations or data transformation using the very same compute node and hardware accelerator the data is already residing on. Arbitrary meta data can be added to the renderings and user defined steering commands can be asynchronously sent back to the running application. Using a aggregating server, ISAAC streams the interactive visualization video and enables user to access their applications from everywhere.

References

James Ahrens, S ́ebastien Jourdain, Patrick O’Leary, John Patchett, David H Rogers, and Mark Petersen. An image-based approach to extreme scale in situ visualization and analysis. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, pages 424–434. IEEE Press, 2014.

Janine C Bennett, Hasan Abbasi, Peer-Timo Bremer, Ray Grout, Attila Gyulassy, Tong Jin, Scott Klasky, Hemanth Kolla, Manish Parashar, Valerio Pascucci, et al. Combining in-situ and in-transit processing to enable extreme-scale scientific analysis. In High Performance Computing, Networking, Storage and Analysis (SC), 2012 International Conference for, pages 1–9. IEEE, 2012.

John Biddiscombe, Jerome Soumagne, Guillaume Oger, David Guibert, and Jean-Guillaume Piccinali. Parallel computational steering and analysis for hpc applications using a paraview interface and the hdf5 dsm virtual file driver. In Eurographics Symposium on Parallel Graphics and Visualization, pages 91–100. Eurographics Association, 2011.

Tim Bray. The JavaScript Object Notation (JSON) Data Interchange Format. https://tools.ietf.org/html/rfc7159, 29 February 2016. [Online; accessed September 2, 2016].

Heiko Burau, Ren ́ee Widera, Wolfgang Honig, Guido Juckeland, Alexander Debus, Thomas Kluge, Ulrich Schramm, Tomas E Cowan, Roland Sauerbrey, and Michael Bussmann. Picongpu: A fully relativistic particle-in-cell code for a gpu cluster. IEEE Transactions on Plasma Science, 38(10):2831–2839, 2010.

Michael Bussmann, Heiko Burau, Thomas E Cowan, Alexander Debus, Alex Huebl, Guido Juckeland, Thomas Kluge, Wolfgang E Nagel, Richard Pausch, Felix Schmitt, et al. Radiative signatures of the relativistic kelvin-helmholtz instability. In Proceedings of the Inter-national Conference on High Performance Computing, Networking, Storage and Analysis, page 5. ACM, 2013.

Andy Cedilnik, Berk Geveci, Kenneth Moreland, James P Ahrens, and Jean M Favre. Remote large data visualization in the paraview framework. In EGPGV, pages 163–170, 2006.

Hank Childs. Visit: An end-user tool for visualizing and analyzing very large data. 2013.

Nathan Fabian, Kenneth Moreland, David Thompson, Andrew C Bauer, Pat Marion, Berk Gevecik, Michel Rasquin, and Kenneth E Jansen. The paraview coprocessing library: A scalable, general purpose in situ visualization library. In Large Data Analysis and Visualization (LDAV), 2011 IEEE Symposium on, pages 89–96. IEEE, 2011.

Oliver Fernandes, David S Blom, Steffen Frey, Alexander H Van Zuijlen, Hester Bijl, and Thomas Ertl. On in-situ visualization for strongly coupled partitioned fluid-structure interaction. In Coupled Problems 2015: Proceedings of the 6th International Conference on Computational Methods for Coupled Problems in Science and Engineering, Venice, Italy, 18-20 May 2015. CIMNE, 2015.

Robert Hagan and Yong Cao. Multi-gpu load balancing for in-situ visualization. In Int. Conf. on Parallel and Distributed Processing Techniques and Applications, 2011.

Roger W Hockney and James W Eastwood. Computer simulation using particles. CRC Press, 1988.

Akira Kageyama and Tomoki Yamada. An approach to exascale visualization: Interactive viewing of in-situ visualization. Computer Physics Communications, 185(1):79–85, 2014.

Sriram Lakshminarasimhan, Neil Shah, Stephane Ethier, Scott Klasky, Rob Latham, Rob Ross, and Nagiza F Samatova. Compressing the incompressible with isabela: In-situ reduction of spatio-temporal data. In European Conference on Parallel Processing, pages 366–379. Springer, 2011.

Peter Lindstrom. Fixed-rate compressed floating-point arrays. IEEE transactions on visualization and computer graphics, 20(12):2674–2683, 2014.

Peter Lindstrom and Martin Isenburg. Fast and efficient compression of floating-point data. IEEE transactions on visualization and computer graphics, 12(5):1245–1250, 2006.

Kwan-Liu Ma. In situ visualization at extreme scale: Challenges and opportunities. IEEE Computer Graphics and Applications, 29(6):14–19, 2009.

Kwan-Liu Ma, James S Painter, Charles D Hansen, and Michael F Krogh. Parallel volume rendering using binary-swap compositing. IEEE Computer Graphics and Applications, 14(4):59–68, 1994.

Kwan-Liu Ma, Chaoli Wang, Hongfeng Yu, and Anna Tikhonova. In-situ processing and visualization for ultrascale simulations. In Journal of Physics: Conference Series, volume 78, page 012043. IOP Publishing, 2007.

Alexander Matthes. ISAAC JSON Commands. http://computationalradiationphysics.github.io/isaac/doc/json/index.html. [Online; accessed September 2, 2016].

Alexander Matthes. ISAAC Website. http://computationalradiationphysics.github.io/isaac/. [Online; accessed September 6, 2016].

Hans Werner Meuer, Erich Strohmaier, Jack Dongarra, Horst Simon, and Martin Meuer. June 2016 — TOP500 Supercomputer Sites. https://www.top500.org/lists/2016/06/, June 2016. [Online; accessed September 2, 2016].

Kenneth Moreland, Utkarsh Ayachit, Berk Geveci, and Kwan-Liu Ma. Dax toolkit: A proposed framework for data analysis and visualization at extreme scale. In Large Data Analysis and Visualization (LDAV), 2011 IEEE Symposium on, pages 97–104. IEEE, 2011.

Liu Ning, Gao Guoxian, Zhang Yingping, and Zhu Dengming. The design and implement of ultra-scale data parallel in-situ visualization system. In Audio Language and Image Processing (ICALIP), 2010 International Conference on, pages 960–963. IEEE, 2010.

Marzia Rivi, Luigi Calori, Giuseppa Muscianisi, and Vladimir Slavnic. In-situ visualization: State-of-the-art and some use cases. PRACE White Paper; PRACE: Brussels, Belgium, 2012.

Sandia National Laboratories. Sandia National Laboratories: IceT Documentation. http://icet.sandia.gov/documentation/index.html, January 2011. [Online; accessed September 5, 2016].

John E Stone, Kirby L Vandivort, and Klaus Schulten. Gpu-accelerated molecular visualization on petascale supercomputing platforms. In Proceedings of the 8th International Workshop on Ultrascale Visualization, page 6. ACM, 2013.

Tiankai Tu, Hongfeng Yu, Leonardo Ramirez-Guzman, Jacobo Bielak, Omar Ghattas, Kwan-Liu Ma, and David R O’hallaron. From mesh generation to scientific visualization: An end-to-end approach to parallel supercomputing. In Proceedings of the 2006 ACM/IEEE conference on Supercomputing, page 91. ACM, 2006.

Brad Whitlock, Jean M. Favre, and Jeremy S. Meredith. Parallel in situ coupling of simulation with a fully featured visualization system. 2011.

Jonathan Woodring, J Ahrens, J Figg, Joanne Wendelberger, Salman Habib, and Katrin Heitmann. In-situ sampling of a large-scale particle simulation for interactive visualization

and analysis. In Computer Graphics Forum, volume 30, pages 1151–1160. Wiley Online Library, 2011.

Hongfeng Yu, Tiankai Tu, Jacobo Bielak, Omar Ghattas, Julio L ́opez, Kwan-Liu Ma, David R O’hallaron, Leonardo Ramirez-Guzman, Nathan Stone, Ricardo Taborda-Rios, et al. Remote runtime steering of integrated terascale simulation and visualization. 2006.

Hongfeng Yu, Chaoli Wang, R Grout, J Chen, and K Ma. A study of in situ visualization for petascale combustion simulations. Technical report, Citeseer, 2009.33. Hongfeng Yu, Chaoli Wang, and Kwan-Liu Ma. Massively parallel volume rendering using 2–3 swap image compositing. In 2008 SC-International Conference for High Performance Computing, Networking, Storage and Analysis, pages 1–11. IEEE, 2008.

Erik Zenker, Ren ́e Widera, Axel Huebl, Guido Juckeland, Andreas Knüpfer, Wolfgang E Nagel, and Michael Bussmann. Performance-portable many-core plasma simulations: Porting picongpu to openpower and beyond. arXiv preprint arXiv:1606.02862, 2016.

Erik Zenker, Benjamin Worpitz, Ren ́e Widera, Axel Huebl, Guido Juckeland, Andreas Knüpfer, Wolfgang E Nagel, and Michael Bussmann. Alpaka-an abstraction library for parallel kernel acceleration. arXiv preprint arXiv:1602.08477, 2016.