In Situ Exploration of Particle Simulations with CPU Ray Tracing

Will Usher; Ingo Wald; Aaron Knoll; Michael Papka; Valerio Pascucci

doi:10.14529/jsfi160401

Authors

Will Usher SCI Institute, University of Utah, Salt Lake City
Ingo Wald Intel Corporation, Santa Clara
Aaron Knoll SCI Institute, University of Utah, Salt Lake City
Michael Papka Argonne National Laboratory, Lemont Northern Illinois University, DeKalb
Valerio Pascucci SCI Institute, University of Utah, Salt Lake City

DOI:

https://doi.org/10.14529/jsfi160401

Abstract

We present a system for interactive in situ visualization of large particle simulations, suitable for general CPU-based HPC architectures. As simulations grow in scale, in situ methods are needed to alleviate IO bottlenecks and visualize data at full spatio-temporal resolution. We use a lightweight loosely-coupled layer serving distributed data from the simulation to a data-parallel renderer running in separate processes. Leveraging the OSPRay ray tracing framework for visualization and balanced P-k-d trees, we can render simulation data in real-time, as they arrive, with negligible memory overhead. This flexible solution allows users to perform exploratory in situ visualization on the same computational resources as the simulation code, on dedicated visualization clusters or remote workstations, via a standalone rendering client that can be connected or disconnected as needed. We evaluate this system on simulations with up to 227M particles in the LAMMPS and Uintah computational frameworks, and show that our approach provides many of the advantages of tightly-coupled systems, with the flexibility to render on a wide variety of remote and coprocessing resources.

References

Alexandre Ancel, Jean-Michel Dischler, and Catherine Mongenet. Load-Balanced Multi-GPU Ambient Occlusion for Direct Volume Rendering.

In Hank Childs, Torsten Kuhlen, and Fabio Marton, editors, Eurographics

Symposium on Parallel Graphics and Visualization. The Eurographics Association, 2012.

Martin Berzins, Justin Luitjens, Qingyu Meng, Todd Harman, Charles A Wight, and Joseph R Peterson.

Uintah: a scalable framework for hazard analysis. In Proceedings of the TeraGrid Conference, page 3. ACM, 2010.

Martin Burtscher and Paruj Ratanaworabhan. pFPC: A parallel compressor for floating-point data. In Data Compression Conference, pages 43-52, 2009.

Hank Childs. In situ terminology project, https://ix.cs.uoregon.edu/ hank/insituterminology.

D. Ellsworth, B. Green, C. Henze, P. Moran, and T. Sandstrom. Concurrent Visualization in a Production Supercomputing Environment. IEEE Transactions on Visualization and Computer Graphics, 12(5):997-1004, 2006.

N. Fabian, K. Moreland, D. Thompson, A.C. Bauer, P. Marion, B. Geveci, M. Rasquin, and K.E. Jansen. The ParaView Coprocessing Library: A scalable, general purpose in situ visualization library. In Symposium on Large Data Analysis and Visualization (LDAV), pages 89-96. IEEE, 2011.

Thomas Fogal, Fabian Proch, Alexander Schiewe, Olaf Hasemann, Andreas Kempf, and Jens Kruger. Freeprocessing: Transparent in situ visualization via data interception. In Eurographics Symposium on Parallel Graphics and Visualization (EGPGV), 2014.

Roland Fraedrich, Jens Schneider, and Rudiger Westermann. Exploring the Millennium Run - Scalable Rendering of Large-Scale Cosmological Datasets. IEEE Transactions on Visualization and Computer Graphics, 15(6):1251-1258, 2009.

Christiaan P Gribble, Thiago Ize, Andrew Kensler, IngoWald, and Steven G Parker. A coherent grid traversal approach to visualizing particle-based simulation data. IEEE Transactions on Visualization and Computer Graphics, (4):758-768, 2007.

Markus Gross and Hanspeter Pfister. Point-Based Graphics. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 2007.

A. V. Pascal Grosset, Manasa Prasad, Cameron Christensen, Aaron Knoll, and Charles Hansen. TOD-tree: Task-overlapped Direct Send Tree Image Compositing for Hybrid MPI Parallelism. In Proceedings of the 15th

Eurographics Symposium on Parallel Graphics and Visualization, pages 67-76. Eurographics Association, 2015.

S. Grottel, M. Krone, C. Muller, G. Reina, and T. Ertl. Megamol: A Prototyping Framework for Particle based Visualization. IEEE Transactions on Visualization and Computer Graphics, 21(2):201-214, 2015.

Robert Haimes and David E Edwards. Visualization in a parallel processing environment. In Proceedings of the 35th AIAA Aerospace Sciences Meeting, number AIAA Paper, pages 97-0348, 1997.

William M. Hsu. Segmented Ray Casting for Data Parallel Volume Rendering. In Proceedings of the Symposium on Parallel Rendering, pages 7-14, 1993.

William Humphrey, Andrew Dalke, and Klaus Schulten. VMD: visual molecular dynamics. Journal of Molecular Graphics, 14(1):33-38, 1996.

S. Kumar, V. Vishwanath, P. Carns, B. Summa, G. Scorzelli, V. Pascucci, R. Ross, J. Chen, H. Kolla, and R. Grout. PIDX: Effcient Parallel I/O for Multi-resolution Multi-dimensional Scientific Datasets. In Proceedings of The IEEE International Conference on Cluster Computing, pages 103-111, 2011.

P. Lindstrom. Fixed-Rate Compressed Floating-Point Arrays. IEEE Transactions on Visualization and Computer Graphics, 20(12):2674-2683, 2014.

J. Lofstead, Fang Zheng, S. Klasky, and K. Schwan. Adaptable, metadata rich IO methods for portable high performance IO. In IEEE International Symposium on Parallel Distributed Processing, pages 1-10, 2009.

Kwan-Liu Ma, James S. Painter, Charles D. Hansen, and Michael F. Krogh. A Data Distributed, Parallel Algorithm for Ray-traced Volume Rendering. In Proceedings of the Symposium on Parallel Rendering, pages 15-22, 1993.

Bruce Howard McCormick, Thomas A DeFanti, and Maxine D Brown. Visualization in scientific computing. IEEE Computer Graphics and Applications, 7(10):69-69, 1987.

Kenneth Moreland, Wesley Kendall, Tom Peterka, and Jian Huang. An Image Compositing Solution at Scale. In Proceedings of International Conference for High Performance Computing, Networking, Storage and

Analysis, pages 25:1-25:10. ACM, 2011.

Steven G Parker and Christopher R Johnson. SCIRun: a scientific programming environment for computational steering. In Proceedings of the ACM/IEEE conference on Supercomputing, 1995.

T. Peterka, J. Kwan, A. Pope, H. Finkel, K. Heitmann, S. Habib, J. Wang, and G. Zagaris. Meshing the Universe: Integrating Analysis in Cosmological Simulations. In High Performance Computing, Networking, Storage and Analysis (SCC), SC Companion, pages 186-195, 2012.

Tom Peterka, Dmitriy Morozov, and Carolyn Phillips. High-performance computation of distributed-memory parallel 3D Voronoi and Delaunay tessellation. In Proceedings of the International Conference for High

Performance Computing, Networking, Storage and Analysis, pages 997-1007, 2014.

Steve Plimpton. Fast parallel algorithms for short-range molecular dynamics. Journal of computational physics, 117(1):1-19, 1995.

Vilas G Pol, Jianguo Wen, Kah Chun Lau, Samantha Callear, Daniel T Bowron, Chi-Kai Lin, Sanket A Deshmukh, Subramanian Sankaranarayanan, Larry A Curtiss, William IF David, et al. Probing the evolution and morphology of hard carbon spheres. Carbon, 68:104-111, 2014.

S. Rizzi, M. Hereld, J. Insley, M.E. Papka, T. Uram, and V. Vishwanath. Large-scale co-visualization for LAMMPS using vl3. In Symposium on Large Data Analysis and Visualization (LDAV), pages 141-142. IEEE, 2015.

Silvio Rizzi, Mark Hereld, Joseph Insley, Michael E Papka, Thomas Uram, and Venkatram Vishwanath. Largescale parallel visualization of article-based simulations using point sprites and level-of-detail. In Proceedings of the 15th Eurographics Symposium on Parallel Graphics and Visualization, pages 1-10. Eurographics Association, 2015.

Sandia National Labs. LAMMPS Molecular Dynamics Simulator.

M. Tarini, P. Cignoni, and C. Montani. Ambient Occlusion and Edge Cueing for Enhancing Real Time Molecular Visualization. IEEE Transactions on Visualization and Computer Graphics, pages 1237-1244, 2006.

T. Tu, H. Yu, L. Ramirez-Guzman, J. Bielak, O. Ghattas, K. l. Ma, and D. R. O'Hallaron. From Mesh Generation to Scientific Visualization: An End-to-End Approach to Parallel Supercomputing. In SC Conference, Proceedings of the ACM/IEEE, pages 12-12, 2006.

V. Vishwanath, M. Hereld, and M. E. Papka. Toward simulation-time data analysis and I/O acceleration on leadership-class systems. In Symposium on Large Data Analysis and Visualization (LDAV), pages 9-14. IEEE, 2011.

Venkatram Vishwanath, Mark Hereld, Vitali Morozov, and Michael E. Papka. Topology-aware Data Movement and Staging for I/O Acceleration on Blue Gene/P Supercomputing Systems. In Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, pages 19:1-19:11, 2011.

I. Wald, G. Johnson, J. Amstutz, C. Brownlee, A. Knoll, J. Jeers, J. Gunther, and P. Navratil. OSPRay - A CPU Ray Tracing Framework for Scientic Visualization. IEEE Transactions on Visualization and Computer

Graphics, PP(99):1-1, 2016.

I. Wald, A. Knoll, G. P. Johnson, W. Usher, V. Pascucci, and M. E. Papka. CPU Ray Tracing Large Particle Data with Balanced P-k-d Trees. In Proceedings of IEEE Visweek, 2015.

J. Woodring, J. Ahrens, J. Figg, J. Wendelberger, S. Habib, and K. Heitmann. In-situ Sampling of a Large-Scale Particle Simulation for Interactive Visualization and Analysis. Computer Graphics Forum, 30(3):1151-1160, 2011.

Hongfeng Yu, Chaoli Wang, R.W. Grout, J.H. Chen, and Kwan-Liu Ma. In Situ Visualization for Large-Scale Combustion Simulations. IEEE Computer Graphics and Applications, 30(3):45-57, 2010.

F. Zhang, S. Lasluisa, T. Jin, I. Rodero, H. Bui, and M. Parashar. In-situ Feature-Based Objects Tracking for Large-Scale Scientic Simulations. In High Performance Computing, Networking, Storage and Analysis (SCC), SC Companion, pages 736-740, 2012.