Preparing for In Situ Processing on Upcoming Leading-edge Supercomputers

Authors

  • James Kress University of Oregon, Eugene Oak Ridge National Laboratory, Oak Ridge
  • Randy Michael Churchill Princeton Plasma Physics Laboratory, Princeton
  • Scott Klasky Oak Ridge National Laboratory, Oak Ridge
  • Mark Kim Oak Ridge National Laboratory, Oak Ridge
  • Hank Childs University of Oregon, Eugene Lawrence Berkeley National Laboratory, Berkeley
  • David Pugmire Oak Ridge National Laboratory, Oak Ridge

DOI:

https://doi.org/10.14529/jsfi160404

Abstract

High performance computing applications are producing increasingly large amounts of data and placing enormous stress on current capabilities for traditional post-hoc visualization techniques. Because of the growing compute and I/O imbalance, data reductions, including in situ visualization, are required. These reduced data are used for analysis and visualization in a variety of different ways. Many of he visualization and analysis requirements are known a priori, but when they are not, scientists are dependent on the reduced data to accurately represent the simulation in post hoc analysis. The contributions of this paper is a description of the directions we are pursuing to assist a large scale fusion simulation code succeed on the next generation of supercomputers. These directions include the role of in situ processing for performing data reductions, as well as the tradeoffs between data size and data integrity within the context of complex operations in a typical scientific workflow.

References

Sean Ahern, Arie Shoshani, Kwan-Liu Ma, Alok Choudhary, Terence Critchlow, Scott Klasky, Valerio Pascucci, J Ahrens, EW Bethel, H Childs, et al. Scientific discovery at the exascale. Report from the DOE ASCR 2011 Workshop on Exascale Data Management, 2011.

CS Chang, S Ku, PH Diamond, Z Lin, S Parker, TS Hahm, and N Samatova. Compressed ion temperature gradient turbulence in diverted tokamak edgea). Physics of Plasmas (1994-present), 16(5):056108, 2009.

Hank Childs, David Pugmire, Sean Ahern, Brad Whitlock, Mark Howison, Prabhat, Gunther H. Weber, and E. Wes Bethel. Extreme scaling of production visualization software on diverse architectures. IEEE Comput. Graph. Appl., 30(3):22–31, May 2010.

Hank Childs, David Pugmire, Sean Ahern, Brad Whitlock, Mark Howison, Prabhat, Gunther H. Weber, and E. Wes Bethel. Visualization at extreme scale concurrency. In E. Wes Bethel, Hank Childs, and Charles Hansen, editors, High Performance Visualization: Enabling Extreme-Scale Scientific Insight. CRC Press, Boca Raton, FL, 2012.

Jong Y Choi, Kesheng Wu, Jacky C Wu, Alex Sim, Qing G Liu, Matthew Wolf, C Chang, and Scott Klasky. Icee: Wide-area in transit d