FT Investigation on Characterization and Optimization of Large-Scale Parallel I/O on Jaguar

FT group members Weikuan Yu and Jeffrey S. Vetter, along with Sarp Oral of the NCCS Technology Integration group, will be presenting their paper "Performance Characterization and Optimization of Parallel I/O on the Cray XT" at the 22nd IEEE International Parallel and Distributed Processing Symposium in Miami, Florida. The paper presents an extensive characterization, tuning, and optimization of parallel I/O on the Cray XT supercomputer at Oak Ridge National Laboratory. This characterization covers both data- and metadata-intensive I/O patterns. The authors shed light on the effectiveness of several parallel I/O techniques, such as data sieving and two-phase collective I/O, on the Cray XT. Moreover, the authors have demonstrated that it is possible, and often prudent, to improve the I/O performance of scientific benchmarks and applications by carefully optimizing their application's I/O operations. For example, they show that the I/O performance of the S3D combustion application can be improved at large scale by tuning the I/O system to avoid a bandwidth degradation of 49% with 8192 processes when compared to 4096 processes.  For more information see the paper.