Performance Analysis and Prediction
Understanding the performance and cost of a particular approach is the first step towards improving it, but understanding performance and cost is often challenging. For approaches that have been implemented on existing systems, obtaining such understanding may require measurement and analysis. For scenarios where the hardware or software under consideration does not yet exist, performance prediction using analytical modeling or simulation may be necessary.
The ORNL Future Technologies Group includes a team of members investigating techniques for understanding performance and cost, with special focus on high-performance computing environments and on emerging computing technologies. Our projects include both empirical approaches for performance measurement and analysis, and performance prediction using modeling and simulation.
Personnel
- Team:
- Primary Collaborators:
- Patrick Worley, Computer Science and Mathematics
Division, ORNL
Projects
As noted above, performance analysis and prediction impacts all of our group's work, from early evaluation studies to parallel I/O. For our work that deals specifically with performance analysis and prediction techniques, we are pursuing several directions. For some projects, we have publications and even code downloads; for other projects that aren't yet that far along, we encourage you to check back for project updates. If possible, related publications are also available from the Future Technologies publications database. Contact any team member with questions or comments.- Modeling Assertions
- With Modeling Assertions (MA), we are exploring techniques for embedding symbolic models into code such that they can be verified automatically at runtime. We are actively pursuing research topics like automatic model generation, dynamic model verification, and large-scale parallel system simulation for performance prediction. This work builds on our successful Performance Assertions technique, that embeds performance expectations into code to be checked at run time using timers and hardware counters.
- For more details, see "A Framework to Develop Symbolic Performance Models of Parallel Applications" from the 5th International Workshop on Performance Modeling, Evaluation, and Optimization of Parallel and Distributed Systems. (2006).
- Scalable Tools
- To be effective for understanding the performance of applications and systems software on large-scale computing systems, a tool must scale at least as well as the system itself. This scalability must be present in all aspects of the tool, from data collection to data analysis to its presentation of results.
- We are exploring not only the use of overlay networks like the MRNet tool infrastructure software for implementing scalable tool communication and data manipulation functionality, but also the impact of such networks in high-performance computing environments where scalable tools are most needed. This work also considers techniques for visualizing performance data from high-end computing systems.
- For more details, see the PPoPP 2006 paper "On-line Automated Performance Diagnosis on Thousands of Processes" and "Exploring Tree-Based Overlay Networks on the Cray XT", presented at the 2007 Dagstuhl seminar on Code Instrumentation and Modeling for Parallel Performance Analysis.
- We also help maintain the mpiP lightweight MPI profiling tool, with special focus on making mpiP effective on the leadership computing platforms at ORNL.
Code Download
- As code becomes available for download, links will be posted here. Please check back!
Related Projects
- Other Future Technologies Group projects where we apply our techniques
- SciDAC II Performance Engineering Research Institute (PERI)
- The Paradyn parallel performance tool project, including MRNet tool infrastructure
- mpiP: Lightweight, Scalable MPI Profiling