Project Funding Agencies Office of Science: OASCR National Nuclear Security Administration: NNSA

Building a Community Infrastructure for Scalable On-Line Performance Analysis Tools Around Open|SpeedShop

“Building a Community Infrastructure for Scalable On-Line Performance Analysis Tools Around Open|SpeedShop” is a project in the program titled “Software Development Tools for Improved Ease-of-Use of Petascale Systems” program of the US Department of Energy's Office of Advanced Computing Research (OASCR). This project is also known by its shortened name of Component Based Tool Framework (CBTF).

The goal of this project is to create a community tool infrastructure that provides individual tool building components to allow others to add or replace modules, to integrate parts into their own tools, or to customize existing solutions. This project builds on the existing Open|SpeedShop infrastructure and decomposes it into individual modules that match the necessary tool components. At the same time the project addresses the challenges found in performance tools for petascale systems in each module. Once assembled, this instantiation of community tool infrastructure will provide an enhanced version of Open|SpeedShop, which, while completely different in its architecture, will provide scalable performance analysis for petascale applications through a familiar interface. This project also builds upon and enhances capabilities and reusability of project partner components as specified in the original project proposal.

CBTF Project Team Members

http://www.krellinst.orgThe Krell Institute http://www.wisc.eduUniversity of Wisconsin http://www.maryland.eduUniversity of Maryland

http://www.ornl.govOak Ridge National Laboratory http://www.llnl.govLawrence Livermore National Laboratory

CBTF Project Team Members: Collaborators

http://www.lanl.govLos Alamos National Laboratory http://www.sandia.govSandia National Laboratories http://www.cmu.eduCarnegie Mellon University

Component Based Tool Framework (CBTF) Project Technical Overview

The goal of the CBTF project is to provide users and tool developers a tool development framework which expedites the ability to combine compatible reusable components into performance tools of differing levels of complexity. In other words, the CBTF project provides support for creating a compatible interface for components, finding compatible components and defining an interface API which facilitates communication between components. In the CBTF framework, a component is defined as a “black box” entity with zero or more inputs and zero or more outputs.

This diagram shows the abstract “black box” view of a component and the concrete, implementation through an API, on the right side. The diagram also illustrates the definition of a component. Components are based on a data-flow model, accepting zero or more inputs, performing arbitrary processing, and emitting zero or more outputs. They are written using C++, inheriting from a base class that provides basic infrastructure. They provide metadata in form of the component’s type, version, and input/output names and types. All components are versioned and multiple versions the same component can co-exist. Finally, components can be packaged in a variety of different ways.

The definition of components and the connection of components can be done using the light weight CBTF library by making the appropriate API calls to generate the single process component network. The diagram below gives a high level overview of how the CBTF library would facilitate the creation of a single process example tool from components built using the CBTF framework. Each network contains one or more component instances and connections between those component instances. Any topology is possible, from pipelines to graphs with cycles. Networks are recursive in the sense that the network is itself a component, with input and outputs, which can be instantiated and further connection to other networks.

Additionally, the single process component network can be generated automatically by specifying the component network definition and component connection using a network specification written in XML. The XML specification file contains all the necessary information about each of the components and how they are connected to each other. Using the CBTF XML and CBTF base libraries, a tool builder can create a tool from a set of components by specifying an XML specification file that defines the components and how they are connected. Specification of the component network in XML is now done manually, but it could be automated by the development of a GUI tool. A GUI tool is planned for creating the description file, as these files are verbose in nature.

The distribution of the CBTF component network across nodes of a distributed system is also supported. The CBTF library named libcbtf-mrnet supports instantiation of component networks at various levels of a MRNet based tree network. The following diagram illustrates how single component or multiple component networks can be deployed at the backend, communication, and frontend node levels within the MRNet tree based network.

The purpose of the diagram above is to show the definition distributed component networks. These are (currently) MRNet based. A complete component network is located on each node in the MRNet network. These component networks are connected via symbolically-named MRNet streams (shown in red) which can move data either up or down the MRNet network. As with non-distributed component networks, distributed component networks are recursively defined and specified via XML files. Finally, light-weight MRNet is supported by allowing messages with specific MRNet tags to be associated with named streams.

For more details visit the CBTF details page(s): CBTF details

Internal project pages (Project partners only)

cbtfw/start.txt · Last modified: 2011/05/13 13:59 by rothpc
Recent changes RSS feed Driven by DokuWiki