Life would be so much easier if only we had the source code...
Home -> Teaching -> MPI Tutorials -> ISC'12
MPI Tutorials

  Full CV [pdf]


  Past Events

Next Generation MPI Programming: Advanced MPI-2 and New Features in MPI-3 at ISC'12

Next Generation MPI Programming: Advanced MPI-2 and New Features in MPI-3


  Torsten Hoefler, University of Illinois at Urbana-Champaign and ETH Zürich
  Martin Schulz, Lawrence Livermore National Laboratory



The Message Passing Interface is the de-facto standard for parallel programming. It offers a comprehensive set of point-to-point and collective communication operations and One Sided access semantics. MPI's complexity is manifold: novice users can compose parallel programs with only six MPI function calls, while advanced users can benefit from the higher abstraction and performance portability of the 200+ remaining calls. Often overlooked features, such as datatypes, can directly optimize program execution and reduce energy consumption. This tutorial will educate MPI users about advanced features in the current MPI-2.2 and is the first to present a detailed overview of new features and capabilities as well as changes in the upcoming MPI-3.0. We will discuss advanced use of MPI datatypes, nonblocking point-to-point and nonblocking collective communications, topology mapping, and neighborhood collective communication. We will also discuss advanced usage of MPI-2 and MPI-3 One Sided operations, which represent a PGAS model to the user. Hybrid programming principles, language bindings and an overview of tools and new query interfaces will round up our presentation of advanced MPI-3.0. We expect that tutorial attendees will be able to use MPIs advanced features to improve their existing code and their productivity for writing new applications.

Detailed Description

Overview and Goals

This tutorial will introduce advanced MPI-2.2 and new MPI-3.0 functions and concepts. We will demonstrate use-cases and examples for the discussed concepts. New concepts such as nonblocking or neighborhood collective communication are intended to raise the level of abstraction to allow the user to specify ``what is done'' and not ``how it is done'' of his algorithms to the library which can then optimize to the particular target architecture. Existing concepts, such as MPI datatypes can often be used to improve application performance and programmer productivity.

The attendees will understand how the concepts are intended to function and under which circumstances the functions should be used or when they should not be used. We will also discuss MPI algorithms for advanced communication problems.

The new MPI-3.0 standard will introduce new interfaces to query and manipulate both configuration and performance data through the newly defined MPI tool information interface. We will provide a first look at its capabilities and provide a series of use cases for usage both inside of applications as well as tools.

Additionally, MPI-3 includes s series of small updates, changes and additions, which we will summarize for the attendees along with a description of their impact and consequences for application developers. We intend for this section to raise awareness for minor features often overshadowed by the major updates, but still provide powerful additions for the optimization or efficient design of MPI codes. We will further briefly discuss open proposals for future MPI versions as well as the process that's behind the standardization of MPI.

Targeted audience

Industry (ISVs), academics, researchers, developers

Content level

50% Intermediate + 50% Advanced

We assume that people are familiar with MPI basics and that they have used MPI before. However, several new features target the intermediate level while others will be more advanced.

Audience prerequisites

Familiarity with C (or Fortran). Some familiarity with MPI (used it before and understand basic concepts). Basic concepts of distributed memory parallel programming.

Preliminary Agenda

Section Title and Outline
9:00-9:20 Hoefler Introduction to Advanced MPI
 * Overview of MPI-3.0 chapters and concepts
9:20-10:15 Hoefler Advanced MPI Concepts - Datatypes
 * Introduction to the declarative concepts and the (generic) semantic power
 * Overview of the constructors
 * Example: stencil code (simple), FFT (advanced), Morton order storage (advanced)
10:15-11:00 Hoefler Nonblocking and MPI-3.0 Nonblocking Collective Communication
 * Introduction: semantics, deadlock avoidance, performance (overlap, early binding)
 * Nonblocking point-to-point and collective communication
 * Software pipelining
 * Semantic advantages of nonblocking collectives: DSDE
11:30-12:15 Hoefler Topology mapping and MPI-3.0 Neighborhood Collective Communication
 * Topology mapping basics, portability
 * Cartesian topologies, use-cases
 * Arbitrary (distributed) graph topologies, use-cases
 * Neighborhood collective formulation and operations, use-cases
12:15-13:00 Hoefler MPI-3.0 One-sided Communication
 * Introduction and concepts, memory models
 * Creating and destroying windows
 * Accessing remote memory, consistency and synchronization
 * Use-cases and examples
Lunch Break
14:00-14:45 Schulz The MPI Info Object and its Use
 * Basic Concept and Usage
 * MPI Info object creation and handling
 * Optimization opportunities for MPI routines utilizing MPI Info objects
 * Enhancements for MPI Info in MPI 3.0
14:45-15:00 Schulz Language Bindings and Impact on the Profiling Interface
 * Removal of C++ Bindings
 * Fortran 2008 Support
 * Impact on the MPI Profiling Interface
15:00-16:00 Schulz The MPI Tool Information Interface
 * Motivation, Concepts, and Capabilities
 * Reading and Setting Control Variables
 * Using Performance Variables to Analyze and Tune Application Performance
16:30-16:50 Schulz Process Acquisition Interface
 * Overview and General Usage Information of the MPIR Interface
 * Step by step explanation on using MPIR
16:50-17:20 Schulz Minor MPI-3 Enhancements and Changes
 * Matched probe/receive
 * Query minor/implementation speciic versions
 * Nonblocking Communicator Duplication
 * Support for large counts using derived datatypes
 * (this list will be adjusted based on the March and May MPI Forum meetings)
17:20-17:50 Schulz Overview of Open Proposals for MPI 3.1 and Beyond
 * Extensions to requests: callbacks and timers
 * Changes to the semantics of MPI Init and MPI Finalize
 * MPI Piggybacking
 * Extended MPI Pcontrol functionality
 * MPI handle debugging
 * (this list will be adjusted based on the March and May MPI Forum meetings)
17:50-18:00 Schulz Conclusions and Wrap Up

serving:© Torsten Hoefler