Anthony Chen

Anthony Chen

A picture of me at Heisler Beach, November 2023

Posts by Collection

portfolio

publications

Communication bounds for convolutional neural networks

Published in PASC '22: Proceedings of the Platform for Advanced Scientific Computing Conference, 2022

Convolutional neural networks (CNNs) are important in a wide variety of machine learning tasks and applications, so optimizing their performance is essential.

Recommended citation: Anthony Chen, James Demmel, Grace Dinh, Mason Haberle, and Olga Holtz. 2022. Communication bounds for convolutional neural networks. In Proceedings of the Platform for Advanced Scientific Computing Conference (PASC '22). Association for Computing Machinery, New York, NY, USA, Article 1, 1–10. https://dl.acm.org/doi/10.1145/3539781.3539784

talks

Communication Bounds for Convolutional Neural Networks

Published:

Convolutional neural networks (CNNs) are important in a wide variety of machine learning tasks and applications, so optimizing their performance is essential. Moving words of data between levels of a memory hierarchy or between processors on a network is much more expensive than the cost of arithmetic, so minimizing communication is critical to optimizing performance. In this paper, we present new precise lower bounds on data movement for convolutions in both single-processor and parallel distributed memory models, as well as algorithms that outperform current implementations such as Im2Col. We obtain performance figures using GEMMINI, a machine learning accelerator, where our tiling provides improvements between 13% and 150% over a vendor supplied algorithm.

Communication Bounds for Convolutional Neural Networks

Published:

Convolutional neural networks (CNNs) are important in a wide variety of machine learning tasks and applications, so optimizing their performance is essential. Moving words of data between levels of a memory hierarchy or between processors on a network is much more expensive than the cost of arithmetic, so minimizing communication is critical to optimizing performance. In this paper, we present new precise lower bounds on data movement for convolutions in both single-processor and parallel distributed memory models, as well as algorithms that outperform current implementations such as Im2Col. We obtain performance figures using GEMMINI, a machine learning accelerator, where our tiling provides improvements between 13% and 150% over a vendor supplied algorithm.

Numerical Methods for Vorticity Dynamics on a Sphere

Published:

Fast summation techniques have proven to be of great importance in a variety of fields. In this talk, I will present a new technique for performing fast summations on spheres, which is suitable for sums coming from spherical convolutions. I present applications of this technique to problems coming from atmospheric and oceanic modeling.

Fast Summation on a Sphere

Published:

Fast summation techniques have proven to be of great importance in a variety of fields. In this talk, I will present a new technique for performing fast summations on spheres, which is suitable for sums coming from spherical convolutions. I present applications of this technique to problems coming from atmospheric and oceanic modeling.

Fast Summation for the Barotropic Vorticity Equations

Published:

The barotropic vorticity equations describe the conservation of absolute vorticity in a rotating fluid. When transformed appropriately, one can discretize these equations as a N-body problem. In this talk, I present a fast summation technique to accelerate the computation of the solution with a spherical tree code.

Fast Summation for the Barotropic Vorticity Equations

Published:

The barotropic vorticity equations describe the conservation of absolute vorticity for a fluid on a rotating sphere. When transformed appropriately, one can rewrite these with a Biot-Savart integral, and with a Lagrangian discretization, one can arrive at a discretized system with an update taking the form of a N-body sum. In this talk, I present a fast summation technique that reduces the asymptotic complexity of this sum from $O(N^2)$ to $O(N\log{N})$ with a new spherical tree code, suitable for a wide range of problems in geophysical fluid dynamics.

Fast Summation for Spherical Convolutions

Published:

Convolutions on the sphere are important in many aspects of the geosciences, and many unstructured numerical methods make heavy use of them. However, when discretized, these convolutions result in sums that scale as $O(N^2)$ in the number of grid points, which is challenging for scaling to large problem sizes. In this talk, I will present a fast summation technique that allows us to approximate such sums while reducing the complexity from $O(N^2)$ to $O(N\log{N})$ with a spherical tree code that is suitable for a wide range of problems in the geosciences.

Fast Convolutions for Geophysical Fluid Dynamics

Published:

Convolutions on the sphere are useful in many Geophysical Fluid Dynamics (GFD) problems, and many unstructured numerical methods make heavy use of them. However, when discretized, these convolutions result in sums that scale as $O(N^2)$ in the number of grid points, which is challenging for scaling to large problem sizes. This work will discuss a fast summation technique that allows us to approximate such sums while reducing the complexity from $O(N^2)$ to $O(N\log N)$. It uses a spherical tree code that is suitable for a wide range of problems that arise in geophysical fluid dynamics.

Exploring a Convolution-Based Self Attraction and Loading Technique for the Ocean Model MOM6

Published:

Self Attraction and Loading (SAL) is an important term in ocean models. The importance can be most clearly seen in the impact on tides, where failure to include SAL results in significant amplitude and phase errors. Traditionally, SAL has been computed using either a scalar approximation or a spherical harmonic transform, but both of these approaches have significant drawbacks. In this work, we discuss an alternative approach based on a spherical convolution, which is further accelerated with a tree code. To test this approach, we implement this computation in MOM6, the Modular Ocean Model, developed by NOAA’s Geophysical Fluid Dynamics Laboratory, and the ocean component in various weather and climate models. We demonstrate improved accuracy in tidal simulations, and also present some results for regional ocean modeling.

Convolution Based Self Attraction and Loading in MOM6

Published:

Self Attraction and Loading (SAL) is an important term in ocean models. The importance can be most clearly seen in the impact on tides, where failure to include SAL results in significant amplitude and phase errors. Traditionally, SAL has been computed using either a scalar approximation or a spherical harmonic transform, but both of these approaches have significant drawbacks. In this work, we discuss an alternative approach based on a spherical convolution, which is further accelerated with a tree code. To test this approach, we implement this computation in MOM6, the Modular Ocean Model, developed by NOAA’s Geophysical Fluid Dynamics Laboratory, and the ocean component in various weather and climate models. We demonstrate improved accuracy in tidal simulations, and also present some results for regional ocean modeling.

teaching