Home

CUTLASS is a collection of CUDA C++ template abstractions for implementing high-performance matrix-multiplication (GEMM) and related computations at all levels and scales within CUDA. It incorporates strategies for hierarchical decomposition and data movement similar to those used to implement cuBLAS and cuDNN. CUTLASS decomposes these "moving parts" into reusable, modular software components abstracted by C++ template classes. These thread-wide, warp-wide, block-wide, and device-wide primitives can be specialized and tuned via custom tiling sizes, data types, and other algorithmic policy. The resulting flexibility simplifies their use as building blocks within custom kernels and applications.

To support a wide variety of applications, CUTLASS provides extensive support for mixed-precision computations, providing specialized data-movement and multiply-accumulate abstractions for half-precision floating point (FP16), BFloat16 (BF16), Tensor Float 32 (TF32), single-precision floating point (FP32), double-precision floating point (FP64) types, integer data types (4b and 8b), and binary data types (1b). CUTLASS demonstrates warp-synchronous matrix multiply operations targeting the programmable, high-throughput Tensor Cores implemented by NVIDIA's Volta, Turing, and Ampere architectures.

CUTLASS implements high-performance Convolution via the implicit GEMM algorithm. Implicit GEMM is the formulation of a convolution operation as a GEMM thereby taking advantage of CUTLASS's modular GEMM pipeline. This allows CUTLASS to build convolutions by reusing highly optimized warp-wide GEMM components and below.

See the Quick Start Guide to get started quickly.

See the functionality listing for the list of operations supported at each level of the execution model hierarchy.

What's New in CUTLASS 2.9

CUTLASS 2.9 is an update to CUTLASS adding:

• First layer Convolution kernels specialized for small channel counts and reduced alignment
• BLAS3 operators accelerated by Tensor Cores
  • SYRK, HERK,
  • SYR2K, HER2K,
  • Out-of-place TRMM, and
  • SYMM, HEMM
• CUTLASS Python demonstrating JIT compilation of CUTLASS kernels and a Python-based runtime using CUDA Python
• GEMM + Softmax example
• Gather and Scatter Fusion with GEMM can gather inputs and scatters outputs based on indices vectors in the same GEMM kernel.
• Back-to-back GEMM/CONV fully supports buffering the previous GEMM/CONV results in the shared memory for the latter one to use.
• Transposed Convolution (a.k.a Deconvolution) support which reuses Dgrad implementation.
• Utility functions that can pad NHWC and convert between NCHW and NHWC.
• Small alignment implicit gemm support for Fprop/Dgrad/Wgrad so that padding is no longer mandated to use tensor cores.
• Epilogue enhancement with performance improvement, more activation functions, and more fusion patterns.
• Optimal performance using CUDA 11.6u2
• Parallel GEMM splitk support in the CUTLASS profiler.
• Updates and bugfixes from the community (thanks!)
• Deprecation announcement: CUTLASS plans to deprecate the following:
  • Maxwell and Pascal GPU architectures
  • Ubuntu 16.04
  • CUDA 10.2

See the CHANGELOG for a detailed listing of releases and updates.