Understanding PyTorch Buffers understanding-buffers.ipynb explains the idea behind PyTorch buffers, which are used to implement the causal attention mechanism in chapter 3 Below is a hands-on video tutorial I recorded to explain the code: