Transformers from scratch with python
This repository contains a Python implementation of the Transformer model, as described in the paper "Attention is All You Need" by Vaswani et al. (2017). The implementation focuses on the core concepts of the Transformer architecture, including embedding layers, positional encoding, and the unique way the model processes sequential data without relying on recurrent layers.
This project aims to provide a clear and concise implementation of the Transformer model, suitable for educational purposes and for those who want to get a deeper understanding of how Transformers work internally. The code includes detailed comments to help users follow along with the Transformer's mechanisms such as self-attention and positional encoding.
- Embedding layer implementation
- Positional Encoding with sine and cosine calculations
- Simple forward method for demonstration purposes
To get started with this project, you will need to clone the repository and install the required dependencies. Follow the steps below:
git clone https://github.com/ahmet-f-gumustas/Transformers-From-Scratch.git
cd transformer-implementation
pip install -r requirements.txt