Programming Transformer Neural Networks with PyTorch – Udacity

Welcome to the Programming Transformer Neural Networks with PyTorch repository! This repository is a companion to Udacity's AI Programming with Python Nanodegree program, specifically focusing on Transformer Neural Networks and their implementation using PyTorch.

Table of Contents

• Programming Transformer Neural Networks with PyTorch – Udacity
  • Table of Contents
  • Course Overview
  • Repository Structure
  • Getting Started
  • Contributing
  • License

Course Overview

This repository supports Udacity's Programming Transformer Neural Networks with PyTorch course, which is part of the AI Programming with Python Nanodegree.

In this course, you will learn:

1. Introduction to Transformer Neural Networks

   • Explore Transformer architecture and its applications, including ChatGPT.
   • Learn about NLP fundamentals such as tokenization and model training using PyTorch.

2. Building Transformer Neural Networks with PyTorch

   • Implement a Transformer model using PyTorch.
   • Understand key components such as tokenization, embeddings, multi-head attention, training, and text generation.

3. Using Pre-Trained Transformers

   • Learn how to fine-tune pre-trained Transformer models.
   • Explore their advantages, limitations, and applications to NLP tasks like text generation and question-answering.

By the end of this course, you will have a solid understanding of how to build, train, and utilize Transformer-based models effectively in real-world applications.

Repository Structure

The repository is organized as follows:

• lesson-1-introduction-to-transformer-neural-networks/ – Contains materials and exercises for Lesson 1, introducing Transformer Neural Networks.
• .github/workflows/ – GitHub Actions workflows for CI/CD processes.
• CODEOWNERS – Specifies contributors responsible for maintaining the repository.
• LICENSE – The license governing the use of this repository.
• README.md – This file, providing an overview of the repository.

Getting Started

To start working with this repository:

1. Clone the repository:

   git clone https://github.com/ruddyscent/ai-programming-python-transformers.git
   cd ai-programming-python-transformers

2. Navigate to the lesson materials:

   cd lesson-1-introduction-to-transformer-neural-networks

3. Follow the instructions in the lesson materials to set up your environment and begin coding.

Contributing

We welcome contributions to enhance the learning experience:

• Fork the repository and create a new branch for your feature or bug fix.
• Submit a pull request with a clear description of your changes.
• Ensure your code follows the repository’s guidelines and passes all tests.

License

This project is licensed under the terms specified in the LICENSE file. Please review the license before using or contributing to this repository.