Deep Learning

This repository contains the backpropagation implementation for a simple feed-forward network, with a cross-entropy loss function. It is entirely implemented from scratch, using only basic python primitives. Then, the forward and backward passes are vectorized with numpy and the results are compared.

Experimental Setup

In order to run the experiments you need to have installed Python, matplolib and Numpy. The following experiment can be run for digit recognotion with MNIST:

• Mini-batch non-vectorized
• Gradient Descent non-vectorized
• One-pass forward-backward
• Synthetic data training with SGD