Skip to content

AlbertoSinigaglia/block-coordinates-gradient-descent-algorithms

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

2 Commits
 
 
 
 
 
 
 
 

Repository files navigation

Analysis of Gradient Descent and BCGD methods

This repository contains the implementation of the Optimization for DataScience @ UniPD first homework, about the efficiency of block coordinates gradient descent methods.

Introduction

Gradient descent is a method to find the minimum of a function, however during the course multiple version of it are being shown.
In this analysis, multiple version will be compared, to check the convergence rate of them both in time and in iterations.

Implementation

In order to implement the project, it's been created a Python Jupyter notebook, containing all the code to recreate any result reported.
Everything can be found inside the project folder

Report

The essay contains the details of the homework, all the calculations needed to develop the project, and all the explanation on why things are being chosen to be done in this way.

Animations

In order to visually see what the algorithms actually do, in the notebook there is the code to generate some GIF at each iteration of all the methods, and the HTML file given is supposed to help to see the difference between them.

Following are some examples:

Current label Current classification Current loss
Gradient descent Gradient descent Gradient descent
Gradient descent improved Gradient descent improved Gradient descent improved
Gradient descent heavy ball Gradient descent heavy ball Gradient descent heavy ball
Gradient descent accelerated Gradient descent accelerated Gradient descent accelerated
Gradient descent exact line search Gradient descent exact line search Gradient descent exact line search
BCGM Cyclic BCGM Cyclic BCGM Cyclic
BCGM Random BCGM Random BCGM Random
BCGM Gauss-Southwell BCGM Gauss-Southwell BCGM Gauss-Southwell
BCGM Gauss-Southwell exact line search BCGM Gauss-Southwell exact line search BCGM Gauss-Southwell exact line search

If you clone/download the repository and open the README file, you will be able to compare the animations since they will be synchronized, this might not happen if you are watching this from Github

About

No description, website, or topics provided.

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published