Skip to content

Commit

Permalink
Update Adam.jl
Browse files Browse the repository at this point in the history
  • Loading branch information
pkofod authored Jan 29, 2024
1 parent 6fd621a commit 36f49e7
Showing 1 changed file with 1 addition and 0 deletions.
1 change: 1 addition & 0 deletions src/optimize/acceleration/Adam.jl
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,7 @@ function solve(problem::OptimizationProblem, x0, adam::Adam, options::Optimizati
# m̂ = m./(1-β₁^t)
# v̂ = v./(1-β₂^t)
# x = x .- α*m̂/(sqrt.(v̂+ϵ))
# This appears to only be approximately true
αₜ = α * sqrt(1 - β₂^iter) / (1 - β₁^iter)
z = z .- αₜ .* m ./ (sqrt.(v) .+ ϵ)

Expand Down

0 comments on commit 36f49e7

Please sign in to comment.