Skip to content

Commit 5c64172

Browse files
committed
chore: fix lint issues.
1 parent d733427 commit 5c64172

File tree

1 file changed

+9
-0
lines changed

1 file changed

+9
-0
lines changed

nemo/core/optim/optimizers.py

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -203,6 +203,15 @@ def get_optimizer(name: str, **kwargs: Optional[Dict[str, Any]]) -> Optimizer:
203203

204204

205205
def init_optimizer_states(optimizer: Optimizer):
206+
"""
207+
Initialize optimizer states for Adam-based optimizers.
208+
209+
This function initializes the exponential moving averages (exp_avg and exp_avg_sq)
210+
for Adam, AdamW, and FusedAdam optimizers if they haven't been initialized yet.
211+
212+
Args:
213+
optimizer: The optimizer instance to initialize states for
214+
"""
206215
adam_nondist_optims = (optim.Adam, optim.AdamW)
207216
if HAVE_APEX:
208217
adam_nondist_optims += (FusedAdam,)

0 commit comments

Comments
 (0)