Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Enable Gradient Accumulation fix across all models + trainer fully in forward() #34283

Open
wants to merge 8 commits into
base: main
Choose a base branch
from
Open

Enable Gradient Accumulation fix across all models + trainer fully in forward() #34283

wants to merge 8 commits into from
+72 −22

Commits on Oct 22, 2024

  1. Enable grad accum fix across all models + trainer fully in forward()

    @muellerzr
    muellerzr committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    0bfce6e View commit details
    Browse the repository at this point in the history

  2. handle peft case

    @muellerzr
    muellerzr committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    8c12bf4 View commit details
    Browse the repository at this point in the history

  3. Account for DDP: need to run scale tests

    @muellerzr
    muellerzr committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    058fe34 View commit details
    Browse the repository at this point in the history

  4. Use accelerator state

    @muellerzr
    muellerzr committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    fc6d674 View commit details
    Browse the repository at this point in the history

  5. Quality

    @muellerzr
    muellerzr committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    0aeb5ac View commit details
    Browse the repository at this point in the history

  6. Guard

    @muellerzr
    muellerzr committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    49b29d2 View commit details
    Browse the repository at this point in the history

  7. Experiment w/ only fairseq fix

    @muellerzr
    muellerzr committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    4f3f86d View commit details
    Browse the repository at this point in the history

  8. Fairseq only

    @muellerzr
    muellerzr committed Oct 22, 2024
    Configuration menu
    Copy the full SHA
    58ee680 View commit details
    Browse the repository at this point in the history