Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add some Bigcode model #260

Merged
merged 18 commits into from
Jul 28, 2023
Merged

Add some Bigcode model #260

merged 18 commits into from
Jul 28, 2023

Commits on Jul 27, 2023

  1. Start sketching the bigcode gpt model.

    @LaurentMazare
    LaurentMazare committed Jul 27, 2023
    Configuration menu
    Copy the full SHA
    571493f View commit details
    Browse the repository at this point in the history

  2. Sketch the bigcode model.

    @LaurentMazare
    LaurentMazare committed Jul 27, 2023
    Configuration menu
    Copy the full SHA
    db20b45 View commit details
    Browse the repository at this point in the history

  3. Implement the attention mechanism.

    @LaurentMazare
    LaurentMazare committed Jul 27, 2023
    Configuration menu
    Copy the full SHA
    24060ee View commit details
    Browse the repository at this point in the history

  4. Random reshaping.

    @LaurentMazare
    LaurentMazare committed Jul 27, 2023
    Configuration menu
    Copy the full SHA
    d4cb6b6 View commit details
    Browse the repository at this point in the history

  5. Sketch more of the example.

    @LaurentMazare
    LaurentMazare committed Jul 27, 2023
    Configuration menu
    Copy the full SHA
    3a245b1 View commit details
    Browse the repository at this point in the history

  6. Add some kv cache.

    @LaurentMazare
    LaurentMazare committed Jul 27, 2023
    Configuration menu
    Copy the full SHA
    b4a8651 View commit details
    Browse the repository at this point in the history

  7. Properly generate the position ids.

    @LaurentMazare
    LaurentMazare committed Jul 27, 2023
    Configuration menu
    Copy the full SHA
    5358449 View commit details
    Browse the repository at this point in the history

  8. Proper attention mask.

    @LaurentMazare
    LaurentMazare committed Jul 27, 2023
    Configuration menu
    Copy the full SHA
    ab0965a View commit details
    Browse the repository at this point in the history

  9. Bail on upcasting.

    @LaurentMazare
    LaurentMazare committed Jul 27, 2023
    Configuration menu
    Copy the full SHA
    d220da5 View commit details
    Browse the repository at this point in the history

  10. Properly apply the attention mask.

    @LaurentMazare
    LaurentMazare committed Jul 27, 2023
    Configuration menu
    Copy the full SHA
    3612217 View commit details
    Browse the repository at this point in the history

Commits on Jul 28, 2023

  1. Add the smaller starcoder variants.

    @LaurentMazare
    LaurentMazare committed Jul 28, 2023
    Configuration menu
    Copy the full SHA
    872d864 View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'origin/main' into bigcode

    @LaurentMazare
    LaurentMazare committed Jul 28, 2023
    Configuration menu
    Copy the full SHA
    2bfb250 View commit details
    Browse the repository at this point in the history

  3. Update for the new hub api.

    @LaurentMazare
    LaurentMazare committed Jul 28, 2023
    Configuration menu
    Copy the full SHA
    8531c5a View commit details
    Browse the repository at this point in the history

  4. Fix a shape issue.

    @LaurentMazare
    LaurentMazare committed Jul 28, 2023
    Configuration menu
    Copy the full SHA
    d4079fa View commit details
    Browse the repository at this point in the history

  5. Fix another shape issue.

    @LaurentMazare
    LaurentMazare committed Jul 28, 2023
    Configuration menu
    Copy the full SHA
    aad831c View commit details
    Browse the repository at this point in the history

  6. Get some logits out.

    @LaurentMazare
    LaurentMazare committed Jul 28, 2023
    Configuration menu
    Copy the full SHA
    d8c9109 View commit details
    Browse the repository at this point in the history

  7. Merge remote-tracking branch 'origin/main' into bigcode

    @LaurentMazare
    LaurentMazare committed Jul 28, 2023
    Configuration menu
    Copy the full SHA
    b911ec5 View commit details
    Browse the repository at this point in the history

  8. Adjust the weigth names.

    @LaurentMazare
    LaurentMazare committed Jul 28, 2023
    Configuration menu
    Copy the full SHA
    8fdc240 View commit details
    Browse the repository at this point in the history