Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Ring attention #181

Open
wants to merge 21 commits into
base: main
Choose a base branch
from
Open

Ring attention #181

wants to merge 21 commits into from

Commits on May 23, 2024

  1. first commit for ring attention

    @zzhhjjj
    zzhhjjj committed May 23, 2024
    Configuration menu
    Copy the full SHA
    471e91b View commit details
    Browse the repository at this point in the history

Commits on Jun 11, 2024

  1. still testing

    @zzhhjjj
    zzhhjjj committed Jun 11, 2024
    Configuration menu
    Copy the full SHA
    111c4fc View commit details
    Browse the repository at this point in the history

Commits on Jun 30, 2024

  1. refactor some code

    @zzhhjjj
    zzhhjjj committed Jun 30, 2024
    Configuration menu
    Copy the full SHA
    3ef191d View commit details
    Browse the repository at this point in the history

  2. add SP to the tests

    @zzhhjjj
    zzhhjjj committed Jun 30, 2024
    Configuration menu
    Copy the full SHA
    019d263 View commit details
    Browse the repository at this point in the history

Commits on Jul 2, 2024

  1. ddp process group for backwawrd

    @zzhhjjj
    zzhhjjj committed Jul 2, 2024
    Configuration menu
    Copy the full SHA
    07ebf2c View commit details
    Browse the repository at this point in the history

  2. Merge branch 'main' into ring_attention

    @zzhhjjj
    zzhhjjj committed Jul 2, 2024
    Configuration menu
    Copy the full SHA
    beefa30 View commit details
    Browse the repository at this point in the history

  3. clean code

    @zzhhjjj
    zzhhjjj committed Jul 2, 2024
    Configuration menu
    Copy the full SHA
    3f504b3 View commit details
    Browse the repository at this point in the history

  4. update tests to add SP process group

    @zzhhjjj
    zzhhjjj committed Jul 2, 2024
    Configuration menu
    Copy the full SHA
    3f841ba View commit details
    Browse the repository at this point in the history

  5. clean code

    @zzhhjjj
    zzhhjjj committed Jul 2, 2024
    Configuration menu
    Copy the full SHA
    9208928 View commit details
    Browse the repository at this point in the history

Commits on Jul 18, 2024

  1. refactor code

    @zzhhjjj
    zzhhjjj committed Jul 18, 2024
    Configuration menu
    Copy the full SHA
    dbb8eb7 View commit details
    Browse the repository at this point in the history

  2. Merge remote-tracking branch 'upstream/main' into ring_attention

    @zzhhjjj
    zzhhjjj committed Jul 18, 2024
    Configuration menu
    Copy the full SHA
    e750a8f View commit details
    Browse the repository at this point in the history

  3. fix tests

    @zzhhjjj
    zzhhjjj committed Jul 18, 2024
    Configuration menu
    Copy the full SHA
    16600f0 View commit details
    Browse the repository at this point in the history

Commits on Jul 19, 2024

  1. fix tests

    @zzhhjjj
    zzhhjjj committed Jul 19, 2024
    Configuration menu
    Copy the full SHA
    175a076 View commit details
    Browse the repository at this point in the history

  2. another test

    @zzhhjjj
    zzhhjjj committed Jul 19, 2024
    Configuration menu
    Copy the full SHA
    61486e7 View commit details
    Browse the repository at this point in the history

Commits on Jul 21, 2024

  1. import flash attention

    @zzhhjjj
    zzhhjjj committed Jul 21, 2024
    Configuration menu
    Copy the full SHA
    d5e8332 View commit details
    Browse the repository at this point in the history

  2. add readme for training

    @zzhhjjj
    zzhhjjj committed Jul 21, 2024
    Configuration menu
    Copy the full SHA
    2a82f5c View commit details
    Browse the repository at this point in the history

  3. correct link

    @zzhhjjj
    zzhhjjj committed Jul 21, 2024
    Configuration menu
    Copy the full SHA
    87649df View commit details
    Browse the repository at this point in the history

Commits on Jul 22, 2024

  1. evaluation script

    @zzhhjjj
    zzhhjjj committed Jul 22, 2024
    Configuration menu
    Copy the full SHA
    c6bbeda View commit details
    Browse the repository at this point in the history

  2. change foler location

    @zzhhjjj
    zzhhjjj committed Jul 22, 2024
    Configuration menu
    Copy the full SHA
    72d9488 View commit details
    Browse the repository at this point in the history

  3. update Readme

    @zzhhjjj
    zzhhjjj committed Jul 22, 2024
    Configuration menu
    Copy the full SHA
    5907339 View commit details
    Browse the repository at this point in the history

  4. update README

    @zzhhjjj
    zzhhjjj committed Jul 22, 2024
    Configuration menu
    Copy the full SHA
    c81601a View commit details
    Browse the repository at this point in the history