When will megatron Flash attention 3 be supported？ #1099
Unanswered
echo-valor
asked this question in
Q&A
Replies: 1 comment
-
|
Hi, any progress about this? |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
Your question
Flash attention 3 has been updated, link:
Megatron's support for Flash attention 3 to improve training efficiency。FlashAttention-3 is optimized for Hopper GPUs (e.g. H100).
Beta Was this translation helpful? Give feedback.
All reactions