Skip to content

Issues: sail-sg/zero-bubble-pipeline-parallelism

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

18 Open 9 Closed
18 Open 9 Closed
Author
Filter by author
Loading
Label
Filter by label
Loading
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Loading
Milestones
Filter by milestone
Loading
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

[QUESTION] Question about ZeroBubbleVPipeScheduler schedule W
#38 opened Aug 1, 2024 by duanjunwen
1
[QUESTION] I used the zero-bubble commit 7ad9c81d for my experiments, and found that the memory usage of this zb-v model exceeds that of the previous zb1 model. What could be the issue? The specific configuration and results are shown in the image.
#36 opened Jul 18, 2024 by lbk-sys
4
[ENHANCEMENT] Support selective B-W Split for all schedules
#32 opened Jul 3, 2024 by ufotalent
@huanggx-sea
1
[ENHANCEMENT] Opensourcing of 1F1B-V and the variant of interleaved 1F1B and benchmark
#31 opened Jul 3, 2024 by ufotalent
@huanggx-sea
1
[ENHANCEMENT] Runtime and scheduler refactor so that we support uniform scheduler API
#30 opened Jul 3, 2024 by ufotalent
@huanggx-sea
1
[ENHANCEMENT] Sync code base to newest megatron
#29 opened Jul 3, 2024 by ufotalent
@QPHutu
[ENHANCEMENT] Adaptive V scheduler
#28 opened Jul 3, 2024 by ufotalent
@QPHutu
[ENHANCEMENT] Support V-Min and V-Half schedules
#27 opened Jul 3, 2024 by ufotalent
@ufotalent
[QUESTION] Measuring Pipeline Bubble Time During Megatron-LM Training
#25 opened Jun 14, 2024 by HodBadichi
1
[QUESTION]1f1b is fast then zero-v
#24 opened Jun 13, 2024 by kuangdao
[QUESTION] The timing for B and W appears to be incorrect
#22 opened May 15, 2024 by RookieHong
2
interleaved 1F1B seems to work better
#21 opened May 14, 2024 by zhj96
3
[QUESTION] May I ask what tool was used to plot Figure 6 in paper.How can I profile bubble time in pipeline parallelism?
#18 opened Apr 18, 2024 by starstream
3
[QUESTION] Whether to split bw when send_backward_recv_forward is not enabled
#17 opened Apr 2, 2024 by AndSonder
4
Support sequence parallel on main branch
#13 opened Dec 26, 2023 by ufotalent
1
Survey on implementing simple version in other codebases besides megatron-lm
#12 opened Dec 26, 2023 by ufotalent
Create a miniversion containing only ZB-H1 and essential changes so other megatron forks can easily integrate
#10 opened Dec 15, 2023 by ufotalent
5
More general ZBV scheduling
#9 opened Dec 12, 2023 by mavenlin
1
ProTip! Follow long discussions with comments:>50.