[PyTorch] Bugfix for wgrad bulk overlap conflict when dgrad overlap is reduce-scatter#1341

Open

denera wants to merge 9 commits intoNVIDIA:mainfrom denera:rs-dgrad-overlap-bugfix

+153-35

Commits on Dec 17, 2024

fixed bug where the bulk wgrad config was not being disabled when rs dgrad overlap is enabled
denera
committed
[pre-commit.ci] auto fixes from pre-commit.com hooks

pre-commit-ci[bot]
authored and
denera
committed
Fix linter warning

timmoon10
authored and
denera
committed
fixed incorrect kwarg name in TP overlap test
denera
committed
fixed UB config reference before assignment and corrected FP8 UB buffer logic
denera
committed
added RS DGRAD option to comm+GEMM overlap layer tester
denera
committed
[pre-commit.ci] auto fixes from pre-commit.com hooks

pre-commit-ci[bot]
authored and
denera
committed
[pre-commit.ci] auto fixes from pre-commit.com hooks

pre-commit-ci[bot]
authored and
denera
committed
fixed GEMM dimensions based on operand transpose flags in TP overlap
denera
committed