Code

Allgather with DID loop split#3284

Code

Merged

wujingyue merged 23 commits intomainNVIDIA/Fuser:mainfrom wjy/commNVIDIA/Fuser:wjy/commCopy head branch name to clipboardDec 9, 2024

+326-141

Commits

Commits on Nov 26, 2024

• Add a repro for #3282

  wujingyue

  committed
  04e06a8

• Remove an assumption in the transpose scheduler.

  wujingyue

  committed
  416f1d0

• Reimplement unshardSizesAndStrides.

  wujingyue

  committed
  2c984c8

• Inherit parallel type for new allocation IDs

  wujingyue

  committed
  44b5091

• Fix broadcast tests

  wujingyue

  committed
  521c783

• Unify unshardedSizes.

  wujingyue

  committed
  9ff10cf

• Fix a test

  wujingyue

  committed
  8bd9486

• Refine the logic in the transpose scheduler

  wujingyue

  committed
  5a16349

Commits on Nov 27, 2024

• Comment

  wujingyue

  committed
  2768970

• Merge branch 'main' into wjy/forward

  wujingyue

  committed
  e38962e

• Resolve two fixmes

  wujingyue

  committed
  400684e

• Add a repro.

  wujingyue

  committed
  3086237

• Generalize postAllgather to support DID loop split.

  wujingyue

  committed
  fe0cec6

Commits on Nov 28, 2024

• Try to reuse getShardedAxis

  wujingyue

  committed
  5d60dd5

• Fix lint

  wujingyue

  committed
  7ccfccd

Commits on Nov 30, 2024

• Merge remote-tracking branch 'origin/main' into wjy/comm

  wujingyue

  committed
  39f2809

Commits on Dec 4, 2024

• Disallow DID on inner splits

  wujingyue

  committed
  7cf2384

Commits on Dec 5, 2024

• Add a test for noncontiguous allgather

  wujingyue

  committed
  c27a585

• Comment

  wujingyue

  committed
  c218c70

• Rename

  wujingyue

  committed
  9c2a218

Commits on Dec 6, 2024

• Comment

  wujingyue

  committed
  5229512

• More comments

  wujingyue

  committed
  5157fe1

Commits on Dec 9, 2024

• Merge branch 'main' into wjy/comm

  wujingyue

  authored
  6ad52dd