Code

Add moe pretrain#49

Code

Open

Taishi-N324 wants to merge 8 commits intomainllm-jp/scripts:mainfrom add-moe-pretrainllm-jp/scripts:add-moe-pretrainCopy head branch name to clipboard

+1,771

Commits

Commits on Oct 14, 2024

• Add v3-moe-recipes-sakura

  Taishi-N324

  committed
  1a7cf42

  Copy full SHA for 1a7cf42

• Fixing seed and adding explanations for environmental variables

  Taishi-N324

  committed
  193f5c4

  Copy full SHA for 193f5c4

• Change path

  Taishi-N324

  committed
  3ae1fdc

  Copy full SHA for 3ae1fdc

Commits on Oct 15, 2024

• Fixes around paths

  Taishi-N324

  committed
  0db7c13

  Copy full SHA for 0db7c13

Commits on Oct 16, 2024

• Add v3-8x1.8b-exp1-sakura scripts

  Taishi-N324

  committed
  c281feb

  Copy full SHA for c281feb

Commits on Oct 22, 2024

• WIP

  Taishi-N324

  committed
  73248b9

  Copy full SHA for 73248b9

• fix name

  Taishi-N324

  committed
  1cf5898

  Copy full SHA for 1cf5898

Commits on Oct 31, 2024

• Add v3-8x1.8b-exp2

  Taishi-N324

  committed
  e59f487

  Copy full SHA for e59f487