Skip to content

Commit

Permalink
update readme.md.
Browse files Browse the repository at this point in the history
  • Loading branch information
b4rtaz committed Jun 29, 2024
1 parent 56b4060 commit 4c38a2e
Showing 1 changed file with 5 additions and 5 deletions.
10 changes: 5 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,17 +31,17 @@ Supported architectures: Llama, Mixtral, Grok
### 🚧 Known Limitations

* You can run Distributed Llama only on 1, 2, 4... 2^n nodes.
* The maximum number of nodes is equal to the number of KV heads in the model [#70](https://github.com/b4rtaz/distributed-llama/issues/70).
* Optimized for (weights format × buffer format):
* The maximum number of nodes is equal to the number of KV heads in the model [#70](https://github.com/b4rtaz/distributed-llama/issues/70).
* CPU support only, GPU support is planned, optimized for (weights format × buffer format):
* ARM CPUs
* ✅ F32 × F32
* ❌ F16 × F32
* Q40 × F32
* Q40 × F32
* ✅ Q40 × Q80
* x86_64 AVX2 CPUs
* F32 × F32
* F32 × F32
* ❌ F16 × F32
* Q40 × F32
* Q40 × F32
* ✅ Q40 × Q80

### 👷 Architecture
Expand Down

0 comments on commit 4c38a2e

Please sign in to comment.