Skip to content

andmev/llama-to-coreml

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

10 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Llama 3.x CoreML Converter

This project converts the Llama 3.x model to CoreML format for deployment on Apple Silicon devices.

Requirements

  • Python 3.8+
  • PyTorch 2.0+
  • macOS 15+ (Sequoia)
  • Apple Silicon Mac

Installation

mise install
pip install -e .
pip install -r requirements.txt

mise is a tool for managing Python versions and virtual environments.

Model Conversion and Compilation

This tool provides two main scripts for working with CoreML models:

Converting Models

To convert a Llama model to CoreML format:

python scripts/convert_model.py \
    --model-path meta-llama/Llama-3.2-3B-Instruct \
    --output-path output/Llama-3.2-3B-Instruct.mlpackage \
    --token your_token_here \
    --compile  # Optional: add this flag to compile after conversion

The --compile flag will automatically compile the model to .mlmodelc format after conversion, creating it in the same directory as the .mlpackage file.

Compiling Models

To compile a CoreML model to .mlmodelc format:

python scripts/compile_model.py \
    --model-path output/Llama-3.2-3B-Instruct.mlpackage \
    --output-path output/Llama-3.2-3B-Instruct.mlmodelc

Notes

  • The model should be downloaded from HuggingFace.
  • The token is optional, but it's required for gated models.
  • This project based on this article

License

This project is licensed under the MIT License. See the LICENSE file for details.

About

Converts models from HuggingFace to CoreML format.

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages