Skip to content

Commit

Permalink
Fix rendering issue in getting sterted (#21)
Browse files Browse the repository at this point in the history 
Signed-off-by: Kelly Brown <[email protected]>
  • Loading branch information
@kelbrown20
kelbrown20 authored Nov 21, 2024
1 parent 85bc66e commit 7e8d3fc
Show file tree
Hide file tree
Showing 6 changed files with 217 additions and 441 deletions.
42 changes: 30 additions & 12 deletions docs/getting-started/download_models.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,30 +6,46 @@ logo: images/ilab_dog.png

# 📥 Download the model

- Run the `ilab model download` command.
1) Run the ilab model download command to download a compact pre-trained version of the `granite-7b-lab-GGUF`, `merlinite-7b-lab-GGUF`, and `Mistral-7B-Instruct-v0.2-GGUF` models (~4.4G each) from HuggingFace.

```shell
ilab model download
```

`ilab model download` downloads a compact pre-trained version of the [model](https://huggingface.co/instructlab/) (~4.4G) from HuggingFace:

*Example output of the models downloading*

```shell
Downloading model from Hugging Face:
Model: instructlab/granite-7b-lab-GGUF@main
Destination: /Users/<user>/.cache/instructlab/models
Downloading model from Hugging Face:
Model: instructlab/merlinite-7b-lab-GGUF@main
Destination: /Users/<user>/.cache/instructlab/models
Downloading model from Hugging Face:
Model: TheBloke/Mistral-7B-Instruct-v0.2-GGUF@main
Destination: /Users/<user>/.cache/instructlab/models

TheBloke/Mistral-7B-Instruct-v0.2-GGUF requires a HF Token to be set.
Please use '--hf-token' or 'export HF_TOKEN' to download all necessary models.
```

a) You may be prompted to use your Hugging Face token to download the `Mistral-7B-Instruct-v0.2-GGUF` model.

```shell
(venv) $ ilab model download
Downloading model from Hugging Face: instructlab/merlinite-7b-lab-GGUF@main to /Users/USERNAME/Library/Caches/instructlab/models...
...
INFO 2024-08-01 15:05:48,464 huggingface_hub.file_download:1893: Download complete. Moving file to /Users/USERNAME/Library/Caches/instructlab/models/merlinite-7b-lab-Q4_K_M.gguf
ilab model download --hf-token <your-huggingface-token>
```

!!! note
⏳ This command can take few minutes or immediately depending on your internet connection or model is cached. If you have issues connecting to Hugging Face, refer to the [Hugging Face discussion forum](https://discuss.huggingface.co/) for more details.

## Downloading an entire Hugging Face repository (Safetensors Model)

- Specify repository, and a Hugging Face token if necessary. For example:
1) Specify repository, and a Hugging Face token if necessary. For example:

```shell
HF_TOKEN=<YOUR HUGGINGFACE TOKEN GOES HERE> ilab model download --repository=instructlab/granite-7b-lab
ilab model download --repository instructlab/granite-7b-lab-GGUF --filename granite-7b-lab-Q4_K_M.gguf --hf-token <your-huggingface-token>
```

These types of models are useful for GPU-enabled systems or anyone looking to serve a model using vLLM. InstructLab provides Safetensor versions of our Granite models on HuggingFace.
Expand All @@ -46,9 +62,11 @@ ilab model list

```shell
(venv) $ ilab model list
+------------------------------+---------------------+--------+
| Model Name | Last Modified | Size |
+------------------------------+---------------------+--------+
| merlinite-7b-lab-Q4_K_M.gguf | 2024-08-01 15:05:48 | 4.1 GB |
+------------------------------+---------------------+--------+
+-------------------------------------+---------------------+--------+
| Model Name | Last Modified | Size |
+-------------------------------------+---------------------+--------+
| granite-7b-lab-Q4_K_M.gguf | 2024-08-01 15:05:48 | 4.1 GB |
| merlinite-7b-lab-Q4_K_M.gguf | 2024-08-01 15:05:48 | 4.1 GB |
| mistral-7b-instruct-v0.2.Q4_K_M.gguf| 2024-08-01 15:05:48 | 4.1 GB |
+-------------------------------------+---------------------+--------+
```
152 changes: 64 additions & 88 deletions docs/getting-started/initilize_ilab.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -6,112 +6,88 @@ logo: images/ilab_dog.png

# 🏗️ Initialize `ilab`

### 🏗️ Initialize `ilab`
1) Initialize `ilab` by running the following command:

1. Initialize `ilab` by running the following command:

```shell
ilab config init
```
```shell
ilab config init
```

2. When prompted, clone the `https://github.com/instructlab/taxonomy.git` repository into the current directory by typing **enter**
2) When prompted, clone the `https://github.com/instructlab/taxonomy.git` repository into the current directory by typing **enter**

**Optional**: If you want to point to an existing local clone of the `taxonomy` repository, you can pass the path interactively or alternatively with the `--taxonomy-path` flag.

`ilab` will use the default configuration file unless otherwise specified. You can override this behavior with the `--config` parameter for any `ilab` command.

3. When prompted, provide the path to your default model. Otherwise, the default of a quantized [Merlinite](https://huggingface.co/instructlab/merlinite-7b-lab-GGUF) model is used.

*Example output of steps 1 - 3*

```shell
----------------------------------------------------
Welcome to the InstructLab CLI
This guide will help you to setup your environment
----------------------------------------------------

Please provide the following values to initiate the environment [press Enter for defaults]:
Path to taxonomy repo [/Users/kellybrown/.local/share/instructlab/taxonomy]:
Path to your model [/Users/kellybrown/.cache/instructlab/models/merlinite-7b-lab-Q4_K_M.gguf]:
```

You can download this model with `ilab model download` command as well.

4. The InstructLab CLI auto-detects your hardware and select the exact system profile that matches your machine. System profiles populate the `config.yaml` file with the proper parameter values based on your detected GPU types and avaiible vRAM.

*Example output of profile auto-detection*

```shell
Generating config file and profiles:
/home/user/.config/instructlab/config.yaml
/home/user/.local/share/instructlab/internal/train_configuration/profiles

We have detected the AMD CPU profile as an exact match for your system.

--------------------------------------------
Initialization completed successfully!
You're ready to start using `ilab`. Enjoy!
--------------------------------------------
```
5. If there is not an exact match for your system, you can manually select a system profile when prompted. There are various flags you can utilize with individual `ilab` commands that allow you to utilize your GPU if applicable.
*Example output of selecting a system profile*
```shell
Please choose a system profile to use.
System profiles apply to all parts of the config file and set hardware specific defaults for each command.
First, please select the hardware vendor your system falls into
[1] APPLE
[2] INTEL
[3] AMD
[4] NVIDIA
Enter the number of your choice [0]: 1
You selected: APPLE
Next, please select the specific hardware configuration that most closely matches your system.
[0] No system profile
[1] APPLE M1 ULTRA
[2] APPLE M1 MAX
[3] APPLE M2 MAX
[4] APPLE M2 ULTRA
[5] APPLE M2 PRO
[6] APPLE M2
[7] APPLE M3 MAX
[8] APPLE M3 PRO
[9] APPLE M3
Enter the number of your choice [hit enter for hardware defaults] [0]: 8
You selected: /Users/kellybrown/.local/share/instructlab/internal/system_profiles/apple/m3/m3_pro.yaml
--------------------------------------------
Initialization completed successfully!
You're ready to start using `ilab`. Enjoy!
--------------------------------------------
```
3) When prompted, provide the path to your default model. Otherwise, the default of a quantized [Merlinite](https://huggingface.co/instructlab/merlinite-7b-lab-GGUF) model is used.

The GPU profiles are listed by GPU type and number of GPUs present. If you happen to have a GPU configuration with a similar amount of vRAM as any of the above profiles, feel free to try them out!
*Example output of steps 1 - 3*
```shell
----------------------------------------------------
Welcome to the InstructLab CLI
This guide will help you to setup your environment
----------------------------------------------------

Please provide the following values to initiate the environment [press Enter for defaults]:
Path to taxonomy repo [/Users/<user>/.local/share/instructlab/taxonomy]:
Path to your model [/Users/<user>/.cache/instructlab/models/merlinite-7b-lab-Q4_K_M.gguf]:
```

### `ilab` directory layout after initializing your system
You can download this model with `ilab model download` command as well.

### Mac directory
4) The InstructLab CLI auto-detects your hardware and select the exact system profile that matches your machine. System profiles populate the `config.yaml` file with the proper parameter values based on your detected GPU types and avaiible vRAM.

After running `ilab config init` your directories will look like the following on a Mac system:
*Example output of profile auto-detection*

```shell
├─ ~/Library/Application\ Support/instructlab/models/ (1)
├─ ~/Library/Application\ Support/instructlab/datasets (2)
├─ ~/Library/Application\ Support/instructlab/taxonomy (3)
├─ ~/Library/Application\ Support/instructlab/checkpoints (4)
```
Generating config file and profiles:
/home/user/.config/instructlab/config.yaml
/home/user/.local/share/instructlab/internal/train_configuration/profiles

1) `/Users/USERNAME/Library/Caches/instructlab/models/`: Contains all downloaded large language models, including the saved output of ones you generate with ilab.
We have detected the AMD CPU profile as an exact match for your system.

2) `~/Library/Application\ Support/instructlab/datasets/`: Contains data output from the SDG phase, built on modifications to the taxonomy repository.
--------------------------------------------
Initialization completed successfully!
You're ready to start using `ilab`. Enjoy!
--------------------------------------------
```
3) `~/Library/Application\ Support/instructlab/taxonomy/`: Contains the skill and knowledge data.
5) If there is not an exact match for your system, you can manually select a system profile when prompted. There are various flags you can utilize with individual `ilab` commands that allow you to utilize your GPU if applicable.
4) `~/Users/USERNAME/Library/Caches/instructlab/checkpoints/`: Contains the output of the training process
*Example output of selecting a system profile*
### Linux directory
```shell
Please choose a system profile to use.
System profiles apply to all parts of the config file and set hardware specific defaults for each command.
First, please select the hardware vendor your system falls into
[1] APPLE
[2] INTEL
[3] AMD
[4] NVIDIA
Enter the number of your choice [0]: 1
You selected: APPLE
Next, please select the specific hardware configuration that most closely matches your system.
[0] No system profile
[1] APPLE M1 ULTRA
[2] APPLE M1 MAX
[3] APPLE M2 MAX
[4] APPLE M2 ULTRA
[5] APPLE M2 PRO
[6] APPLE M2
[7] APPLE M3 MAX
[8] APPLE M3 PRO
[9] APPLE M3
Enter the number of your choice [hit enter for hardware defaults] [0]: 8
You selected: /Users/kellybrown/.local/share/instructlab/internal/system_profiles/apple/m3/m3_pro.yaml
--------------------------------------------
Initialization completed successfully!
You're ready to start using `ilab`. Enjoy!
--------------------------------------------
```

The GPU profiles are listed by GPU type and number of GPUs present. If you happen to have a GPU configuration with a similar amount of vRAM as any of the above profiles, feel free to try them out!

### `ilab` directory layout after initializing your system

After running `ilab config init` your directories will look like the following on a Linux system:

Expand Down
84 changes: 42 additions & 42 deletions docs/getting-started/linux_amd.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -45,59 +45,59 @@ The following steps in this document use [Python venv](https://docs.python.org/3

1) Install with AMD ROCm

```bash
python3 -m venv --upgrade-deps venv
source venv/bin/activate
pip cache remove llama_cpp_python
pip install 'instructlab[rocm]' \
--extra-index-url https://download.pytorch.org/whl/rocm6.0 \
-C cmake.args="-DLLAMA_HIPBLAS=on" \
-C cmake.args="-DAMDGPU_TARGETS=all" \
-C cmake.args="-DCMAKE_C_COMPILER=/opt/rocm/llvm/bin/clang" \
-C cmake.args="-DCMAKE_CXX_COMPILER=/opt/rocm/llvm/bin/clang++" \
-C cmake.args="-DCMAKE_PREFIX_PATH=/opt/rocm" \
-C cmake.args="-DLLAMA_NATIVE=off"
```
```bash
python3 -m venv --upgrade-deps venv
source venv/bin/activate
pip cache remove llama_cpp_python
pip install 'instructlab[rocm]' \
--extra-index-url https://download.pytorch.org/whl/rocm6.0 \
-C cmake.args="-DLLAMA_HIPBLAS=on" \
-C cmake.args="-DAMDGPU_TARGETS=all" \
-C cmake.args="-DCMAKE_C_COMPILER=/opt/rocm/llvm/bin/clang" \
-C cmake.args="-DCMAKE_CXX_COMPILER=/opt/rocm/llvm/bin/clang++" \
-C cmake.args="-DCMAKE_PREFIX_PATH=/opt/rocm" \
-C cmake.args="-DLLAMA_NATIVE=off"
```

On Fedora 40+, use `-DCMAKE_C_COMPILER=clang-17` and `-DCMAKE_CXX_COMPILER=clang++-17.`

2) From your `venv` environment, verify `ilab` is installed correctly, by running the `ilab` command.

```shell
ilab
```
```shell
ilab
```

*Example output of the `ilab` command*
*Example output of the `ilab` command*

```shell
(venv) $ ilab
Usage: ilab [OPTIONS] COMMAND [ARGS]...
```shell
(venv) $ ilab
Usage: ilab [OPTIONS] COMMAND [ARGS]...

CLI for interacting with InstructLab.
CLI for interacting with InstructLab.

If this is your first time running ilab, it's best to start with `ilab
config init` to create the environment.
If this is your first time running ilab, it's best to start with `ilab
config init` to create the environment.
Options:
--config PATH Path to a configuration file. [default:
Options:
--config PATH Path to a configuration file. [default:
/Users/kellybrown/.config/instructlab/config.yaml]
-v, --verbose Enable debug logging (repeat for even more verbosity)
--version Show the version and exit.
--help Show this message and exit.
Commands:
config Command Group for Interacting with the Config of InstructLab.
data Command Group for Interacting with the Data generated by...
model Command Group for Interacting with the Models in InstructLab.
system Command group for all system-related command calls
taxonomy Command Group for Interacting with the Taxonomy of InstructLab.
Aliases:
chat model chat
generate data generate
serve model serve
train model train
```
-v, --verbose Enable debug logging (repeat for even more verbosity)
--version Show the version and exit.
--help Show this message and exit.
Commands:
config Command Group for Interacting with the Config of InstructLab.
data Command Group for Interacting with the Data generated by...
model Command Group for Interacting with the Models in InstructLab.
system Command group for all system-related command calls
taxonomy Command Group for Interacting with the Taxonomy of InstructLab.
Aliases:
chat model chat
generate data generate
serve model serve
train model train
```
!!! important
Every `ilab` command needs to be run from within your Python virtual environment. You can enter the Python environment by running the `source venv/bin/activate` command.
Expand Down
Loading

0 comments on commit 7e8d3fc

Please sign in to comment.