Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ILAB Phase 3 Github Markdown Documentation #254

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

VaniHaripriya
Copy link

@VaniHaripriya VaniHaripriya commented Jan 20, 2025

Description

Added instructions to run InstructLab Pipeline with Data Science Pipelines on RHOAI

How Has This Been Tested?

Merge criteria:

  • The commits are squashed in a cohesive manner and have meaningful messages.
  • Testing instructions have been added in the PR body (for PRs involving changes that are not immediately obvious).
  • The developer has manually tested the changes and verified that the changes work

@VaniHaripriya VaniHaripriya changed the title Rhoaieng 18230 new ILAB Phase 3 Github Markdown Documentation Jan 20, 2025
@VaniHaripriya VaniHaripriya force-pushed the RHOAIENG-18230-New branch 2 times, most recently from 56a33a6 to 55716b8 Compare January 20, 2025 23:36
@@ -0,0 +1,646 @@
# Running InstructLab Pipeline with Data Science Pipelines on RHOAI
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

by definition, this doc should be in a different directory than standalone, since this is running on DSP

standalone/instructlab-pipelines-with-dsp-on-rhoai-.md Outdated Show resolved Hide resolved
standalone/instructlab-pipelines-with-dsp-on-rhoai-.md Outdated Show resolved Hide resolved
standalone/instructlab-pipelines-with-dsp-on-rhoai-.md Outdated Show resolved Hide resolved
standalone/instructlab-pipelines-with-dsp-on-rhoai-.md Outdated Show resolved Hide resolved

Upload the created tar archive to your object store.

The `standalone.py` script will do a simple validation check on the directory structure, here is a sample of what
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

standalone.py no longer used in Phase 2, change this to 'The pipeline'

standalone/instructlab-pipelines-with-dsp-on-rhoai-.md Outdated Show resolved Hide resolved
standalone/instructlab-pipelines-with-dsp-on-rhoai-.md Outdated Show resolved Hide resolved
standalone/instructlab-pipelines-with-dsp-on-rhoai-.md Outdated Show resolved Hide resolved
|`k8s_storage_class_name` |A Kubernetes StorageClass name for persistent volumes. Selected StorageClass must support RWX PersistentVolumes.|


[RBAC configuration]: https://github.com/opendatahub-io/ilab-on-ocp/tree/main/standalone#rbac-requirements-when-running-in-a-kubernetes-job
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should copy this documentation somewhere not in 'standalone' folder

* Red Hat - Authorino installed
* Red Hat Openshift Serverless installed
* Red Hat OpenShift Service Mesh v2 (v3 is not compatible with RHOAI)
* Red Hat OpenShift AI

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
* Red Hat OpenShift AI
* Red Hat OpenShift AI and operator dependencies documented at [OpenShift AI Supported Configurations](https://access.redhat.com/articles/rhoai-supported-configs))

standalone/instructlab-pipelines-with-dsp-on-rhoai-.md Outdated Show resolved Hide resolved
$ cp -r <path-to-model-downloaded-dir>/rhelai1/granite-7b-starter/* s3-data/model
```

Add your taxonomy tree to the `taxonomy` directory

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this step is not needed. The pipeline has a parameter with an external git repository containing the taxonomy. @gmfrasca @rimolive can you confirm?


<p align="center"><img src="assets/images/parameters.png" width=50%\></p>

#### Available Pipeline Parameters:

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about adding two additional sections providing example pipeline parameters?

  • Example pipeline parameters for a quick pipeline run:
    Example parameters to do a quick test that can be run in aprox 90m using x NVIDIA XXX GPUs

  • Example pipeline parametes for a full pipeline run

@rimolive and @gmfrasca can probably provide these examples

@VaniHaripriya VaniHaripriya force-pushed the RHOAIENG-18230-New branch 3 times, most recently from 2b49ac2 to 0576ffc Compare January 29, 2025 12:22
@VaniHaripriya VaniHaripriya marked this pull request as ready for review January 29, 2025 14:38

```bash
oc new-project nfs
oc apply -f ./nfs-server-deployment.yaml

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The document doesn't explain where is this file downloaded or what should be the content

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see now that the file is in the standalone folder. Consider adding a link in the markdown doc


For creating NFS storage-class
```bash
oc apply -f ./nfs-storage-class.yaml

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The document doesn't explain where is this file downloaded or what should be the content

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see now that the file is in the standalone folder. Consider adding a link in the markdown doc

run-pipeline-on-rhoai-.md Outdated Show resolved Hide resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants