[punet] CI for quantization import/compilation/golden check #76
Assignees
Labels
Comments
|
Could you take a look at #70 and nod-ai/SHARK-TestSuite#272 ? Those are for llama, and it would be nice to have similar testing for punet and llama. For the first, I'm not particularly attached to the bash script but just wanted something running in this repo somehow. The second distills the program down to a common test format we can use from IREE on presubmit. |
|
Oh thanks, will have a look. |
stellaraccident
added a commit
that referenced
this issue
Jun 29, 2024
* Imports/validates the FP16 model running eagerly. * Imports/validates the int8 model running eagerly. * Exports the models. Progress on #76
ScottTodd
added a commit
that referenced
this issue
Jul 1, 2024
Sample run (with failures): https://github.com/nod-ai/sharktank/actions/runs/9752141473/job/26915109873 Definitely room for improvement, but it's a start. Progress on #76
ScottTodd
added a commit
to nod-ai/SHARK-TestSuite
that referenced
this issue
Jul 8, 2024
Related to nod-ai/SHARK-Platform#76 * Model source: https://github.com/nod-ai/sharktank/tree/main/sharktank/sharktank/models/punet * Python test: https://github.com/nod-ai/sharktank/blob/main/sharktank/integration/models/punet/integration_test.py * Python run script: https://github.com/nod-ai/sharktank/blob/main/sharktank/sharktank/models/punet/tools/run_punet.py * Weights are labeled in the sharktank sources. The primarily come from https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0 and https://huggingface.co/amd-shark/sdxl-quant-models Only testing compilation and basic execution for now. Result comparisons will follow once they are enabled upstream. TBD if the quantized outputs will be different enough across backends to need extra infrastructure configuration there.
renxida
pushed a commit
to nod-ai/SHARK-TestSuite
that referenced
this issue
Jul 18, 2024
Related to nod-ai/SHARK-Platform#76 * Model source: https://github.com/nod-ai/sharktank/tree/main/sharktank/sharktank/models/punet * Python test: https://github.com/nod-ai/sharktank/blob/main/sharktank/integration/models/punet/integration_test.py * Python run script: https://github.com/nod-ai/sharktank/blob/main/sharktank/sharktank/models/punet/tools/run_punet.py * Weights are labeled in the sharktank sources. The primarily come from https://huggingface.co/stabilityai/stable-diffusion-xl-base-1.0 and https://huggingface.co/amd-shark/sdxl-quant-models Only testing compilation and basic execution for now. Result comparisons will follow once they are enabled upstream. TBD if the quantized outputs will be different enough across backends to need extra infrastructure configuration there.
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
No description provided.
The text was updated successfully, but these errors were encountered: