Skip to content

Commit

Permalink
Merge pull request #56 from Giskard-AI/readme-image-classification-ob…
Browse files Browse the repository at this point in the history
…ject-detection

Add readme for image classification and object detection
  • Loading branch information
rabah-khalek authored Aug 19, 2024
2 parents 6287112 + 9acb2c7 commit 9f85799
Show file tree
Hide file tree
Showing 5 changed files with 83 additions and 2 deletions.
5 changes: 5 additions & 0 deletions .github/workflows/do-release.yml
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,16 @@ jobs:
steps:
- name: Checkout code
uses: actions/[email protected]

- name: Remove dark theme logo from README
run: sed -i 's/.*#gh-dark-mode-only.*//' README.md

- name: Setup PDM
uses: pdm-project/setup-pdm@v3
with:
python-version: "3.10"
cache: false

- name: Build dist
run: pdm build

Expand Down
3 changes: 2 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -27,7 +27,8 @@
## Supported computer vision tasks

- **Facial Landmark Detection** ([Readme](https://github.com/Giskard-AI/giskard-vision/blob/main/giskard_vision/landmark_detection/Readme.md))
- **Image Classification** 🔜 Coming soon!
- **Image Classification** ([Readme](https://github.com/Giskard-AI/giskard-vision/blob/main/giskard_vision/image_classification/Readme.md))
- **Object Detection** ([Readme](https://github.com/Giskard-AI/giskard-vision/blob/main/giskard_vision/object_detection/Readme.md))

## Install

Expand Down
34 changes: 34 additions & 0 deletions giskard_vision/image_classification/Readme.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# `image_classification` module

This module contains model wrappers, dataloaders, tests and all the ingredients needed to evaluate your single-label image classification models.
In particular this module allows you to evaluate your model against the following criteria:

- Performance on images with different basic image attributes.
- Performance on images with various metadata from the datasets.
- Robustness against image perturbations like blurring, resizing, recoloring (performed by `opencv`: https://github.com/opencv/opencv)

## Wrapped Datasets

- [geirhos_conflict_stimuli](https://www.tensorflow.org/datasets/catalog/geirhos_conflict_stimuli) through Tensorflow Datasets
- [CIFAR100](https://huggingface.co/datasets/uoft-cs/cifar100) through Hugging Face
- [Skin cancer](https://huggingface.co/datasets/marmal88/skin_cancer) through Hugging Face


## Scan and Supported Classification

Once the model and dataloader (`dl`) are wrapped, you can scan the model with the scan API in Giskard vision core:

```python
from giskard_vision.core.scanner import scan

results = scan(model, dl)
```

It adapts the [scan API in Giskard Python library](https://github.com/Giskard-AI/giskard#2--scan-your-model-for-issues) to magically scan the vision model with the dataloader.

Currently, due to the constraint of the scan API, we support a subset of image classification tasks:

- [x] Multiclass and single label
- [ ] Multiclass and multi-label

We will be working to remove such limit for the scan.
12 changes: 11 additions & 1 deletion giskard_vision/landmark_detection/Readme.md
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,17 @@ You can also check our publicly hosted versions of these datasets on S3:
- https://poc-face-aligment.s3.eu-north-1.amazonaws.com/300W/300w.zip.004
- FFHQ (only meta data): https://poc-face-aligment.s3.eu-north-1.amazonaws.com/ffhq/json.zip

## Metrics
## Scan and Metrics

Once the model and dataloader (`dl`) are wrapped, you can scan the model with the scan API in Giskard vision core:

```python
from giskard_vision.core.scanner import scan

results = scan(model, dl)
```

It adapts the [scan API in Giskard Python library](https://github.com/Giskard-AI/giskard#2--scan-your-model-for-issues) to magically scan the vision model with the dataloader. The considered metrics are:

- [x] ME: Mean Euclidean distances
- [x] NME: Normalised Mean Euclidean distances
Expand Down
31 changes: 31 additions & 0 deletions giskard_vision/object_detection/Readme.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
# `object_detection` module

This module contains model wrappers, dataloaders, tests and all the ingredients needed to evaluate your object detection models.
In particular this module allows you to evaluate your model against the following criteria:

- Performance on images with different basic image attributes.
- Performance on images with various metadata from the datasets.
- Robustness against image perturbations like blurring, resizing, recoloring (performed by `opencv`: https://github.com/opencv/opencv)

## Wrapped Datasets

- [Racoon](https://www.kaggle.com/datasets/debasisdotcom/racoon-detection/data)
- [300W](https://ibug.doc.ic.ac.uk/resources/300-W/), using the boundary box around all face landmarks
- [ffhq](https://github.com/DCGM/ffhq-features-dataset), using the boundary box around all face landmarks
- [Living room passes](https://huggingface.co/datasets/Nfiniteai/living-room-passes) through Hugging Face

## Scan and Metrics

Once the model and dataloader (`dl`) are wrapped, you can scan the model with the scan API in Giskard vision core:

```python
from giskard_vision.core.scanner import scan

results = scan(model, dl)
```

It adapts the [scan API in Giskard Python library](https://github.com/Giskard-AI/giskard#2--scan-your-model-for-issues) to magically scan the vision model with the dataloader. The considered metric is:

- [x] Intersection over Union (IoU)

Currently, we only support one object both in the model prediction and ground truth, due to the constraint of the scan API. We will be working to remove such limit.

0 comments on commit 9f85799

Please sign in to comment.