updated main and samples readme (#135)

Author: saipj

Readme.md

@@ -169,21 +169,9 @@ By default, torch-ort-infer depends on PyTorch 1.12 and ONNX Runtime OpenVINO EP
 
     - `python -m torch_ort.configure`
 
-## Verify your installation
-
-Once you have created your environment, execute the following steps to validate that your installation is correct.
-
-1. Clone this repo
-
-    - `git clone [email protected]:pytorch/ort.git`
-<br/><br/>
-2. Install extra dependencies
-
-    - `pip install wget pandas transformers`
-<br/><br/>
-3. Run the inference script
+## Samples
 
-    - `python ./ort/torch_ort_inference/tests/bert_for_sequence_classification.py`
+To see OpenVINO™ integration with Torch-ORT in action, see [demos](/torch_ort_inference/demos), which shows you how to run inference on some of the most popular Deep Learning models.
 
 ## Add ONNX Runtime for PyTorch to your PyTorch inference script
 
@@ -212,12 +200,42 @@ If no provider options are specified by user, OpenVINO™ Execution Provider is
 backend = "CPU"
 precision = "FP32"
 ```
+
 For more details on APIs, see [usage.md](/torch_ort_inference/docs/usage.md).
 
 ### Note
 
 Experimental support on Intel® MyriadX VPU in this preview. 
 
+## Code Sample
+
+Below is an example of how you can leverage OpenVINO™ integration with Torch-ORT in a simple NLP usecase. 
+A pretrained [BERT model](https://huggingface.co/textattack/bert-base-uncased-CoLA) fine-tuned on the CoLA dataset from HuggingFace model hub is used to predict grammar correctness on a given input text. 
+
+
+```python 
+from transformers 
+import AutoTokenizer, AutoModelForSequenceClassification
+import numpy as np
+from torch_ort import ORTInferenceModule
+tokenizer = AutoTokenizer.from_pretrained(
+            "textattack/bert-base-uncased-CoLA")
+model = AutoModelForSequenceClassification.from_pretrained(
+        "textattack/bert-base-uncased-CoLA")
+# Wrap model in ORTInferenceModule to prepare the model for inference using OpenVINO Execution Provider on CPU
+model = ORTInferenceModule(model)
+text = "Replace me any text by you'd like ."
+encoded_input = tokenizer(text, return_tensors='pt')
+output = model(**encoded_input)
+# Post processing
+logits = output.logits
+logits = logits.detach().cpu().numpy()
+# predictions
+pred = np.argmax(logits, axis=1).flatten()
+print("Grammar correctness label (0=unacceptable, 1=acceptable)")
+print(pred)
+```
+
 ## License
 
 This project has an MIT license, as found in the [LICENSE](LICENSE) file.

torch_ort_inference/demos/bert.md

@@ -0,0 +1,85 @@
+# Bert for Sequence Classification
+
+This demo shows how to use Intel® OpenVINO™ integration with Torch-ORT to check grammar in text with ONNX Runtime OpenVINO Execution Provider.
+
+We use a sequence classification model [textattack/bert-base-uncased-CoLA](https://huggingface.co/textattack/bert-base-uncased-CoLA) from HuggingFace models. This model is trained on the BERT architecture to check grammar.
+
+## Model Metadata
+| Domain | Application | Industry  | Framework | Input Data Format |
+| ------------- | --------  | -------- | --------- | -------------- | 
+| NLP | Sequence Classification | General | PyTorch | Text |
+
+## Prerequisites
+
+- Ubuntu 18.04, 20.04
+- Python* 3.7, 3.8 or 3.9
+
+## Install in a local Python environment
+
+1. Upgrade pip
+
+    - `pip install --upgrade pip`
+<br/><br/>
+
+2. Install torch-ort-infer with OpenVINO dependencies
+
+    - `pip install torch-ort-infer[openvino]`
+<br/><br/>
+3. Run post-installation script
+
+    - `python -m torch_ort.configure`
+
+## Verify your installation
+
+Once you have created your environment, execute the following steps to validate that your installation is correct.
+
+1. Clone this repo
+
+    - `git clone https://github.com/pytorch/ort.git`
+<br/><br/>
+2. Install extra dependencies
+
+    - `pip install wget pandas transformers`
+<br/><br/>
+3. Run the inference script with default options
+
+    - `python ./ort/torch_ort_inference/demos/bert_for_sequence_classification.py`
+<br/><br/>
+    **Note**: OpenVINOExecutionProvider is enabled with CPU and FP32 by default.
+
+## Run demo with custom options
+```
+usage: bert_for_sequence_classification.py [-h] [--pytorch-only] [--input INPUT] [--input-file INPUT_FILE] [--provider PROVIDER] [--backend BACKEND] [--precision PRECISION]
+
+PyTorch BERT Sequence Classification Example
+
+optional arguments:
+-h, --help                  show this help message and exit
+--pytorch-only              disables ONNX Runtime inference
+--input "INPUT"             input sentence, put it in quotes
+--input-file INPUT_FILE     path to input file in .tsv format
+--provider PROVIDER         ONNX Runtime Execution Provider
+--backend BACKEND           OpenVINO target device (CPU, GPU).
+--precision PRECISION       OpenVINO target device precision (FP16 or FP32)
+```
+    
+**Note**: Default options and inputs are selected if no arguments are given
+    
+## Expected output
+```
+OpenVINOExecutionProvider is enabled with CPU and FP32 by default.
+Input not provided! Using default input...
+
+Number of sentences: 2
+        Grammar correctness label (0=unacceptable, 1=acceptable)
+
+        'This is a BERT sample.' : 1
+        'User input is valid not.' : 0
+
+Average inference time: 25.2306ms
+Total Inference time: 50.4613ms
+```
+
+**Note**: This demo has a warm-up run and then inference time is measured on the subsequent runs. The execution time of first run is in general higher compared to the next runs as it includes inline conversion to ONNX, many one-time graph transformations and optimizations steps.
+
+For more details on APIs, see [usage.md](/torch_ort_inference/docs/usage.md)

…ests/bert_for_sequence_classification.py …emos/bert_for_sequence_classification.pytorch_ort_inference/tests/bert_for_sequence_classification.py renamed to torch_ort_inference/demos/bert_for_sequence_classification.py torch_ort_inference/tests/bert_for_sequence_classification.py renamed to torch_ort_inference/demos/bert_for_sequence_classification.py

@@ -0,0 +1,96 @@
+
+#  ResNet-50 Image Classification
+
+This demo shows how to use Intel® OpenVINO™ integration with Torch-ORT to classify objects in images with ONNX Runtime OpenVINO Execution Provider.
+
+We use an image classification model [ResNet-50](https://pytorch.org/vision/stable/models/generated/torchvision.models.resnet50.html#torchvision.models.resnet50) from Torchvision and [ImageNet labels](https://raw.githubusercontent.com/pytorch/hub/master/imagenet_classes.txt) to classify objects. In the labels file, you'll find 1,000 different categories that were used in the Imagenet competition.
+  
+
+## Data
+The input to the model is a 224 x 224 image (airplane in our case), and the **output** is a list of estimated class probabilities.
+
+<p align="center" width="100%"> <img src="plane.jpg" alt="drawing" height="300" width="400"/>
+
+## Model Metadata
+| Domain | Application | Industry  | Framework | Training Data | Input Data Format |
+| ------------- | --------  | -------- | --------- | --------- | -------------- | 
+| Vision | Image Classification | General | Pytorch | [ImageNet](http://www.image-net.org/) | Image (RGB/HWC)|
+
+## Pre-requisites
+
+- Ubuntu 18.04, 20.04
+
+- Python* 3.7, 3.8 or 3.9
+
+## Install in a local Python environment
+
+1. Upgrade pip
+
+    - `pip install --upgrade pip`
+<br/><br/>
+
+2. Install torch-ort-infer with OpenVINO dependencies
+
+    - `pip install torch-ort-infer[openvino]`
+<br/><br/>
+3. Run post-installation script
+
+    - `python -m torch_ort.configure`
+
+## Verify your installation
+
+Once you have created your environment, execute the following steps to validate that your installation is correct.
+
+1. Clone this repo
+
+    - `git clone https://github.com/pytorch/ort.git`
+<br/><br/>
+2. Install extra dependencies
+
+    - `pip install wget Pillow torchvision`
+<br/><br/>
+3. Run the inference script
+
+    - `python ./ort/torch_ort_inference/demos/resnet_image_classification.py --input-file ./ort/torch_ort_inference/demos/plane.jpg`
+<br/><br/>
+    **Note**: OpenVINOExecutionProvider is enabled with CPU and FP32 by default.
+
+## Run demo with custom options
+```
+usage: resnet_image_classification.py [-h] [--pytorch-only] [--labels LABELS] --input-file INPUT_FILE [--provider PROVIDER] [--backend BACKEND] [--precision PRECISION]
+
+PyTorch Image Classification Example
+
+optional arguments:
+  -h, --help                show this help message and exit
+  --pytorch-only            disables ONNX Runtime inference
+  --labels LABELS           path to labels file
+  --input-file INPUT_FILE   path to input image file
+  --provider PROVIDER       ONNX Runtime Execution Provider
+  --backend BACKEND         OpenVINO target device (CPU, GPU or MYRIAD)
+  --precision PRECISION     OpenVINO target device precision (FP16 or FP32)
+```
+    
+**Note**: Some default options are selected if no arguments are given
+
+## Expected Output
+
+For the input image of an airplane, you can see the output something similar to:
+
+```
+Labels , Probabilities:
+airliner 0.9133861660957336
+wing 0.08387967199087143
+airship 0.001151240081526339
+warplane 0.00030989135848358274
+projectile 0.0002502237621229142
+```
+
+Here, the network classifies the image as an airplane, with a high score of 0.91.
+
+**Note**: This demo has a warm-up run and then inference time is measured on the subsequent runs. The execution time of first run is in general higher compared to the next runs as it includes inline conversion to ONNX, many one-time graph transformations and optimizations steps.
+
+For more details on APIs, see [usage.md](/torch_ort_inference/docs/usage.md).
+
+
+