Implement a quality check via the Dialogflow API

[anbo-de]

Context

The quality of a chatbot should be measured in multiple dimensions. The custom script should be written and test cases should be prepared for each dimension of quality check. Script should demonstrate quality check report after its execution.

Definition of Done (DoD)

• Quality check for the intent recognition (Precision, Recall)
• Quality check for the components recognition (Precision, Recall)
• Quality check for KGQA system (in multiple languages)

[jayeshdesai4520]

Hi, can I have context about this issue

[RicardoUsbeck]

We were discussing, that you need to test whether certain intents stay stable. That is, implement tests which fire not only one turn but multiple so you can see whether a conversation works or not.

[jayeshdesai4520]

Hi, I know two platforms to test the chatbot

1. https://www.botium.ai/
2. https://qbox.ai/GetStarted

Botium has a free plan to test

qbox gives only 5 free tests

I have used both of them just to test the current intents of our chatbot and both of them works fine

below are the images of testing

Botium -
screenshot

qbox.ai -
screenshot

[anbo-de]

@jayeshdesai4520 Where is the current quality check available/accessible?

[jayeshdesai4520]

@anbo-de here is the screenshot of the test which I ran today Botium was taking so much time so I did use Qbox.ai
https://i.imgur.com/H1SGatb.png

so the result is I need to add more training phrases in 6 intents according to Test

[anbo-de]

@jayeshdesai4520 Please create list of test cases as preparation.