doc updates - modules list and gpt behavior example (#99)

Co-authored-by: Jamie Broomall <[email protected]>
whylabs · Jul 21, 2023 · 57ed00c · 57ed00c
1 parent 384b72f
commit 57ed00c
Show file tree

Hide file tree

Showing 2 changed files with 14 additions and 10 deletions.
diff --git a/langkit/docs/modules.md b/langkit/docs/modules.md
@@ -1,13 +1,15 @@
 # Modules List
 
-- [Injections](#injections)
-- [Input/Output](#inputoutput)
-- [Regexes](#regexes)
-- [Sentiment](#sentiment)
-- [Text Statistics](#text-statistics)
-- [Themes](#themes)
-- [Topics](#topics)
-- [Toxicity](#toxicity)
+|             **Module**              |                               **Description**                               | **Target**          |            **Notes**             |
+| :---------------------------------: | :-------------------------------------------------------------------------: | ------------------- | :------------------------------: |
+|      [Injections](#injections)      |                   Prompt injection classification scores                    | Prompt              |                                  |
+|    [Input/Output](#inputoutput)     |               Semantic similarity between prompt and response               | Prompt and Response |        Default llm metric        |
+|         [Regexes](#regexes)         |              Regex pattern matching for sensitive information               | Any string column   | Default llm metric, light-weight |
+|       [Sentiment](#sentiment)       |                             Sentiment Analysis                              | Any string column   |        Default llm metric        |
+| [Text Statistics](#text-statistics) |           Text quality, readability, complexity, and grade level.           | Any string column   | Default llm metric, light-weight |
+|          [Themes](#themes)          | Semantic similarity between set of known jailbreak and LLM refusal examples | Any string column   |        Default llm metric        |
+|          [Topics](#topics)          |  Text classification into predefined topics - law, finance, medical, etc.   | Any string column   |                                  |
+|        [Toxicity](#toxicity)        |                   Toxicity, harmfulness and offensiveness                   | Any string column   |        Default llm metric        |
 
 ## Injections
 
@@ -30,6 +32,8 @@ The `prompt.injection` computed column will contain classification scores from a
 
 It currently uses the HuggingFace's model [`JasperLS/gelectra-base-injection`](https://huggingface.co/JasperLS/gelectra-base-injection) to make predictions.
 
+> Note: The current model has been known to yield high false positive rates and might not be suited for production use.
+
 ## Input/Output
 
 The `input_output` module will compute similarity scores between two columns called `prompt` and `response`. It will create a new column named `response.relevance_to_prompt`

diff --git a/langkit/examples/ChatGPT_Behavioral_Monitoring.ipynb b/langkit/examples/ChatGPT_Behavioral_Monitoring.ipynb
@@ -15,7 +15,7 @@
     "\n",
     "In this blog, we will  discuss seven groups of metrics you can use to keep track of the LLM’s behavior. We will calculate these metrics for chatGPT’s responses for a fixed set of 200 prompts across 35 days  and track how chatGPT’s behavior evolves within the period. Our focus task will be long-form question answering, and we will use LangKit, whylogs and WhyLabs to calculate, track and monitor the model’s behavior across time.\n",
     "\n",
-    "You can check the resulting dashboard for this project at [WhyLabs](placeholder) (no sign-in required)."
+    "You can check the resulting dashboard for this project at [WhyLabs](https://hub.whylabsapp.com/resources/demo-chatgpt-behavior-ELI5/columns/response.difficult_words?dateRange=2023-03-05-to-2023-04-09&targetOrgId=demo&sessionToken=session-8gcsnbVy) (no sign-in required)."
    ]
   },
   {
@@ -1906,7 +1906,7 @@
     "\n",
     "There you have it, you should have your dashboard populated with the daily profiles!\n",
     "\n",
-    "As mentioned, you can check a demo dashboard with the same results at [WhyLabs](placeholder) (no sign-in required).\n",
+    "As mentioned, you can check a demo dashboard with the same results at [WhyLabs](https://hub.whylabsapp.com/resources/demo-chatgpt-behavior-ELI5/columns/response.difficult_words?dateRange=2023-03-05-to-2023-04-09&targetOrgId=demo&sessionToken=session-8gcsnbVy) (no sign-in required).\n",
     "\n",
     "We have a brief discussion on the results in the blog post [Behavioral Monitoring of Large Language Models](placeholder) that accompanies this example. But we encourage you to explore the results yourself and draw your own conclusions!"
    ]