LLM Integration #5861

TmmmmmR · 2024-10-30T09:41:38Z

Overview

This extension integrates LLM with ZAP and includes two main features:

API Sequencing: Import Swagger/OpenAPI definitions to generate sequences of HTTP calls for subsequent scanning operations.
Alert Review: Examine an alert and determine the confidence level based on evidence from ZAP, complete with an explanation for the updated confidence level.

Related Issues

Specify any related issues or pull requests by linking to them.

Checklist

For more details, please refer to the developer rules and guidelines.

github-actions · 2024-10-30T09:41:52Z

All contributors have signed the CLA ✍️ ✅
_{Posted by the CLA Assistant Lite bot.}

kingthorin

I'll have to finish the rest of my review in a bit, but here's some starting bits.

README.md

addOns/llm/gradle.properties

addOns/llm/src/main/java/org/zaproxy/addon/llm/ExtensionLlm.java

addOns/llm/src/main/java/org/zaproxy/addon/llm/services/LlmCommunicationService.java

addOns/llm/src/main/java/org/zaproxy/addon/llm/ExtensionLlm.java

addOns/llm/src/main/java/org/zaproxy/addon/llm/communication/Confidence.java

addOns/llm/src/main/java/org/zaproxy/addon/llm/services/LlmAssistant.java

kingthorin · 2024-10-31T13:00:27Z

addOns/llm/src/main/java/org/zaproxy/addon/llm/services/LlmCommunicationService.java

+                getExtAlert().updateAlert(updatedAlert);
+                getExtAlert().updateAlertInTree(originalAlert, updatedAlert);
                if (alert.getHistoryRef() != null) {
-                    alert.getHistoryRef().updateAlert(updated_alert);
+                    alert.getHistoryRef().updateAlert(updatedAlert);
                    if (alert.getHistoryRef().getSiteNode() != null) {
                        // Needed if the same alert was raised on another href for the same
                        // SiteNode
-                        alert.getHistoryRef().getSiteNode().updateAlert(updated_alert);
+                        alert.getHistoryRef().getSiteNode().updateAlert(updatedAlert);


I'm confused about these conditionals and doing the same action over and over, shouldn't the call hat 167 have done it regardless of the conditions?

I don't know why, but without this I get this exception when I do alert review for a node of alerts :

Index -1 out of bounds for length 10 java.lang.ArrayIndexOutOfBoundsException: Index -1 out of bounds for length 10

I used the same code from this extension : https://github.com/zaproxy/zap-extensions/blob/main/addOns/alertFilters/src/main/java/org/zaproxy/zap/extension/alertFilters/ExtensionAlertFilters.java#L459

@kingthorin : do you see another way to walk through alerts ?

This is the "normal" loop we use for reading. I guess part of the problem you're encountering might be timing with the updates. I'd have to play with it and see if your conditions can be simplified or worked around somehow.

https://github.com/zaproxy/community-scripts/blob/main/standalone/Loop%20through%20alerts.js

addOns/llm/src/main/java/org/zaproxy/addon/llm/ui/ImportDialog.java

kingthorin

Should only have the base help and messages.

kingthorin · 2024-10-31T16:56:38Z

There's likely a bunch of places swagger should be replaced with openapi.

https://swagger.io/blog/api-strategy/difference-between-swagger-and-openapi/

yns000 · 2024-11-08T16:05:38Z

Hi TmmmmmR,

I started my review on this, but ended up having a few questions, can I propose we setup a meeting between yourself and the 2 reviewers so that to see a demo of the addon?

Thank you,
Yiannis

TmmmmmR · 2024-11-11T14:38:35Z

Hello @yns000, yes of course ! I'm available on slack under the dev-llm channel, my username is temmar.

addOns/llm/src/main/java/org/zaproxy/addon/llm/services/LlmAssistant.java

addOns/llm/src/main/java/org/zaproxy/addon/llm/services/LlmCommunicationService.java

addOns/llm/src/main/java/org/zaproxy/addon/llm/ui/ImportDialog.java

addOns/llm/src/main/java/org/zaproxy/addon/llm/ui/settings/LlmOptionsPanel.java

addOns/llm/src/main/java/org/zaproxy/addon/llm/ui/settings/LlmOptionsParam.java

addOns/llm/src/main/java/org/zaproxy/addon/llm/utils/HistoryPersister.java

addOns/llm/src/main/javahelp/org/zaproxy/addon/llm/resources/help/contents/about.html

TmmmmmR · 2024-11-20T08:55:44Z

I have read the CLA Document and I hereby sign the CLA.

Signed-off-by: Abdessamad TEMMAR <[email protected]>

psiinon · 2024-12-20T14:50:25Z

You'll need to add "llm" here https://github.com/zaproxy/zap-extensions/pull/5861/files otherwise it cant be deployed.
I've done that locally and I can see the extension in ZAP but I cant see any GUI changes for it. I'll look into that some more..

psiinon

Added some comments which should get the help displayed correctly

psiinon · 2024-12-20T16:26:20Z

addOns/llm/src/main/javahelp/org/zaproxy/addon/llm/resources/help/helpset.hs

+  PUBLIC "-//Sun Microsystems Inc.//DTD JavaHelp HelpSet Version 2.0//EN"
+         "http://java.sun.com/products/javahelp/helpset_2_0.dtd">
+<helpset version="2.0" xml:lang="en-GB">
+  <title>Simple Example Add-On</title>


-> LLM Integration (or somethong like that)

psiinon · 2024-12-20T16:26:36Z

addOns/llm/src/main/javahelp/org/zaproxy/addon/llm/resources/help/helpset.hs

+  <title>Simple Example Add-On</title>
+
+  <maps>
+     <homeID>top</homeID>


top -> addon.llm

psiinon · 2024-12-20T16:27:15Z

addOns/llm/src/main/javahelp/org/zaproxy/addon/llm/resources/help/index.xml

+
+<index version="2.0">
+    <!-- index entries are merged (sorted) into core index -->
+	<indexitem text="simple" target="simple" />


<indexitem text="llm" target="addon.llm" />

psiinon · 2024-12-20T16:28:36Z

addOns/llm/src/main/javahelp/org/zaproxy/addon/llm/resources/help/map.jhm

+         "http://java.sun.com/products/javahelp/map_1_0.dtd">
+
+<map version="1.0">
+	<mapID target="simple-icon" url="contents/images/cake.png" />


all instances of simple -> llm. Will need a new icon - I'll look for something suitable 😁 Probably dont need the about.html page?

psiinon · 2024-12-20T16:29:30Z

addOns/llm/src/main/javahelp/org/zaproxy/addon/llm/resources/help/toc.xml

+<toc version="2.0">
+	<tocitem text="ZAP User Guide" tocid="toplevelitem">
+		<tocitem text="Add Ons" tocid="addons">
+			<tocitem text="Simple Example" image="simple-icon" target="simple">


Simple > LLM, simple -> llm and drop the about.html page?

psiinon · 2024-12-20T16:30:01Z

addOns/llm/src/main/javahelp/org/zaproxy/addon/llm/resources/help/contents/about.html

@@ -0,0 +1,23 @@
+<!DOCTYPE html PUBLIC "-//W3C//DTD HTML 3.2 Final//EN">


This file is probably not really needed?

kingthorin reviewed Oct 30, 2024

View reviewed changes

kingthorin reviewed Oct 31, 2024

View reviewed changes

addOns/llm/src/main/java/org/zaproxy/addon/llm/ExtensionLlm.java Outdated Show resolved Hide resolved

kingthorin reviewed Oct 31, 2024

View reviewed changes

addOns/llm/src/main/java/org/zaproxy/addon/llm/communication/Confidence.java Show resolved Hide resolved

kingthorin reviewed Oct 31, 2024

View reviewed changes

addOns/llm/src/main/java/org/zaproxy/addon/llm/services/LlmAssistant.java Outdated Show resolved Hide resolved

kingthorin reviewed Oct 31, 2024

View reviewed changes

addOns/llm/src/main/java/org/zaproxy/addon/llm/ui/ImportDialog.java Outdated Show resolved Hide resolved

kingthorin reviewed Oct 31, 2024

View reviewed changes