fix(aiguard): handle empty AI Guard response on error by smola · Pull Request #16556 · DataDog/dd-trace-py

smola · 2026-02-18T13:03:35Z

Description

Handle AI Guard evaluate responses that return empty or unparsable JSON by defaulting to an empty result and letting existing error handling surface a clear client error instead of raising a TypeError. This prevents NoneType checks from crashing and keeps error reporting consistent when the backend returns no body.

Fixes APPSEC-61354, a bug observed in the wild, possibly on a network error.

Testing

None

Risks

Low. Change only affects error handling when the response body is empty or unparsable.

Additional Notes

None

PR by Bits
View session in Datadog

Comment @DataDog to request changes

datadog-official · 2026-02-18T13:03:37Z

[View session in Datadog]

Bits Dev status: ✅ Done

CI Auto-fix: Disabled | Enable

Comment @DataDog to request changes

cit-pr-commenter-54b7da · 2026-02-18T13:05:15Z

Codeowners resolved as

ddtrace/appsec/ai_guard/_api_client.py                                  @DataDog/asm-python
releasenotes/notes/fix-ai-guard-empty-result-9ceb9de92d746a86.yaml      @DataDog/apm-python
tests/appsec/ai_guard/api/test_api_client.py                            @DataDog/asm-python

pr-commenter · 2026-02-18T13:24:46Z

Performance SLOs

Comparing candidate dd/fix/ai-guard-empty-response (147af46) with baseline main (b5b77a2)

📈 Performance Regressions (1 suite)

📈 iastaspectsospath - 24/24

✅ ospathbasename_aspect

Time: ✅ 513.729µs (SLO: <700.000µs 📉 -26.6%) vs baseline: 📈 +20.5%

Memory: ✅ 42.762MB (SLO: <46.000MB -7.0%) vs baseline: +5.2%

✅ ospathbasename_noaspect

Time: ✅ 433.286µs (SLO: <700.000µs 📉 -38.1%) vs baseline: +0.8%

Memory: ✅ 42.585MB (SLO: <46.000MB -7.4%) vs baseline: +5.0%

✅ ospathjoin_aspect

Time: ✅ 628.105µs (SLO: <700.000µs 📉 -10.3%) vs baseline: ~same

Memory: ✅ 42.507MB (SLO: <46.000MB -7.6%) vs baseline: +4.8%

✅ ospathjoin_noaspect

Time: ✅ 634.615µs (SLO: <700.000µs -9.3%) vs baseline: +0.5%

Memory: ✅ 42.625MB (SLO: <46.000MB -7.3%) vs baseline: +5.0%

✅ ospathnormcase_aspect

Time: ✅ 348.763µs (SLO: <700.000µs 📉 -50.2%) vs baseline: -0.4%

Memory: ✅ 42.605MB (SLO: <46.000MB -7.4%) vs baseline: +4.8%

✅ ospathnormcase_noaspect

Time: ✅ 357.245µs (SLO: <700.000µs 📉 -49.0%) vs baseline: -1.0%

Memory: ✅ 42.546MB (SLO: <46.000MB -7.5%) vs baseline: +4.5%

✅ ospathsplit_aspect

Time: ✅ 492.273µs (SLO: <700.000µs 📉 -29.7%) vs baseline: -0.7%

Memory: ✅ 42.703MB (SLO: <46.000MB -7.2%) vs baseline: +5.1%

✅ ospathsplit_noaspect

Time: ✅ 500.661µs (SLO: <700.000µs 📉 -28.5%) vs baseline: +0.6%

Memory: ✅ 42.526MB (SLO: <46.000MB -7.6%) vs baseline: +4.7%

✅ ospathsplitdrive_aspect

Time: ✅ 378.406µs (SLO: <700.000µs 📉 -45.9%) vs baseline: +0.9%

Memory: ✅ 42.585MB (SLO: <46.000MB -7.4%) vs baseline: +4.9%

✅ ospathsplitdrive_noaspect

Time: ✅ 73.032µs (SLO: <700.000µs 📉 -89.6%) vs baseline: ~same

Memory: ✅ 42.664MB (SLO: <46.000MB -7.3%) vs baseline: +5.3%

✅ ospathsplitext_aspect

Time: ✅ 455.548µs (SLO: <700.000µs 📉 -34.9%) vs baseline: ~same

Memory: ✅ 42.664MB (SLO: <46.000MB -7.3%) vs baseline: +5.1%

✅ ospathsplitext_noaspect

Time: ✅ 466.414µs (SLO: <700.000µs 📉 -33.4%) vs baseline: ~same

Memory: ✅ 42.743MB (SLO: <46.000MB -7.1%) vs baseline: +5.1%

✅ All Tests Passing (2 suites)

✅ iastaspectssplit - 12/12

✅ rsplit_aspect

Time: ✅ 155.769µs (SLO: <250.000µs 📉 -37.7%) vs baseline: +3.4%

Memory: ✅ 42.644MB (SLO: <46.000MB -7.3%) vs baseline: +5.1%

✅ rsplit_noaspect

Time: ✅ 157.560µs (SLO: <250.000µs 📉 -37.0%) vs baseline: ~same

Memory: ✅ 42.546MB (SLO: <46.000MB -7.5%) vs baseline: +4.6%

✅ split_aspect

Time: ✅ 148.557µs (SLO: <250.000µs 📉 -40.6%) vs baseline: -0.7%

Memory: ✅ 42.625MB (SLO: <46.000MB -7.3%) vs baseline: +5.1%

✅ split_noaspect

Time: ✅ 154.135µs (SLO: <250.000µs 📉 -38.3%) vs baseline: +0.6%

Memory: ✅ 42.526MB (SLO: <46.000MB -7.6%) vs baseline: +4.4%

✅ splitlines_aspect

Time: ✅ 148.415µs (SLO: <250.000µs 📉 -40.6%) vs baseline: +1.9%

Memory: ✅ 42.507MB (SLO: <46.000MB -7.6%) vs baseline: +4.5%

✅ splitlines_noaspect

Time: ✅ 150.372µs (SLO: <250.000µs 📉 -39.9%) vs baseline: -0.6%

Memory: ✅ 42.566MB (SLO: <46.000MB -7.5%) vs baseline: +4.9%

✅ iastpropagation - 8/8

✅ no-propagation

Time: ✅ 48.128µs (SLO: <60.000µs 📉 -19.8%) vs baseline: -1.0%

Memory: ✅ 39.007MB (SLO: <42.000MB -7.1%) vs baseline: +5.0%

✅ propagation_enabled

Time: ✅ 136.307µs (SLO: <190.000µs 📉 -28.3%) vs baseline: +0.7%

Memory: ✅ 38.889MB (SLO: <42.000MB -7.4%) vs baseline: +5.0%

✅ propagation_enabled_100

Time: ✅ 1.564ms (SLO: <2.300ms 📉 -32.0%) vs baseline: -0.2%

Memory: ✅ 39.184MB (SLO: <42.000MB -6.7%) vs baseline: +5.7%

✅ propagation_enabled_1000

Time: ✅ 29.206ms (SLO: <34.550ms 📉 -15.5%) vs baseline: +0.4%

Memory: ✅ 39.027MB (SLO: <42.000MB -7.1%) vs baseline: +4.9%

ℹ️ Scenarios Missing SLO Configuration (20 scenarios)

The following scenarios exist in candidate data but have no SLO thresholds configured:

iast_aspects-re_expand_aspect
iast_aspects-re_expand_noaspect
iast_aspects-re_findall_aspect
iast_aspects-re_findall_noaspect
iast_aspects-re_finditer_aspect
iast_aspects-re_finditer_noaspect
iast_aspects-re_fullmatch_aspect
iast_aspects-re_fullmatch_noaspect
iast_aspects-re_group_aspect
iast_aspects-re_group_noaspect
iast_aspects-re_groups_aspect
iast_aspects-re_groups_noaspect
iast_aspects-re_match_aspect
iast_aspects-re_match_noaspect
iast_aspects-re_search_aspect
iast_aspects-re_search_noaspect
iast_aspects-re_sub_aspect
iast_aspects-re_sub_noaspect
iast_aspects-re_subn_aspect
iast_aspects-re_subn_noaspect

brettlangdon

release note lgtm

smola · 2026-02-19T11:13:09Z

@cursor review

avara1986 · 2026-02-19T12:18:47Z

@codex review

chatgpt-codex-connector · 2026-02-19T12:20:52Z

Codex Review: Didn't find any major issues. Bravo.

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Copilot

Pull request overview

This PR fixes a TypeError that occurs when AI Guard service responses return empty or unparsable JSON bodies. The fix defaults the result to an empty dictionary when get_json() returns None, allowing existing error handling logic to properly surface client errors instead of crashing on NoneType operations.

Changes:

Modified _api_client.py to use response.get_json() or {} pattern to safely handle None responses
Added test case to verify HTTP error handling with empty JSON body
Added release note documenting the TypeError fix

Reviewed changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 1 comment.

File	Description
ddtrace/appsec/ai_guard/_api_client.py	Changed response parsing to default to empty dict when get_json() returns None
tests/appsec/ai_guard/api/test_api_client.py	Added test for HTTP error (status 500) with empty JSON body
releasenotes/notes/fix-ai-guard-empty-result-9ceb9de92d746a86.yaml	Added release note documenting the TypeError fix

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-02-19T12:21:47Z

tests/appsec/ai_guard/api/test_api_client.py

+def test_evaluate_http_error_empty_json_body(mock_execute_request, telemetry_mock, ai_guard_client):
+    """Test HTTP error handling when the response body is empty."""
+    mock_response = Mock()
+    mock_response.status = 500
+    mock_response.get_json.return_value = None
+    mock_execute_request.return_value = mock_response
+
+    with pytest.raises(AIGuardClientError) as exc_info:
+        ai_guard_client.evaluate(TOOL_CALL)
+
+    assert str(exc_info.value) == "AI Guard service call failed, status: 500"
+    assert exc_info.value.status == 500
+    assert exc_info.value.errors == []
+    assert_telemetry(telemetry_mock, "ai_guard.requests", (("error", "true"),))


The test only covers the case when status is 500 with an empty JSON body. Consider adding a test case for status 200 with an empty JSON body to ensure that the malformed response error handler works correctly in this scenario. While the existing try-except block at line 250 in the source code should handle this case properly, having an explicit test would provide better coverage and documentation of this edge case.

datadog-datadog-prod-us1 bot added the Bits AI label Feb 18, 2026

smola requested a review from a team as a code owner February 18, 2026 13:03

smola added the Bits AI label Feb 18, 2026

smola changed the title ~~Handle empty AI Guard response~~ fix(aiguard): handle empty AI Guard response on error Feb 18, 2026

smola added the ASM Application Security Monitoring label Feb 18, 2026

smola force-pushed the dd/fix/ai-guard-empty-response branch from 24f2b44 to 7245eb5 Compare February 18, 2026 14:17

smola requested a review from a team as a code owner February 18, 2026 14:17

smola requested review from Kyle-Verhoog and brettlangdon February 18, 2026 14:17

brettlangdon approved these changes Feb 18, 2026

View reviewed changes

This comment has been minimized.

Sign in to view

fix(aiguard): handle empty AI Guard response on error

147af46

smola force-pushed the dd/fix/ai-guard-empty-response branch from 7245eb5 to 147af46 Compare February 19, 2026 08:55

avara1986 requested a review from Copilot February 19, 2026 12:18

Copilot started reviewing on behalf of avara1986 February 19, 2026 12:19 View session

avara1986 approved these changes Feb 19, 2026

View reviewed changes

Copilot AI reviewed Feb 19, 2026

View reviewed changes

Kyle-Verhoog approved these changes Feb 19, 2026

View reviewed changes

Comments

Conversation

smola commented Feb 18, 2026 • edited by atlassian bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Testing

Risks

Additional Notes

Uh oh!

datadog-official bot commented Feb 18, 2026

Uh oh!

cit-pr-commenter-54b7da bot commented Feb 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codeowners resolved as

Uh oh!

pr-commenter bot commented Feb 18, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Performance SLOs

✅ ospathbasename_aspect

✅ ospathbasename_noaspect

✅ ospathjoin_aspect

✅ ospathjoin_noaspect

✅ ospathnormcase_aspect

✅ ospathnormcase_noaspect

✅ ospathsplit_aspect

✅ ospathsplit_noaspect

✅ ospathsplitdrive_aspect

✅ ospathsplitdrive_noaspect

✅ ospathsplitext_aspect

✅ ospathsplitext_noaspect

✅ rsplit_aspect

✅ rsplit_noaspect

✅ split_aspect

✅ split_noaspect

✅ splitlines_aspect

✅ splitlines_noaspect

✅ no-propagation

✅ propagation_enabled

✅ propagation_enabled_100

✅ propagation_enabled_1000

Uh oh!

brettlangdon left a comment

Choose a reason for hiding this comment

Uh oh!

This comment has been minimized.

smola commented Feb 19, 2026

Uh oh!

avara1986 commented Feb 19, 2026

Uh oh!

chatgpt-codex-connector bot commented Feb 19, 2026

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Feb 19, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

smola commented Feb 18, 2026 •

edited by atlassian bot

Loading

cit-pr-commenter-54b7da bot commented Feb 18, 2026 •

edited

Loading

pr-commenter bot commented Feb 18, 2026 •

edited

Loading