Conversation
Co-authored-by: RamyElkest <1733553+RamyElkest@users.noreply.github.com>
Codeowners resolved as |
|
@codex Rename AIGuardStrandsHookProvider and similar to AIGuardHook. |
|
To use Codex here, create a Codex account and connect to github. |
|
@codex review |
|
Codex Review: Didn't find any major issues. Keep them coming! ℹ️ About Codex in GitHubCodex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
If Codex has suggestions, it will comment; otherwise it will react with 👍. When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback". |
|
✨ Fix all issues with BitsAI or with Cursor
|
Performance SLOsComparing candidate dd/ramy.elkest/ai-guard-strands-hooks (174cab3) with baseline main (2819240) 📈 Performance Regressions (2 suites)📈 iastaspects - 118/118✅ add_aspectTime: ✅ 102.894µs (SLO: <130.000µs 📉 -20.9%) vs baseline: +1.4% Memory: ✅ 42.939MB (SLO: <46.000MB -6.7%) vs baseline: +4.4% ✅ add_inplace_aspectTime: ✅ 100.930µs (SLO: <130.000µs 📉 -22.4%) vs baseline: -2.2% Memory: ✅ 42.900MB (SLO: <46.000MB -6.7%) vs baseline: +4.2% ✅ add_inplace_noaspectTime: ✅ 28.406µs (SLO: <40.000µs 📉 -29.0%) vs baseline: +0.4% Memory: ✅ 42.939MB (SLO: <46.000MB -6.7%) vs baseline: +4.3% ✅ add_noaspectTime: ✅ 49.274µs (SLO: <70.000µs 📉 -29.6%) vs baseline: +0.6% Memory: ✅ 42.841MB (SLO: <46.000MB -6.9%) vs baseline: +4.1% ✅ bytearray_aspectTime: ✅ 251.954µs (SLO: <400.000µs 📉 -37.0%) vs baseline: +1.6% Memory: ✅ 42.880MB (SLO: <46.000MB -6.8%) vs baseline: +4.2% ✅ bytearray_extend_aspectTime: ✅ 631.609µs (SLO: <800.000µs 📉 -21.0%) vs baseline: -3.2% Memory: ✅ 42.979MB (SLO: <46.000MB -6.6%) vs baseline: +4.4% ✅ bytearray_extend_noaspectTime: ✅ 263.113µs (SLO: <400.000µs 📉 -34.2%) vs baseline: -3.3% Memory: ✅ 42.880MB (SLO: <46.000MB -6.8%) vs baseline: +4.3% ✅ bytearray_noaspectTime: ✅ 136.416µs (SLO: <300.000µs 📉 -54.5%) vs baseline: -3.4% Memory: ✅ 42.841MB (SLO: <46.000MB -6.9%) vs baseline: +4.2% ✅ bytes_aspectTime: ✅ 217.617µs (SLO: <300.000µs 📉 -27.5%) vs baseline: ~same Memory: ✅ 42.979MB (SLO: <46.000MB -6.6%) vs baseline: +4.4% ✅ bytes_noaspectTime: ✅ 133.912µs (SLO: <200.000µs 📉 -33.0%) vs baseline: -0.3% Memory: ✅ 42.861MB (SLO: <46.000MB -6.8%) vs baseline: +4.4% ✅ bytesio_aspectTime: ✅ 3.788ms (SLO: <5.000ms 📉 -24.2%) vs baseline: -0.4% Memory: ✅ 42.900MB (SLO: <46.000MB -6.7%) vs baseline: +4.4% ✅ bytesio_noaspectTime: ✅ 314.207µs (SLO: <420.000µs 📉 -25.2%) vs baseline: -0.5% Memory: ✅ 42.900MB (SLO: <46.000MB -6.7%) vs baseline: +4.3% ✅ capitalize_aspectTime: ✅ 89.543µs (SLO: <300.000µs 📉 -70.2%) vs baseline: +0.9% Memory: ✅ 42.900MB (SLO: <46.000MB -6.7%) vs baseline: +4.4% ✅ capitalize_noaspectTime: ✅ 251.347µs (SLO: <300.000µs 📉 -16.2%) vs baseline: -0.6% Memory: ✅ 42.900MB (SLO: <46.000MB -6.7%) vs baseline: +4.3% ✅ casefold_aspectTime: ✅ 89.257µs (SLO: <500.000µs 📉 -82.1%) vs baseline: +1.2% Memory: ✅ 42.959MB (SLO: <46.000MB -6.6%) vs baseline: +4.3% ✅ casefold_noaspectTime: ✅ 308.129µs (SLO: <500.000µs 📉 -38.4%) vs baseline: -1.9% Memory: ✅ 42.939MB (SLO: <46.000MB -6.7%) vs baseline: +4.6% ✅ decode_aspectTime: ✅ 86.621µs (SLO: <100.000µs 📉 -13.4%) vs baseline: +0.2% Memory: ✅ 42.880MB (SLO: <46.000MB -6.8%) vs baseline: +4.2% ✅ decode_noaspectTime: ✅ 151.735µs (SLO: <210.000µs 📉 -27.7%) vs baseline: -1.6% Memory: ✅ 42.900MB (SLO: <46.000MB -6.7%) vs baseline: +4.1% ✅ encode_aspectTime: ✅ 84.277µs (SLO: <200.000µs 📉 -57.9%) vs baseline: +0.1% Memory: ✅ 42.900MB (SLO: <46.000MB -6.7%) vs baseline: +4.1% ✅ encode_noaspectTime: ✅ 139.487µs (SLO: <200.000µs 📉 -30.3%) vs baseline: -2.3% Memory: ✅ 42.959MB (SLO: <46.000MB -6.6%) vs baseline: +4.6% ✅ format_aspectTime: ✅ 14.714ms (SLO: <19.200ms 📉 -23.4%) vs baseline: +0.6% Memory: ✅ 43.037MB (SLO: <46.000MB -6.4%) vs baseline: +4.4% ✅ format_map_aspectTime: ✅ 16.454ms (SLO: <21.500ms 📉 -23.5%) vs baseline: ~same Memory: ✅ 42.959MB (SLO: <46.000MB -6.6%) vs baseline: +4.2% ✅ format_map_noaspectTime: ✅ 374.962µs (SLO: <500.000µs 📉 -25.0%) vs baseline: +0.2% Memory: ✅ 42.880MB (SLO: <46.000MB -6.8%) vs baseline: +4.3% ✅ format_noaspectTime: ✅ 301.623µs (SLO: <500.000µs 📉 -39.7%) vs baseline: -0.9% Memory: ✅ 42.880MB (SLO: <46.000MB -6.8%) vs baseline: +4.3% ✅ index_aspectTime: ✅ 124.958µs (SLO: <300.000µs 📉 -58.3%) vs baseline: +3.3% Memory: ✅ 42.979MB (SLO: <46.000MB -6.6%) vs baseline: +4.5% ✅ index_noaspectTime: ✅ 40.376µs (SLO: <300.000µs 📉 -86.5%) vs baseline: +0.4% Memory: ✅ 42.821MB (SLO: <46.000MB -6.9%) vs baseline: +4.1% ✅ join_aspectTime: ✅ 211.016µs (SLO: <300.000µs 📉 -29.7%) vs baseline: -1.2% Memory: ✅ 42.861MB (SLO: <46.000MB -6.8%) vs baseline: +4.2% ✅ join_noaspectTime: ✅ 143.841µs (SLO: <300.000µs 📉 -52.1%) vs baseline: +1.1% Memory: ✅ 42.880MB (SLO: <46.000MB -6.8%) vs baseline: +4.2% ✅ ljust_aspectTime: ✅ 581.619µs (SLO: <700.000µs 📉 -16.9%) vs baseline: 📈 +14.7% Memory: ✅ 42.900MB (SLO: <46.000MB -6.7%) vs baseline: +4.3% ✅ ljust_noaspectTime: ✅ 256.153µs (SLO: <300.000µs 📉 -14.6%) vs baseline: -1.8% Memory: ✅ 42.939MB (SLO: <46.000MB -6.7%) vs baseline: +4.4% ✅ lower_aspectTime: ✅ 292.275µs (SLO: <500.000µs 📉 -41.5%) vs baseline: -2.7% Memory: ✅ 42.939MB (SLO: <46.000MB -6.7%) vs baseline: +4.4% ✅ lower_noaspectTime: ✅ 236.186µs (SLO: <300.000µs 📉 -21.3%) vs baseline: -0.1% Memory: ✅ 42.880MB (SLO: <46.000MB -6.8%) vs baseline: +4.4% ✅ lstrip_aspectTime: ✅ 0.270ms (SLO: <3.000ms 📉 -91.0%) vs baseline: -3.3% Memory: ✅ 42.861MB (SLO: <46.000MB -6.8%) vs baseline: +4.1% ✅ lstrip_noaspectTime: ✅ 0.178ms (SLO: <3.000ms 📉 -94.1%) vs baseline: +0.7% Memory: ✅ 42.920MB (SLO: <46.000MB -6.7%) vs baseline: +4.6% ✅ modulo_aspectTime: ✅ 14.256ms (SLO: <18.750ms 📉 -24.0%) vs baseline: -0.5% Memory: ✅ 42.959MB (SLO: <46.000MB -6.6%) vs baseline: +4.1% ✅ modulo_aspect_for_bytearray_bytearrayTime: ✅ 14.807ms (SLO: <19.350ms 📉 -23.5%) vs baseline: +0.3% Memory: ✅ 43.096MB (SLO: <46.000MB -6.3%) vs baseline: +4.4% ✅ modulo_aspect_for_bytesTime: ✅ 14.375ms (SLO: <18.900ms 📉 -23.9%) vs baseline: -0.1% Memory: ✅ 43.057MB (SLO: <46.000MB -6.4%) vs baseline: +4.4% ✅ modulo_aspect_for_bytes_bytearrayTime: ✅ 14.557ms (SLO: <19.150ms 📉 -24.0%) vs baseline: -1.1% Memory: ✅ 42.979MB (SLO: <46.000MB -6.6%) vs baseline: +4.3% ✅ modulo_noaspectTime: ✅ 0.359ms (SLO: <3.000ms 📉 -88.0%) vs baseline: -0.7% Memory: ✅ 42.998MB (SLO: <46.000MB -6.5%) vs baseline: +4.6% ✅ replace_aspectTime: ✅ 18.444ms (SLO: <24.000ms 📉 -23.1%) vs baseline: +0.3% Memory: ✅ 42.998MB (SLO: <46.000MB -6.5%) vs baseline: +4.4% ✅ replace_noaspectTime: ✅ 279.578µs (SLO: <300.000µs -6.8%) vs baseline: -1.1% Memory: ✅ 42.939MB (SLO: <46.000MB -6.7%) vs baseline: +4.4% ✅ repr_aspectTime: ✅ 310.441µs (SLO: <420.000µs 📉 -26.1%) vs baseline: -3.9% Memory: ✅ 42.900MB (SLO: <46.000MB -6.7%) vs baseline: +4.2% ✅ repr_noaspectTime: ✅ 46.776µs (SLO: <90.000µs 📉 -48.0%) vs baseline: +0.5% Memory: ✅ 42.900MB (SLO: <46.000MB -6.7%) vs baseline: +4.6% ✅ rstrip_aspectTime: ✅ 380.755µs (SLO: <500.000µs 📉 -23.8%) vs baseline: -1.2% Memory: ✅ 42.880MB (SLO: <46.000MB -6.8%) vs baseline: +4.4% ✅ rstrip_noaspectTime: ✅ 184.223µs (SLO: <300.000µs 📉 -38.6%) vs baseline: +0.6% Memory: ✅ 42.841MB (SLO: <46.000MB -6.9%) vs baseline: +4.5% ✅ slice_aspectTime: ✅ 184.100µs (SLO: <300.000µs 📉 -38.6%) vs baseline: +1.4% Memory: ✅ 42.861MB (SLO: <46.000MB -6.8%) vs baseline: +4.2% ✅ slice_noaspectTime: ✅ 53.747µs (SLO: <90.000µs 📉 -40.3%) vs baseline: -0.6% Memory: ✅ 42.880MB (SLO: <46.000MB -6.8%) vs baseline: +4.1% ✅ stringio_aspectTime: ✅ 4.418ms (SLO: <5.000ms 📉 -11.6%) vs baseline: 📈 +14.2% Memory: ✅ 42.900MB (SLO: <46.000MB -6.7%) vs baseline: +4.6% ✅ stringio_noaspectTime: ✅ 347.228µs (SLO: <500.000µs 📉 -30.6%) vs baseline: -0.8% Memory: ✅ 42.861MB (SLO: <46.000MB -6.8%) vs baseline: +4.3% ✅ strip_aspectTime: ✅ 270.681µs (SLO: <350.000µs 📉 -22.7%) vs baseline: -1.6% Memory: ✅ 42.900MB (SLO: <46.000MB -6.7%) vs baseline: +4.4% ✅ strip_noaspectTime: ✅ 175.991µs (SLO: <240.000µs 📉 -26.7%) vs baseline: -0.8% Memory: ✅ 42.939MB (SLO: <46.000MB -6.7%) vs baseline: +4.3% ✅ swapcase_aspectTime: ✅ 333.782µs (SLO: <500.000µs 📉 -33.2%) vs baseline: -2.0% Memory: ✅ 42.821MB (SLO: <46.000MB -6.9%) vs baseline: +4.1% ✅ swapcase_noaspectTime: ✅ 270.009µs (SLO: <400.000µs 📉 -32.5%) vs baseline: -0.9% Memory: ✅ 42.979MB (SLO: <46.000MB -6.6%) vs baseline: +4.4% ✅ title_aspectTime: ✅ 317.759µs (SLO: <500.000µs 📉 -36.4%) vs baseline: -4.4% Memory: ✅ 42.900MB (SLO: <46.000MB -6.7%) vs baseline: +4.3% ✅ title_noaspectTime: ✅ 257.031µs (SLO: <400.000µs 📉 -35.7%) vs baseline: -1.3% Memory: ✅ 42.939MB (SLO: <46.000MB -6.7%) vs baseline: +4.4% ✅ translate_aspectTime: ✅ 493.561µs (SLO: <700.000µs 📉 -29.5%) vs baseline: -1.2% Memory: ✅ 42.802MB (SLO: <46.000MB -7.0%) vs baseline: +4.1% ✅ translate_noaspectTime: ✅ 422.539µs (SLO: <500.000µs 📉 -15.5%) vs baseline: -1.9% Memory: ✅ 42.880MB (SLO: <46.000MB -6.8%) vs baseline: +4.1% ✅ upper_aspectTime: ✅ 296.345µs (SLO: <500.000µs 📉 -40.7%) vs baseline: -0.8% Memory: ✅ 42.861MB (SLO: <46.000MB -6.8%) vs baseline: +4.2% ✅ upper_noaspectTime: ✅ 236.124µs (SLO: <400.000µs 📉 -41.0%) vs baseline: +0.3% Memory: ✅ 42.920MB (SLO: <46.000MB -6.7%) vs baseline: +4.4% 📈 iastaspectsospath - 24/24✅ ospathbasename_aspectTime: ✅ 508.476µs (SLO: <700.000µs 📉 -27.4%) vs baseline: 📈 +20.1% Memory: ✅ 42.802MB (SLO: <46.000MB -7.0%) vs baseline: +4.9% ✅ ospathbasename_noaspectTime: ✅ 429.515µs (SLO: <700.000µs 📉 -38.6%) vs baseline: -0.4% Memory: ✅ 42.546MB (SLO: <46.000MB -7.5%) vs baseline: +4.4% ✅ ospathjoin_aspectTime: ✅ 627.433µs (SLO: <700.000µs 📉 -10.4%) vs baseline: +0.9% Memory: ✅ 42.546MB (SLO: <46.000MB -7.5%) vs baseline: +4.2% ✅ ospathjoin_noaspectTime: ✅ 634.606µs (SLO: <700.000µs -9.3%) vs baseline: +0.5% Memory: ✅ 42.684MB (SLO: <46.000MB -7.2%) vs baseline: +4.3% ✅ ospathnormcase_aspectTime: ✅ 348.944µs (SLO: <700.000µs 📉 -50.2%) vs baseline: -1.1% Memory: ✅ 42.743MB (SLO: <46.000MB -7.1%) vs baseline: +4.8% ✅ ospathnormcase_noaspectTime: ✅ 358.039µs (SLO: <700.000µs 📉 -48.9%) vs baseline: +0.2% Memory: ✅ 42.467MB (SLO: <46.000MB -7.7%) vs baseline: +3.9% ✅ ospathsplit_aspectTime: ✅ 486.755µs (SLO: <700.000µs 📉 -30.5%) vs baseline: -1.0% Memory: ✅ 42.467MB (SLO: <46.000MB -7.7%) vs baseline: +4.3% ✅ ospathsplit_noaspectTime: ✅ 499.600µs (SLO: <700.000µs 📉 -28.6%) vs baseline: -0.5% Memory: ✅ 42.546MB (SLO: <46.000MB -7.5%) vs baseline: +4.2% ✅ ospathsplitdrive_aspectTime: ✅ 375.926µs (SLO: <700.000µs 📉 -46.3%) vs baseline: +0.3% Memory: ✅ 42.526MB (SLO: <46.000MB -7.6%) vs baseline: +4.4% ✅ ospathsplitdrive_noaspectTime: ✅ 73.039µs (SLO: <700.000µs 📉 -89.6%) vs baseline: -0.4% Memory: ✅ 42.585MB (SLO: <46.000MB -7.4%) vs baseline: +4.5% ✅ ospathsplitext_aspectTime: ✅ 459.423µs (SLO: <700.000µs 📉 -34.4%) vs baseline: ~same Memory: ✅ 42.664MB (SLO: <46.000MB -7.3%) vs baseline: +4.2% ✅ ospathsplitext_noaspectTime: ✅ 464.469µs (SLO: <700.000µs 📉 -33.6%) vs baseline: -1.4% Memory: ✅ 42.644MB (SLO: <46.000MB -7.3%) vs baseline: +4.6%
|
PR by Bits
View session in Datadog
Comment @DataDog to request changes
Feedback (especially what can be better) welcome in #code-gen-aka-bits-dev-feedback!
Description
Add AI Guard integration with AWS Strands Agents SDK as a
HookProvider. This enables developers to secure their Strands-based AI agents by evaluating prompts, model responses, and tool calls against Datadog AI Guard security policies.Usage:
The hook provider implements three lifecycle hooks:
before_model_invocation— Scans user prompts before sending to the LLMafter_model_invocation— Scans model responses for policy violationsbefore_tool_invocation— Scans tool calls before executionWhen a policy violation is detected and blocking is enabled,
AIGuardAbortErroris raised to prevent the operation.The implementation converts Strands messages (Bedrock Converse API format) to AI Guard's message format and gracefully handles the
strands-agentsSDK being an optional dependency.Ref: APPSEC-61346
Testing
before_model_invocation,after_model_invocation,before_tool_invocationwith allow/block scenariosai_guard_strandssuite)Risks
**kwargspattern) and message format (Bedrock Converse API) are based on the documented Strands hooks interface and the existing Bedrock integration in dd-trace-py. Method signatures may need adjustment once validated against the actual SDK.Additional Notes
strands.agent.hooks.HookProviderwhen the SDK is installed, falling back toobjectfor environments without ittext,toolUse,toolResult(including JSON sub-content)ddtrace.appsec.ai_guardSDK moduleAPPSEC-61346