Detect OWASP project level non-compliance and adjust health scores #3354
Conversation
Summary by CodeRabbit
✏️ Tip: You can customize this high-level summary in your review settings. WalkthroughAdds a boolean flag to ProjectHealthMetrics for OWASP level non-compliance, a project-level mapping utility, a command to fetch official project levels and mark non-compliant projects, updates health-score calculation to apply a penalty, triggers these commands after initial aggregation, and includes migration and tests. Changes
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly related PRs
Suggested reviewers
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing touches
🧪 Generate unit tests (beta)
📜 Recent review detailsConfiguration used: Path: .coderabbit.yaml Review profile: CHILL Plan: Pro 📒 Files selected for processing (1)
🚧 Files skipped from review as they are similar to previous changes (1)
✏️ Tip: You can disable this entire section by setting Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Fix all issues with AI agents
In
`@backend/apps/owasp/management/commands/owasp_update_project_level_compliance.py`:
- Around line 74-76: The command currently queries all ProjectHealthMetrics
(metrics = ProjectHealthMetrics.objects.select_related("project")) which
includes historical records; replace this with only the latest metrics by
calling ProjectHealthMetrics.get_latest_health_metrics(). Ensure you still use
select_related("project") (e.g.,
ProjectHealthMetrics.get_latest_health_metrics().select_related("project")) and
then iterate/update that queryset (updated_metrics variable) so compliance
checks run only against each project's most recent metric.
🧹 Nitpick comments (9)
backend/apps/owasp/utils/project_level.py (2)
36-39: Consider removing redundant Decimal re-parsing.The function signature declares
level: Decimal, but the implementation converts it viaDecimal(str(level)). This double-conversion is defensive but unnecessary if callers honor the type hint. The current approach does provide robustness against duck-typed inputs at runtime.If intentional for safety against untyped callers, consider adding a brief inline comment explaining the defensive conversion.
44-44: Static analysis hint: consider quoting type expression incast().Ruff TC006 suggests adding quotes:
cast("ProjectLevel | None", ...). This is a minor typing style preference for forward-compatibility.♻️ Optional fix
- return cast(ProjectLevel | None, _LEVEL_MAP.get(parsed_level)) + return cast("ProjectLevel | None", _LEVEL_MAP.get(parsed_level))backend/tests/apps/owasp/management/commands/owasp_update_project_health_scores_test.py (1)
87-160: Good test coverage for non-compliance penalty, but consider verifying penalty magnitude.The test correctly validates that:
- Bulk save is invoked
- Score remains non-negative (clamped)
- The non-compliance flag persists
- Output includes the project name
However, the test doesn't verify that a penalty was actually applied. Consider capturing the score value and comparing against an expected value (base score minus
LEVEL_NON_COMPLIANCE_PENALTY) to ensure the penalty logic is exercised, not just that the command completes.💡 Optional enhancement
from apps.owasp.management.commands.owasp_update_project_health_scores import ( LEVEL_NON_COMPLIANCE_PENALTY, ) # After command execution, verify penalty was applied: # Calculate expected base score from test data, then: # assert mock_metric.score == expected_base_score - LEVEL_NON_COMPLIANCE_PENALTYbackend/tests/apps/owasp/utils/project_level_test.py (1)
17-43: Consider adding edge case tests for improved coverage.The tests cover the main happy paths well. However, consider adding tests for:
- Invalid input types (e.g.,
None, non-numeric strings)- Unmapped positive decimal values (e.g.,
Decimal("2.5")should returnNone)These edge cases are handled in
map_levelbut aren't verified by tests.🧪 Suggested additional tests
+ def test_unmapped_level_returns_none(self): + """Unmapped positive levels should return None.""" + assert map_level(Decimal("2.5")) is None + assert map_level(Decimal("5")) is None + + def test_invalid_input_returns_none(self): + """Invalid inputs should return None.""" + assert map_level(None) is None + assert map_level("invalid") is Nonebackend/tests/apps/owasp/management/commands/owasp_update_project_level_compliance_test.py (3)
29-39: Mockproject.owasp_urlto ensure consistent test behavior.The test relies on name-based lookup since
project.owasp_urlisn't mocked and may evaluate to a falsy/truthy MagicMock. Explicitly settingmetric.project.owasp_url = Noneensures the test consistently exercises the name-based fallback path.♻️ Suggested fix
metric = MagicMock() metric.project.name = "Test Project" metric.project.level = ProjectLevel.PRODUCTION + metric.project.owasp_url = None metric.level_non_compliant = False
60-69: Same improvement: mockproject.owasp_urlfor deterministic behavior.For consistency with the first test, explicitly set
metric.project.owasp_url = None.♻️ Suggested fix
metric = MagicMock() metric.project.name = "Test Project" metric.project.level = ProjectLevel.PRODUCTION + metric.project.owasp_url = None metric.level_non_compliant = False
9-70: Consider adding tests for error handling and edge cases.The current tests cover the main compliance detection logic well. For more robust coverage, consider adding tests for:
- HTTP error responses (e.g.,
status_code=500orraise_for_status()throwing)- Projects not found in official data (should be skipped without error)
- Invalid level values in the API response
backend/apps/owasp/management/commands/owasp_update_project_level_compliance.py (2)
91-96: Add change detection to avoid unnecessary database writes.The current implementation adds metrics to
updated_metricseven whenlevel_non_complianthasn't changed. This causes unnecessary database operations.♻️ Suggested fix
expected_level = map_level(official_level) if expected_level is None: continue - metric.level_non_compliant = project.level != expected_level - updated_metrics.append(metric) + is_non_compliant = project.level != expected_level + if metric.level_non_compliant != is_non_compliant: + metric.level_non_compliant = is_non_compliant + updated_metrics.append(metric)
54-56: Consider adding error handling for network failures.The HTTP request can fail due to network issues, timeouts, or server errors. For a periodic background job, wrapping this in try/except with appropriate logging would improve resilience.
♻️ Suggested fix
- response = requests.get(LEVELS_URL, timeout=15) - response.raise_for_status() - official_data = response.json() + try: + response = requests.get(LEVELS_URL, timeout=15) + response.raise_for_status() + official_data = response.json() + except requests.RequestException as exc: + self.stderr.write(self.style.ERROR(f"Failed to fetch official levels: {exc}")) + return
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (10)
backend/apps/owasp/management/commands/owasp_aggregate_projects.pybackend/apps/owasp/management/commands/owasp_update_project_health_scores.pybackend/apps/owasp/management/commands/owasp_update_project_level_compliance.pybackend/apps/owasp/migrations/0070_projecthealthmetrics_level_non_compliant.pybackend/apps/owasp/models/project_health_metrics.pybackend/apps/owasp/utils/project_level.pybackend/tests/apps/owasp/management/commands/owasp_aggregate_projects_test.pybackend/tests/apps/owasp/management/commands/owasp_update_project_health_scores_test.pybackend/tests/apps/owasp/management/commands/owasp_update_project_level_compliance_test.pybackend/tests/apps/owasp/utils/project_level_test.py
🧰 Additional context used
🧠 Learnings (5)
📚 Learning: 2025-12-18T05:39:42.678Z
Learnt from: rudransh-shrivastava
Repo: OWASP/Nest PR: 2948
File: backend/apps/owasp/management/commands/owasp_generate_community_snapshot_video.py:40-40
Timestamp: 2025-12-18T05:39:42.678Z
Learning: In Django management commands, prefer using self.stdout.write(...) over print(...) for user-facing stdout output. This aligns with Django conventions and improves testability. When emitting messages, consider using self.stdout.write and, for styled messages, use self.style.SUCCESS/ERROR as appropriate to maintain consistent command output formatting. Apply this guideline to all Python files within any project's management/commands directory.
Applied to files:
backend/apps/owasp/management/commands/owasp_aggregate_projects.pybackend/apps/owasp/management/commands/owasp_update_project_health_scores.pybackend/tests/apps/owasp/management/commands/owasp_update_project_health_scores_test.pybackend/tests/apps/owasp/management/commands/owasp_update_project_level_compliance_test.pybackend/tests/apps/owasp/management/commands/owasp_aggregate_projects_test.pybackend/apps/owasp/management/commands/owasp_update_project_level_compliance.py
📚 Learning: 2025-12-31T05:17:39.659Z
Learnt from: kart-u
Repo: OWASP/Nest PR: 3101
File: backend/apps/common/extensions.py:92-98
Timestamp: 2025-12-31T05:17:39.659Z
Learning: In this codebase, import OperationType for GraphQL operations from the graphql-core package rather than from strawberry. Use 'from graphql import OperationType'. Strawberry re-exports via graphql-core internally, so relying on strawberry's API may be brittle. Apply this rule to all Python files that deal with GraphQL operation types; ensure imports come from graphql (graphql-core) and not from strawberry packages. This improves compatibility and avoids coupling to strawberry's internals.
Applied to files:
backend/apps/owasp/management/commands/owasp_aggregate_projects.pybackend/apps/owasp/management/commands/owasp_update_project_health_scores.pybackend/apps/owasp/migrations/0070_projecthealthmetrics_level_non_compliant.pybackend/apps/owasp/utils/project_level.pybackend/tests/apps/owasp/utils/project_level_test.pybackend/tests/apps/owasp/management/commands/owasp_update_project_health_scores_test.pybackend/tests/apps/owasp/management/commands/owasp_update_project_level_compliance_test.pybackend/tests/apps/owasp/management/commands/owasp_aggregate_projects_test.pybackend/apps/owasp/management/commands/owasp_update_project_level_compliance.pybackend/apps/owasp/models/project_health_metrics.py
📚 Learning: 2026-01-01T17:48:23.963Z
Learnt from: rudransh-shrivastava
Repo: OWASP/Nest PR: 2948
File: backend/apps/owasp/management/commands/owasp_generate_community_snapshot_video.py:41-47
Timestamp: 2026-01-01T17:48:23.963Z
Learning: In Django code, be aware that a QuerySet's boolean evaluation (e.g., if not queryset) runs a database query to determine emptiness. While it is technically valid to use the queryset in a boolean context, use queryset.exists() for existence checks to avoid unnecessary queries and improve performance. Applicable broadly to Python/Django files rather than just this specific path.
Applied to files:
backend/apps/owasp/management/commands/owasp_aggregate_projects.pybackend/apps/owasp/management/commands/owasp_update_project_health_scores.pybackend/apps/owasp/migrations/0070_projecthealthmetrics_level_non_compliant.pybackend/apps/owasp/utils/project_level.pybackend/tests/apps/owasp/utils/project_level_test.pybackend/tests/apps/owasp/management/commands/owasp_update_project_health_scores_test.pybackend/tests/apps/owasp/management/commands/owasp_update_project_level_compliance_test.pybackend/tests/apps/owasp/management/commands/owasp_aggregate_projects_test.pybackend/apps/owasp/management/commands/owasp_update_project_level_compliance.pybackend/apps/owasp/models/project_health_metrics.py
📚 Learning: 2026-01-01T18:57:05.007Z
Learnt from: rudransh-shrivastava
Repo: OWASP/Nest PR: 2948
File: backend/apps/owasp/video.py:189-215
Timestamp: 2026-01-01T18:57:05.007Z
Learning: In the OWASP backend area, maintain the established pattern: when dealing with sponsors, include all entries from Sponsor.objects.all() (including NOT_SPONSOR) and perform in-memory sorting using the same criteria/pattern used by the GraphQL sponsor query implemented in backend/apps/owasp/api/internal/queries/sponsor.py. Apply this behavior consistently to files in backend/apps/owasp (not just video.py), and ensure code paths that render sponsor lists follow this in-code sorting approach rather than pre-filtering NOT_SPONSOR entries before sorting.
Applied to files:
backend/apps/owasp/management/commands/owasp_aggregate_projects.pybackend/apps/owasp/management/commands/owasp_update_project_health_scores.pybackend/apps/owasp/migrations/0070_projecthealthmetrics_level_non_compliant.pybackend/apps/owasp/utils/project_level.pybackend/apps/owasp/management/commands/owasp_update_project_level_compliance.pybackend/apps/owasp/models/project_health_metrics.py
📚 Learning: 2026-01-06T12:38:23.460Z
Learnt from: ahmedxgouda
Repo: OWASP/Nest PR: 3122
File: backend/apps/owasp/api/internal/nodes/project.py:52-61
Timestamp: 2026-01-06T12:38:23.460Z
Learning: In backend/apps/owasp/api/internal/nodes/project.py, the health_metrics_list method intentionally orders by nest_created_at ascending (oldest first) to return metrics in chronological order for trend analysis, which differs from health_metrics_latest that orders descending to get the most recent metric.
Applied to files:
backend/apps/owasp/management/commands/owasp_update_project_health_scores.pybackend/apps/owasp/models/project_health_metrics.py
🧬 Code graph analysis (5)
backend/apps/owasp/management/commands/owasp_update_project_health_scores.py (2)
backend/apps/owasp/models/project_health_metrics.py (1)
ProjectHealthMetrics(16-240)backend/apps/owasp/models/project_health_requirements.py (1)
ProjectHealthRequirements(9-63)
backend/apps/owasp/utils/project_level.py (1)
backend/apps/owasp/models/enums/project.py (1)
ProjectLevel(37-44)
backend/tests/apps/owasp/utils/project_level_test.py (2)
backend/apps/owasp/models/enums/project.py (1)
ProjectLevel(37-44)backend/apps/owasp/utils/project_level.py (1)
map_level(26-44)
backend/tests/apps/owasp/management/commands/owasp_update_project_level_compliance_test.py (2)
backend/apps/owasp/models/enums/project.py (1)
ProjectLevel(37-44)backend/tests/apps/owasp/management/commands/owasp_aggregate_contributions_test.py (1)
select_related(36-38)
backend/apps/owasp/management/commands/owasp_update_project_level_compliance.py (3)
backend/apps/owasp/models/project_health_metrics.py (1)
ProjectHealthMetrics(16-240)backend/apps/owasp/utils/project_level.py (1)
map_level(26-44)backend/apps/owasp/management/commands/owasp_aggregate_projects.py (2)
Command(9-124)handle(21-124)
🪛 Ruff (0.14.11)
backend/apps/owasp/management/commands/owasp_update_project_health_scores.py
1-9: Multi-line docstring summary should start at the first line
Remove whitespace after opening quotes
(D212)
20-27: Multi-line docstring summary should start at the first line
Remove whitespace after opening quotes
(D212)
32-39: Multi-line docstring summary should start at the first line
Remove whitespace after opening quotes
(D212)
backend/apps/owasp/utils/project_level.py
1-8: 1 blank line required between summary line and description
(D205)
1-8: Multi-line docstring summary should start at the first line
Remove whitespace after opening quotes
(D212)
32-32: Missing blank line after last section ("Returns")
Add blank line after "Returns"
(D413)
44-44: Add quotes to type expression in typing.cast()
Add quotes
(TC006)
backend/tests/apps/owasp/utils/project_level_test.py
1-7: Multi-line docstring summary should start at the first line
Remove whitespace after opening quotes
(D212)
backend/apps/owasp/management/commands/owasp_update_project_level_compliance.py
1-12: Multi-line docstring summary should start at the first line
Remove whitespace after opening quotes
(D212)
34-41: Multi-line docstring summary should start at the first line
Remove whitespace after opening quotes
(D212)
39-39: Docstring contains ambiguous ’ (RIGHT SINGLE QUOTATION MARK). Did you mean ``` (GRAVE ACCENT)?
(RUF002)
46-53: Multi-line docstring summary should start at the first line
Remove whitespace after opening quotes
(D212)
🔇 Additional comments (11)
backend/apps/owasp/utils/project_level.py (1)
16-23: LGTM! Clear mapping from OWASP levels to internal enum.The mapping correctly handles both integer and fractional levels (3.5 for flagship), covering all ProjectLevel enum values.
backend/apps/owasp/management/commands/owasp_aggregate_projects.py (1)
119-124: LGTM! Proper integration of compliance and scoring updates.The conditional execution on
offset == 0ensures these operations run once per aggregation cycle rather than on every paginated batch. The use ofself.stdout.writewithself.style.NOTICEfollows Django conventions for management command output. Based on learnings, this is the preferred pattern overprint().backend/tests/apps/owasp/management/commands/owasp_update_project_health_scores_test.py (1)
66-66: Good addition to explicitly set compliance state.Setting
level_non_compliant = Falsein the successful update test ensures clear baseline behavior for the compliant case.backend/tests/apps/owasp/management/commands/owasp_aggregate_projects_test.py (1)
79-89: LGTM! Thorough test coverage for conditional command invocation.The test correctly:
- Patches
call_commandat the module level where it's imported- Uses parametrized cases with both
offset=0andoffset!=0scenarios- Verifies both commands are called when
offset=0usingassert_any_call- Verifies no commands are called when
offset!=0usingassert_not_calledThis provides good coverage for the new orchestration logic.
backend/apps/owasp/models/project_health_metrics.py (1)
46-50: LGTM! Well-defined model field for tracking level compliance.The field is appropriately:
- Named consistently with existing fields
- Defaulted to
False(compliant until flagged)- Documented with clear help text
- Positioned near related compliance fields (
is_funding_requirements_compliant,is_leader_requirements_compliant)The corresponding migration (0070) properly adds this field with the same attributes.
backend/apps/owasp/migrations/0070_projecthealthmetrics_level_non_compliant.py (1)
1-21: LGTM!The migration correctly adds the
level_non_compliantBooleanField with sensible defaults and documentation. The dependency chain is properly maintained.backend/apps/owasp/management/commands/owasp_update_project_level_compliance.py (1)
28-30: LGTM!The
normalize_namefunction provides reasonable fuzzy matching by removing the "owasp" prefix and non-alphanumeric characters.backend/apps/owasp/management/commands/owasp_update_project_health_scores.py (4)
16-17: LGTM!The
LEVEL_NON_COMPLIANCE_PENALTYconstant is well-named and the value of 10.0 points (10% of max score) is a reasonable penalty for non-compliance.
87-93: LGTM!The scoring logic correctly handles both numeric and boolean fields. Using
int()for comparison works for booleans (True=1,False=0) and ensures consistent type handling.
95-99: LGTM!The non-compliance penalty is correctly applied and the score is properly clamped to prevent negative values.
101-105: LGTM!The conditional bulk save with explicit field specification is efficient and follows best practices.
✏️ Tip: You can disable this entire section by setting review_details to false in your review settings.
backend/apps/owasp/management/commands/owasp_update_project_level_compliance.py
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Fix all issues with AI agents
In
`@backend/apps/owasp/management/commands/owasp_update_project_level_compliance.py`:
- Around line 51-53: Wrap the HTTP fetch of LEVELS_URL in a try/except that
catches requests.exceptions.RequestException (and optionally ValueError from
.json()), call response.raise_for_status() inside the try, and on any exception
log the error and exit/return early or set official_data to a safe default to
avoid crashing the management command; update the block that currently assigns
response = requests.get(LEVELS_URL, timeout=15) / response.raise_for_status() /
official_data = response.json() to perform the guarded fetch and JSON parse and
handle failures gracefully.
♻️ Duplicate comments (1)
backend/apps/owasp/management/commands/owasp_update_project_level_compliance.py (1)
71-72: Filter to latest metrics only to avoid processing stale records.The current query retrieves all
ProjectHealthMetricsrecords, including historical entries. Useget_latest_health_metrics()to process only the most recent metric per active project, which is the appropriate scope for compliance detection.- metrics = ProjectHealthMetrics.objects.select_related("project") + metrics = ProjectHealthMetrics.get_latest_health_metrics().select_related("project")
🧹 Nitpick comments (3)
backend/tests/apps/owasp/utils/project_level_test.py (1)
19-42: Consider adding edge case tests for unmapped and invalid inputs.The current tests cover all defined mappings, which is good. However, consider adding tests for:
- Unmapped positive values (e.g.,
Decimal("1.5"),Decimal(5)) to verify they returnNone- Invalid inputs (e.g.,
None,"invalid") to verify the exception handling inmap_levelThese would strengthen confidence in the error handling paths.
💡 Suggested additional tests
def test_unmapped_positive_level_returns_none(self): """Unmapped positive levels should return None.""" assert map_level(Decimal("1.5")) is None assert map_level(Decimal(5)) is None def test_invalid_input_returns_none(self): """Invalid inputs should return None without raising exceptions.""" assert map_level(None) is None assert map_level("invalid") is Nonebackend/apps/owasp/management/commands/owasp_update_project_health_scores.py (1)
98-104: Success message prints even when no metrics are updated.The success message at Line 104 always prints, even when
metrics_to_updateis empty (e.g., no metrics withscore__isnull=True). Consider adjusting the message to reflect the actual count updated.💡 Suggested improvement
- self.stdout.write(self.style.SUCCESS("Updated project health scores successfully.")) + self.stdout.write( + self.style.SUCCESS(f"Updated {len(metrics_to_update)} project health scores.") + )backend/tests/apps/owasp/management/commands/owasp_update_project_level_compliance_test.py (1)
9-66: Tests verify core compliance logic but miss error paths.The two test cases correctly verify:
- Non-compliant when API level differs from local level
- Compliant when levels match
Consider adding tests for:
- Network/timeout errors from
requests.get(command should handle gracefully or propagate)- Projects not found in official data (should be skipped, not marked)
- Invalid level values in API response
These would improve confidence in production resilience.
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (5)
backend/apps/owasp/management/commands/owasp_update_project_health_scores.pybackend/apps/owasp/management/commands/owasp_update_project_level_compliance.pybackend/apps/owasp/utils/project_level.pybackend/tests/apps/owasp/management/commands/owasp_update_project_level_compliance_test.pybackend/tests/apps/owasp/utils/project_level_test.py
🧰 Additional context used
🧠 Learnings (6)
📚 Learning: 2026-01-06T12:38:23.460Z
Learnt from: ahmedxgouda
Repo: OWASP/Nest PR: 3122
File: backend/apps/owasp/api/internal/nodes/project.py:52-61
Timestamp: 2026-01-06T12:38:23.460Z
Learning: In backend/apps/owasp/api/internal/nodes/project.py, the health_metrics_list method intentionally orders by nest_created_at ascending (oldest first) to return metrics in chronological order for trend analysis, which differs from health_metrics_latest that orders descending to get the most recent metric.
Applied to files:
backend/apps/owasp/management/commands/owasp_update_project_health_scores.pybackend/apps/owasp/management/commands/owasp_update_project_level_compliance.py
📚 Learning: 2025-12-18T05:39:42.678Z
Learnt from: rudransh-shrivastava
Repo: OWASP/Nest PR: 2948
File: backend/apps/owasp/management/commands/owasp_generate_community_snapshot_video.py:40-40
Timestamp: 2025-12-18T05:39:42.678Z
Learning: In Django management commands, prefer using self.stdout.write(...) over print(...) for user-facing stdout output. This aligns with Django conventions and improves testability. When emitting messages, consider using self.stdout.write and, for styled messages, use self.style.SUCCESS/ERROR as appropriate to maintain consistent command output formatting. Apply this guideline to all Python files within any project's management/commands directory.
Applied to files:
backend/apps/owasp/management/commands/owasp_update_project_health_scores.pybackend/apps/owasp/management/commands/owasp_update_project_level_compliance.pybackend/tests/apps/owasp/management/commands/owasp_update_project_level_compliance_test.py
📚 Learning: 2025-12-31T05:17:39.659Z
Learnt from: kart-u
Repo: OWASP/Nest PR: 3101
File: backend/apps/common/extensions.py:92-98
Timestamp: 2025-12-31T05:17:39.659Z
Learning: In this codebase, import OperationType for GraphQL operations from the graphql-core package rather than from strawberry. Use 'from graphql import OperationType'. Strawberry re-exports via graphql-core internally, so relying on strawberry's API may be brittle. Apply this rule to all Python files that deal with GraphQL operation types; ensure imports come from graphql (graphql-core) and not from strawberry packages. This improves compatibility and avoids coupling to strawberry's internals.
Applied to files:
backend/apps/owasp/management/commands/owasp_update_project_health_scores.pybackend/apps/owasp/management/commands/owasp_update_project_level_compliance.pybackend/tests/apps/owasp/utils/project_level_test.pybackend/apps/owasp/utils/project_level.pybackend/tests/apps/owasp/management/commands/owasp_update_project_level_compliance_test.py
📚 Learning: 2026-01-01T17:48:23.963Z
Learnt from: rudransh-shrivastava
Repo: OWASP/Nest PR: 2948
File: backend/apps/owasp/management/commands/owasp_generate_community_snapshot_video.py:41-47
Timestamp: 2026-01-01T17:48:23.963Z
Learning: In Django code, be aware that a QuerySet's boolean evaluation (e.g., if not queryset) runs a database query to determine emptiness. While it is technically valid to use the queryset in a boolean context, use queryset.exists() for existence checks to avoid unnecessary queries and improve performance. Applicable broadly to Python/Django files rather than just this specific path.
Applied to files:
backend/apps/owasp/management/commands/owasp_update_project_health_scores.pybackend/apps/owasp/management/commands/owasp_update_project_level_compliance.pybackend/tests/apps/owasp/utils/project_level_test.pybackend/apps/owasp/utils/project_level.pybackend/tests/apps/owasp/management/commands/owasp_update_project_level_compliance_test.py
📚 Learning: 2026-01-01T18:57:05.007Z
Learnt from: rudransh-shrivastava
Repo: OWASP/Nest PR: 2948
File: backend/apps/owasp/video.py:189-215
Timestamp: 2026-01-01T18:57:05.007Z
Learning: In the OWASP backend area, maintain the established pattern: when dealing with sponsors, include all entries from Sponsor.objects.all() (including NOT_SPONSOR) and perform in-memory sorting using the same criteria/pattern used by the GraphQL sponsor query implemented in backend/apps/owasp/api/internal/queries/sponsor.py. Apply this behavior consistently to files in backend/apps/owasp (not just video.py), and ensure code paths that render sponsor lists follow this in-code sorting approach rather than pre-filtering NOT_SPONSOR entries before sorting.
Applied to files:
backend/apps/owasp/management/commands/owasp_update_project_health_scores.pybackend/apps/owasp/management/commands/owasp_update_project_level_compliance.pybackend/apps/owasp/utils/project_level.py
📚 Learning: 2025-08-04T15:55:08.017Z
Learnt from: ahmedxgouda
Repo: OWASP/Nest PR: 1967
File: backend/apps/owasp/api/internal/filters/project_health_metrics.py:24-27
Timestamp: 2025-08-04T15:55:08.017Z
Learning: In the OWASP Nest project, the `is_active` filter in `ProjectHealthMetricsFilter` is intentionally designed as a workaround to eliminate inactive projects from the dashboard. It only filters FOR active projects when `value=True`, and returns an empty Q() when `value=False` to avoid showing inactive projects. This is not meant to be a general boolean filter but a specific solution to exclude inactive projects from the project health metrics dashboard.
Applied to files:
backend/apps/owasp/management/commands/owasp_update_project_level_compliance.py
🧬 Code graph analysis (5)
backend/apps/owasp/management/commands/owasp_update_project_health_scores.py (2)
backend/apps/owasp/models/project_health_metrics.py (1)
ProjectHealthMetrics(16-240)backend/apps/owasp/models/project_health_requirements.py (1)
ProjectHealthRequirements(9-63)
backend/apps/owasp/management/commands/owasp_update_project_level_compliance.py (2)
backend/apps/owasp/models/project_health_metrics.py (1)
ProjectHealthMetrics(16-240)backend/apps/owasp/utils/project_level.py (1)
map_level(23-42)
backend/tests/apps/owasp/utils/project_level_test.py (2)
backend/apps/owasp/models/enums/project.py (1)
ProjectLevel(37-44)backend/apps/owasp/utils/project_level.py (1)
map_level(23-42)
backend/apps/owasp/utils/project_level.py (1)
backend/apps/owasp/models/enums/project.py (1)
ProjectLevel(37-44)
backend/tests/apps/owasp/management/commands/owasp_update_project_level_compliance_test.py (3)
backend/apps/owasp/models/enums/project.py (1)
ProjectLevel(37-44)backend/tests/apps/common/models_test.py (1)
mock_objects(19-24)backend/tests/apps/owasp/management/commands/owasp_aggregate_contributions_test.py (1)
select_related(36-38)
🔇 Additional comments (3)
backend/apps/owasp/utils/project_level.py (1)
23-42: Type hint is stricter than actual behavior.The type hint specifies
level: Decimal, but the implementation gracefully handles non-Decimal inputs by converting viaDecimal(str(level)). This is defensive coding that works well, but the signature could be broadened tolevel: Decimal | int | float | strto reflect actual accepted types—or keep the current hint to encourage callers to passDecimalexplicitly. Either approach is valid; just noting the discrepancy.backend/apps/owasp/management/commands/owasp_update_project_health_scores.py (1)
84-90: Boolean field comparison viaint()is implicit but functional.The
forward_fieldsincludeis_funding_requirements_compliantandis_leader_requirements_compliant, which are booleans. Usingint(getattr(metric, field))convertsTrueto1andFalseto0, and the comparisonint(metric_value) >= int(requirement_value)works correctly:
- If requirement is
True(1), metric must beTrue(1) to score- If requirement is
False(0), metric always passes (0 >= 0 or 1 >= 0)This works but relies on implicit bool-to-int conversion. Consider adding a brief comment to clarify this intentional behavior for future maintainers.
backend/apps/owasp/management/commands/owasp_update_project_level_compliance.py (1)
27-29: LGTM on name normalization.The
normalize_namefunction appropriately strips "owasp" and non-alphanumeric characters for fuzzy matching between local and official project names. This handles common variations like "OWASP Foo" vs "Foo".
✏️ Tip: You can disable this entire section by setting review_details to false in your review settings.
backend/apps/owasp/management/commands/owasp_update_project_level_compliance.py
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Fix all issues with AI agents
In
`@backend/apps/owasp/management/commands/owasp_update_project_level_compliance.py`:
- Around line 51-60: The try/except block around fetching LEVELS_URL has
inconsistent indentation making the file invalid; fix by aligning the except
with the try (i.e., dedent the except and its body to the same indentation level
as the try), so the flow around requests.get(LEVELS_URL, timeout=15),
response.raise_for_status(), and official_data = response.json() is properly
enclosed and the self.stdout.write(self.style.ERROR(...)) and return are
executed in the except handler; verify references to LEVELS_URL, response,
official_data, and self.stdout.write remain unchanged.
♻️ Duplicate comments (1)
backend/apps/owasp/management/commands/owasp_update_project_level_compliance.py (1)
51-76: Harden fetch/parsing: handle JSON decode + unexpected payload shape.
response.json()can raiseValueError, and the code assumesofficial_datais an iterable of dicts.Proposed fix
- try: - response = requests.get(LEVELS_URL, timeout=15) - response.raise_for_status() - official_data = response.json() - except requests.RequestException as exc: + try: + response = requests.get(LEVELS_URL, timeout=15) + response.raise_for_status() + official_data = response.json() + except (requests.RequestException, ValueError) as exc: self.stdout.write( self.style.ERROR(f"Failed to fetch official project levels: {exc}") ) return + + if not isinstance(official_data, list): + self.stdout.write( + self.style.ERROR("Unexpected official project levels payload (expected a list).") + ) + return @@ - for item in official_data: + for item in official_data: + if not isinstance(item, dict): + continue raw_level = item.get("level")
🧹 Nitpick comments (1)
backend/apps/owasp/management/commands/owasp_update_project_level_compliance.py (1)
77-105: Avoid unnecessary DB writes: only bulk-save whenlevel_non_compliantactually changes.
Right now you append every matched metric, even if the computed flag equals the stored value.Proposed fix
- metric.level_non_compliant = project.level != expected_level - updated_metrics.append(metric) + new_flag = project.level != expected_level + if metric.level_non_compliant != new_flag: + metric.level_non_compliant = new_flag + updated_metrics.append(metric)
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (1)
backend/apps/owasp/management/commands/owasp_update_project_level_compliance.py
🧰 Additional context used
🧠 Learnings (6)
📚 Learning: 2026-01-06T12:38:23.460Z
Learnt from: ahmedxgouda
Repo: OWASP/Nest PR: 3122
File: backend/apps/owasp/api/internal/nodes/project.py:52-61
Timestamp: 2026-01-06T12:38:23.460Z
Learning: In backend/apps/owasp/api/internal/nodes/project.py, the health_metrics_list method intentionally orders by nest_created_at ascending (oldest first) to return metrics in chronological order for trend analysis, which differs from health_metrics_latest that orders descending to get the most recent metric.
Applied to files:
backend/apps/owasp/management/commands/owasp_update_project_level_compliance.py
📚 Learning: 2025-08-04T15:55:08.017Z
Learnt from: ahmedxgouda
Repo: OWASP/Nest PR: 1967
File: backend/apps/owasp/api/internal/filters/project_health_metrics.py:24-27
Timestamp: 2025-08-04T15:55:08.017Z
Learning: In the OWASP Nest project, the `is_active` filter in `ProjectHealthMetricsFilter` is intentionally designed as a workaround to eliminate inactive projects from the dashboard. It only filters FOR active projects when `value=True`, and returns an empty Q() when `value=False` to avoid showing inactive projects. This is not meant to be a general boolean filter but a specific solution to exclude inactive projects from the project health metrics dashboard.
Applied to files:
backend/apps/owasp/management/commands/owasp_update_project_level_compliance.py
📚 Learning: 2025-12-18T05:39:42.678Z
Learnt from: rudransh-shrivastava
Repo: OWASP/Nest PR: 2948
File: backend/apps/owasp/management/commands/owasp_generate_community_snapshot_video.py:40-40
Timestamp: 2025-12-18T05:39:42.678Z
Learning: In Django management commands, prefer using self.stdout.write(...) over print(...) for user-facing stdout output. This aligns with Django conventions and improves testability. When emitting messages, consider using self.stdout.write and, for styled messages, use self.style.SUCCESS/ERROR as appropriate to maintain consistent command output formatting. Apply this guideline to all Python files within any project's management/commands directory.
Applied to files:
backend/apps/owasp/management/commands/owasp_update_project_level_compliance.py
📚 Learning: 2025-12-31T05:17:39.659Z
Learnt from: kart-u
Repo: OWASP/Nest PR: 3101
File: backend/apps/common/extensions.py:92-98
Timestamp: 2025-12-31T05:17:39.659Z
Learning: In this codebase, import OperationType for GraphQL operations from the graphql-core package rather than from strawberry. Use 'from graphql import OperationType'. Strawberry re-exports via graphql-core internally, so relying on strawberry's API may be brittle. Apply this rule to all Python files that deal with GraphQL operation types; ensure imports come from graphql (graphql-core) and not from strawberry packages. This improves compatibility and avoids coupling to strawberry's internals.
Applied to files:
backend/apps/owasp/management/commands/owasp_update_project_level_compliance.py
📚 Learning: 2026-01-01T17:48:23.963Z
Learnt from: rudransh-shrivastava
Repo: OWASP/Nest PR: 2948
File: backend/apps/owasp/management/commands/owasp_generate_community_snapshot_video.py:41-47
Timestamp: 2026-01-01T17:48:23.963Z
Learning: In Django code, be aware that a QuerySet's boolean evaluation (e.g., if not queryset) runs a database query to determine emptiness. While it is technically valid to use the queryset in a boolean context, use queryset.exists() for existence checks to avoid unnecessary queries and improve performance. Applicable broadly to Python/Django files rather than just this specific path.
Applied to files:
backend/apps/owasp/management/commands/owasp_update_project_level_compliance.py
📚 Learning: 2026-01-01T18:57:05.007Z
Learnt from: rudransh-shrivastava
Repo: OWASP/Nest PR: 2948
File: backend/apps/owasp/video.py:189-215
Timestamp: 2026-01-01T18:57:05.007Z
Learning: In the OWASP backend area, maintain the established pattern: when dealing with sponsors, include all entries from Sponsor.objects.all() (including NOT_SPONSOR) and perform in-memory sorting using the same criteria/pattern used by the GraphQL sponsor query implemented in backend/apps/owasp/api/internal/queries/sponsor.py. Apply this behavior consistently to files in backend/apps/owasp (not just video.py), and ensure code paths that render sponsor lists follow this in-code sorting approach rather than pre-filtering NOT_SPONSOR entries before sorting.
Applied to files:
backend/apps/owasp/management/commands/owasp_update_project_level_compliance.py
🧬 Code graph analysis (1)
backend/apps/owasp/management/commands/owasp_update_project_level_compliance.py (2)
backend/apps/owasp/models/project_health_metrics.py (2)
ProjectHealthMetrics(16-240)get_latest_health_metrics(165-179)backend/apps/owasp/utils/project_level.py (1)
map_level(23-42)
🪛 Ruff (0.14.11)
backend/apps/owasp/management/commands/owasp_update_project_level_compliance.py
55-55: unindent does not match any outer indentation level
(invalid-syntax)
55-55: Expected a statement
(invalid-syntax)
55-55: Expected a statement
(invalid-syntax)
55-56: Expected an expression
(invalid-syntax)
56-56: Unexpected indentation
(invalid-syntax)
61-61: Expected except or finally after try block
(invalid-syntax)
🔇 Additional comments (1)
backend/apps/owasp/management/commands/owasp_update_project_level_compliance.py (1)
27-30: Helper + latest-metrics scope + stdout usage look good.
normalize_name()is simple and testable,get_latest_health_metrics()is the right scope, andself.stdout.write(self.style.*)matches the established command pattern. (Based on learnings.)Also applies to: 77-109
✏️ Tip: You can disable this entire section by setting review_details to false in your review settings.
backend/apps/owasp/management/commands/owasp_update_project_level_compliance.py
Show resolved
Hide resolved
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Fix all issues with AI agents
In
`@backend/apps/owasp/management/commands/owasp_update_project_level_compliance.py`:
- Around line 51-57: response.json() can raise a JSON decode error that isn't a
requests.RequestException, so update the try/except around the
requests.get/response.json() call (in the command where LEVELS_URL is fetched)
to also catch json.JSONDecodeError or ValueError; in the new except block (or by
adding a separate except ValueError as exc) write the same error to
self.stderr.write(self.style.ERROR(...)) and return to prevent an unhandled
exception.
📜 Review details
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
backend/apps/owasp/management/commands/owasp_update_project_level_compliance.pybackend/tests/apps/owasp/management/commands/owasp_update_project_level_compliance_test.py
🚧 Files skipped from review as they are similar to previous changes (1)
- backend/tests/apps/owasp/management/commands/owasp_update_project_level_compliance_test.py
🧰 Additional context used
🧠 Learnings (6)
📚 Learning: 2026-01-06T12:38:23.460Z
Learnt from: ahmedxgouda
Repo: OWASP/Nest PR: 3122
File: backend/apps/owasp/api/internal/nodes/project.py:52-61
Timestamp: 2026-01-06T12:38:23.460Z
Learning: In backend/apps/owasp/api/internal/nodes/project.py, the health_metrics_list method intentionally orders by nest_created_at ascending (oldest first) to return metrics in chronological order for trend analysis, which differs from health_metrics_latest that orders descending to get the most recent metric.
Applied to files:
backend/apps/owasp/management/commands/owasp_update_project_level_compliance.py
📚 Learning: 2025-08-04T15:55:08.017Z
Learnt from: ahmedxgouda
Repo: OWASP/Nest PR: 1967
File: backend/apps/owasp/api/internal/filters/project_health_metrics.py:24-27
Timestamp: 2025-08-04T15:55:08.017Z
Learning: In the OWASP Nest project, the `is_active` filter in `ProjectHealthMetricsFilter` is intentionally designed as a workaround to eliminate inactive projects from the dashboard. It only filters FOR active projects when `value=True`, and returns an empty Q() when `value=False` to avoid showing inactive projects. This is not meant to be a general boolean filter but a specific solution to exclude inactive projects from the project health metrics dashboard.
Applied to files:
backend/apps/owasp/management/commands/owasp_update_project_level_compliance.py
📚 Learning: 2025-12-18T05:39:42.678Z
Learnt from: rudransh-shrivastava
Repo: OWASP/Nest PR: 2948
File: backend/apps/owasp/management/commands/owasp_generate_community_snapshot_video.py:40-40
Timestamp: 2025-12-18T05:39:42.678Z
Learning: In Django management commands, prefer using self.stdout.write(...) over print(...) for user-facing stdout output. This aligns with Django conventions and improves testability. When emitting messages, consider using self.stdout.write and, for styled messages, use self.style.SUCCESS/ERROR as appropriate to maintain consistent command output formatting. Apply this guideline to all Python files within any project's management/commands directory.
Applied to files:
backend/apps/owasp/management/commands/owasp_update_project_level_compliance.py
📚 Learning: 2025-12-31T05:17:39.659Z
Learnt from: kart-u
Repo: OWASP/Nest PR: 3101
File: backend/apps/common/extensions.py:92-98
Timestamp: 2025-12-31T05:17:39.659Z
Learning: In this codebase, import OperationType for GraphQL operations from the graphql-core package rather than from strawberry. Use 'from graphql import OperationType'. Strawberry re-exports via graphql-core internally, so relying on strawberry's API may be brittle. Apply this rule to all Python files that deal with GraphQL operation types; ensure imports come from graphql (graphql-core) and not from strawberry packages. This improves compatibility and avoids coupling to strawberry's internals.
Applied to files:
backend/apps/owasp/management/commands/owasp_update_project_level_compliance.py
📚 Learning: 2026-01-01T17:48:23.963Z
Learnt from: rudransh-shrivastava
Repo: OWASP/Nest PR: 2948
File: backend/apps/owasp/management/commands/owasp_generate_community_snapshot_video.py:41-47
Timestamp: 2026-01-01T17:48:23.963Z
Learning: In Django code, be aware that a QuerySet's boolean evaluation (e.g., if not queryset) runs a database query to determine emptiness. While it is technically valid to use the queryset in a boolean context, use queryset.exists() for existence checks to avoid unnecessary queries and improve performance. Applicable broadly to Python/Django files rather than just this specific path.
Applied to files:
backend/apps/owasp/management/commands/owasp_update_project_level_compliance.py
📚 Learning: 2026-01-01T18:57:05.007Z
Learnt from: rudransh-shrivastava
Repo: OWASP/Nest PR: 2948
File: backend/apps/owasp/video.py:189-215
Timestamp: 2026-01-01T18:57:05.007Z
Learning: In the OWASP backend area, maintain the established pattern: when dealing with sponsors, include all entries from Sponsor.objects.all() (including NOT_SPONSOR) and perform in-memory sorting using the same criteria/pattern used by the GraphQL sponsor query implemented in backend/apps/owasp/api/internal/queries/sponsor.py. Apply this behavior consistently to files in backend/apps/owasp (not just video.py), and ensure code paths that render sponsor lists follow this in-code sorting approach rather than pre-filtering NOT_SPONSOR entries before sorting.
Applied to files:
backend/apps/owasp/management/commands/owasp_update_project_level_compliance.py
🔇 Additional comments (2)
backend/apps/owasp/management/commands/owasp_update_project_level_compliance.py (2)
78-97: LGTM!The iteration logic correctly:
- Resolves official levels via repo slug first, then falls back to normalized name matching
- Gracefully skips projects without official data or unmappable levels
- Efficiently uses
select_related("project")to avoid N+1 queries
27-29: LGTM!The normalization function correctly strips "owasp" prefix and non-alphanumeric characters for fuzzy matching, and is consistently used for both building and querying the lookup map.
✏️ Tip: You can disable this entire section by setting review_details to false in your review settings.
backend/apps/owasp/management/commands/owasp_update_project_level_compliance.py
Show resolved
Hide resolved
|
1 participant
Proposed change
Resolves #2039
This PR adds end-to-end detection of OWASP project level non-compliance and ensures project health scores reflect this misalignment using the official OWASP source of truth.
What this does
Why this approach
Tests
Added tests to verify:
All tests pass locally and CI coverage remains above the required threshold.
Checklist