Skip to content

Constrain skill findings to diff hunk line range #150

@dcramer

Description

@dcramer

Problem

Warden findings that reference lines outside the diff hunks can't be posted as inline PR comments. GitHub rejects them with a 422 ("line could not be resolved"), triggering the fallback in poster.ts that moves comments into the review body as plain text.

Evidence

Both were posted as review body text instead of inline comments.

Root cause

The LLM has Read/Grep tools and explores beyond the hunk, but the prompt doesn't constrain startLine to the hunk range. The system prompt says "focus only on the changes shown" but doesn't explicitly tell the LLM that findings must reference lines within the hunk.

Proposed fix

1. Update system prompt (src/sdk/prompt.ts:75-84)

Add to the requirements in buildHunkSystemPrompt:

  • location.startLine MUST be within the hunk's line range (shown in the ## Hunk header). If the issue is in surrounding code, anchor the finding to the most relevant changed line within the hunk and explain in the description where the actual problem is.
  • Update "focus only on the changes shown" to: "Focus your analysis on the changes shown in the hunk. Context before/after is provided for understanding only -- findings must reference lines within the hunk range."

2. Add validation guard (src/sdk/analyze.ts:~420)

Defense-in-depth: after parseHunkOutput, filter findings with startLine outside the hunk range:

const hunkRange = getHunkLineRange(hunkCtx.hunk);
const inRangeFindings = parseResult.findings.filter((f) => {
  if (!f.location) return true; // Keep locationless findings
  return f.location.startLine >= hunkRange.start && f.location.startLine <= hunkRange.end;
});

Log a warning when findings are dropped so it's visible in Sentry.

3. Add tests

Test that out-of-range findings are filtered while locationless and in-range findings are preserved.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions