Merge pull request #13177 from anthropics/fvolcic/code-review-command-updates

This commit is contained in:
Franklin Volcic
2025-12-05 18:22:08 -08:00
committed by GitHub

View File

@@ -1,65 +1,91 @@
--- ---
allowed-tools: Bash(gh issue view:*), Bash(gh search:*), Bash(gh issue list:*), Bash(gh pr comment:*), Bash(gh pr diff:*), Bash(gh pr view:*), Bash(gh pr list:*) allowed-tools: Bash(gh issue view:*), Bash(gh search:*), Bash(gh issue list:*), Bash(gh pr comment:*), Bash(gh pr diff:*), Bash(gh pr view:*), Bash(gh pr list:*)
description: Code review a pull request description: Code review a pull request
disable-model-invocation: false
--- ---
Provide a code review for the given pull request. Provide a code review for the given pull request.
To do this, follow these steps precisely: To do this, follow these steps precisely:
1. Use a Haiku agent to check if the pull request (a) is closed, (b) is a draft, (c) does not need a code review (eg. because it is an automated pull request, or is very simple and obviously ok), or (d) already has a code review from you from earlier. If so, do not proceed. 1. Launch a haiku agent to check if any of the following are true:
2. Use another Haiku agent to give you a list of file paths to (but not the contents of) any relevant CLAUDE.md files from the codebase: the root CLAUDE.md file (if one exists), as well as any CLAUDE.md files in the directories whose files the pull request modified - The pull request is closed
3. Use a Haiku agent to view the pull request, and ask the agent to return a summary of the change - The pull request is a draft
4. Then, launch 5 parallel Sonnet agents to independently code review the change. The agents should do the following, then return a list of issues and the reason each issue was flagged (eg. CLAUDE.md adherence, bug, historical git context, etc.): - The pull request does not need code review (e.g. automated PR, trivial change that is obviously correct)
a. Agent #1: Audit the changes to make sure they compily with the CLAUDE.md. Note that CLAUDE.md is guidance for Claude as it writes code, so not all instructions will be applicable during code review. - You have already submitted a code review on this pull request
b. Agent #2: Read the file changes in the pull request, then do a shallow scan for obvious bugs. Avoid reading extra context beyond the changes, focusing just on the changes themselves. Focus on large bugs, and avoid small issues and nitpicks. Ignore likely false positives.
c. Agent #3: Read the git blame and history of the code modified, to identify any bugs in light of that historical context If any condition is true, stop and do not proceed.
d. Agent #4: Read previous pull requests that touched these files, and check for any comments on those pull requests that may also apply to the current pull request.
e. Agent #5: Read code comments in the modified files, and make sure the changes in the pull request comply with any guidance in the comments. Note: Still review Claude generated PR's.
5. For each issue found in #4, launch a parallel Haiku agent that takes the PR, issue description, and list of CLAUDE.md files (from step 2), and returns a score to indicate the agent's level of confidence for whether the issue is real or false positive. To do that, the agent should score each issue on a scale from 0-100, indicating its level of confidence. For issues that were flagged due to CLAUDE.md instructions, the agent should double check that the CLAUDE.md actually calls out that issue specifically. The scale is (give this rubric to the agent verbatim):
a. 0: Not confident at all. This is a false positive that doesn't stand up to light scrutiny, or is a pre-existing issue. 2. Launch a haiku agent to return a list of file paths (not their contents) for all relevant CLAUDE.md files including:
b. 25: Somewhat confident. This might be a real issue, but may also be a false positive. The agent wasn't able to verify that it's a real issue. If the issue is stylistic, it is one that was not explicitly called out in the relevant CLAUDE.md. - The root CLAUDE.md file, if it exists
c. 50: Moderately confident. The agent was able to verify this is a real issue, but it might be a nitpick or not happen very often in practice. Relative to the rest of the PR, it's not very important. - Any CLAUDE.md files in directories containing files modified by the pull request
d. 75: Highly confident. The agent double checked the issue, and verified that it is very likely it is a real issue that will be hit in practice. The existing approach in the PR is insufficient. The issue is very important and will directly impact the code's functionality, or it is an issue that is directly mentioned in the relevant CLAUDE.md.
e. 100: Absolutely certain. The agent double checked the issue, and confirmed that it is definitely a real issue, that will happen frequently in practice. The evidence directly confirms this. 3. Launch a sonnet agent to view the pull request and return a summary of the changes
6. Filter out any issues with a score less than 80. If there are no issues that meet this criteria, do not proceed.
7. Use a Haiku agent to repeat the eligibility check from #1, to make sure that the pull request is still eligible for code review. 4. Launch 4 agents in parallel to independently review the changes. Each agent should return the list of issues, where each issue includes a description and the reason it was flagged (e.g. "CLAUDE.md adherence", "bug"). The agents should do the following:
8. Finally, use the gh bash command to comment back on the pull request with the result. When writing your comment, keep in mind to:
Agents 1 + 2: CLAUDE.md compliance sonnet agents
Audit changes for CLAUDE.md compliance in parallel. Note: When evaluating CLAUDE.md compliance for a file, you should only consider CLAUDE.md files that share a file path with the file or parents.
Agent 3: Opus bug agent (parallel subagent with agent 4)
Scan for obvious bugs. Focus only on the diff itself without reading extra context. Flag only significant bugs; ignore nitpicks and likely false positives. Do not flag issues that you cannot validate without looking at context outside of the git diff.
Agent 4: Opus bug agent (parallel subagent with agent 3)
Look for problems that exist in the introduced code. This could be security issues, incorrect logic, etc. Only look for issues that fall within the changed code.
**CRITICAL: We only want HIGH SIGNAL issues.** This means:
- Objective bugs that will cause incorrect behavior at runtime
- Clear, unambiguous CLAUDE.md violations where you can quote the exact rule being broken
We do NOT want:
- Subjective concerns or "suggestions"
- Style preferences not explicitly required by CLAUDE.md
- Potential issues that "might" be problems
- Anything requiring interpretation or judgment calls
If you are not certain an issue is real, do not flag it. False positives erode trust and waste reviewer time.
In addition to the above, each subagent should be told the PR title and description. This will help provide context regarding the author's intent.
5. For each issue found in the previous step by agents 3 and 4, launch parallel subagents to validate the issue. These subagents should get the PR title and description along with a description of the issue. The agent's job is to review the issue to validate that the stated issue is truly an issue with high confidence. For example, if an issue such as "variable is not defined" was flagged, the subagent's job would be to validate that is actually true in the code. Another example would be CLAUDE.md issues. The agent should validate that the CLAUDE.md rule that was violated is scoped for this file and is actually violated. Use Opus subagents for bugs and logic issues, and sonnet agents for CLAUDE.md violations.
6. Filter out any issues that were not validated in step 5. This step will give us our list of high signal issues for our review.
7. Finally, comment on the pull request.
When writing your comment, follow these guidelines:
a. Keep your output brief a. Keep your output brief
b. Avoid emojis b. Avoid emojis
c. Link and cite relevant code, files, and URLs c. Link and cite relevant code, files, and URLs for each issue
d. When citing CLAUDE.md violations, you MUST quote the exact text from CLAUDE.md that is being violated (e.g., CLAUDE.md says: "Use snake_case for variable names")
Examples of false positives, for steps 4 and 5: Use this list when evaluating issues in Steps 4 and 5 (these are false positives, do NOT flag):
- Pre-existing issues - Pre-existing issues
- Something that looks like a bug but is not actually a bug - Something that appears to be a bug but is actually correct
- Pedantic nitpicks that a senior engineer wouldn't call out - Pedantic nitpicks that a senior engineer would not flag
- Issues that a linter, typechecker, or compiler would catch (eg. missing or incorrect imports, type errors, broken tests, formatting issues, pedantic style issues like newlines). No need to run these build steps yourself -- it is safe to assume that they will be run separately as part of CI. - Issues that a linter will catch (do not run the linter to verify)
- General code quality issues (eg. lack of test coverage, general security issues, poor documentation), unless explicitly required in CLAUDE.md - General code quality concerns (e.g., lack of test coverage, general security issues) unless explicitly required in CLAUDE.md
- Issues that are called out in CLAUDE.md, but explicitly silenced in the code (eg. due to a lint ignore comment) - Issues mentioned in CLAUDE.md but explicitly silenced in the code (e.g., via a lint ignore comment)
- Changes in functionality that are likely intentional or are directly related to the broader change
- Real issues, but on lines that the user did not modify in their pull request
Notes: Notes:
- Do not check build signal or attempt to build or typecheck the app. These will run separately, and are not relevant to your code review. - Use gh CLI to interact with GitHub (e.g., fetch pull requests, create comments). Do not use web fetch.
- Use `gh` to interact with Github (eg. to fetch a pull request, or to create inline comments), rather than web fetch - Create a todo list before starting.
- Make a todo list first - You must cite and link each issue (e.g., if referring to a CLAUDE.md, include a link to it).
- You must cite and link each bug (eg. if referring to a CLAUDE.md, you must link it)
- For your final comment, follow the following format precisely (assuming for this example that you found 3 issues): - For your final comment, follow the following format precisely (assuming for this example that you found 3 issues):
--- ---
### Code review ## Code review
Found 3 issues: Found 3 issues:
1. <brief description of bug> (CLAUDE.md says "<...>") 1. <brief description of bug> (CLAUDE.md says: "<exact quote from CLAUDE.md>")
<link to file and line with full sha1 + line range for context, note that you MUST provide the full sha and not use bash here, eg. https://github.com/anthropics/claude-code/blob/1d54823877c4de72b2316a64032a54afc404e619/README.md#L13-L17> <link to file and line with full sha1 + line range for context, eg. https://github.com/anthropics/claude-code/blob/1d54823877c4de72b2316a64032a54afc404e619/README.md#L13-L17>
2. <brief description of bug> (some/other/CLAUDE.md says "<...>") 2. <brief description of bug> (some/other/CLAUDE.md says: "<exact quote from CLAUDE.md>")
<link to file and line with full sha1 + line range for context> <link to file and line with full sha1 + line range for context>
@@ -69,6 +95,7 @@ Found 3 issues:
🤖 Generated with [Claude Code](https://claude.ai/code) 🤖 Generated with [Claude Code](https://claude.ai/code)
<sub>- If this code review was useful, please react with 👍. Otherwise, react with 👎.</sub> <sub>- If this code review was useful, please react with 👍. Otherwise, react with 👎.</sub>
--- ---
@@ -77,13 +104,15 @@ Found 3 issues:
--- ---
### Code review ## Auto code review
No issues found. Checked for bugs and CLAUDE.md compliance. No issues found. Checked for bugs and CLAUDE.md compliance.
🤖 Generated with [Claude Code](https://claude.ai/code) 🤖 Generated with [Claude Code](https://claude.ai/code)
- When linking to code, follow the following format precisely, otherwise the Markdown preview won't render correctly: https://github.com/anthropics/claude-cli-internal/blob/c21d3c10bc8e898b7ac1a2d745bdc9bc4e423afe/package.json#L10-L15 ---
- When linking to code, follow the following format precisely, otherwise the Markdown preview won't render correctly: https://github.com/anthropics/claude-code/blob/c21d3c10bc8e898b7ac1a2d745bdc9bc4e423afe/package.json#L10-L15
- Requires full git sha - Requires full git sha
- You must provide the full sha. Commands like `https://github.com/owner/repo/blob/$(git rev-parse HEAD)/foo/bar` will not work, since your comment will be directly rendered in Markdown. - You must provide the full sha. Commands like `https://github.com/owner/repo/blob/$(git rev-parse HEAD)/foo/bar` will not work, since your comment will be directly rendered in Markdown.
- Repo name must match the repo you're code reviewing - Repo name must match the repo you're code reviewing