Feature request: CoPilot First Reviewer

[scottqueen-bixal]

Summary

Finding some reoccurring themes for prompting CoPilot as a "first reviewer" of local pre-commit, working branch code. We would like to provide a baseline prompt to review code and provide feedback...where possible, including suggested changes which can be applied with a couple of clicks.

The intention is to reduce engineer time needed to perform code reviews. By running some of these prompts and the PR diff through an LLM, we can ensure that code quality is held to a high standard. This removes the need for a human to catch syntax issues, documentation correctness, inefficiencies, and missing code coverage.

Describe the solution you'd like

• agent prompt(s)

List of inclusion

• evaluate my code for correctness
• evaluate my function docs and type hints for gaps and readability
• evaluate my approach for this functionality. Suggest alternative approaches. Look for more efficient ways to achieve the result.
• evaluate my code for quality and maintainability
• analyze this code for refactoring opportunities, look for ways to reduce code or simplify logic
• identify gaps in my unit test scenarios for this code
• evaluate if the log statements are accurate in describing the state of the logic throughout

consider

• user elicitation
• https://docs.github.com/en/copilot/tutorials/customization-library/prompt-files/review-code
• https://github.com/github/awesome-copilot/blob/main/instructions/code-review-generic.instructions.md

Alternatives you've considered

Copilot code review is a premium feature, available with the Copilot Pro, Copilot Pro+, Copilot Business, and Copilot Enterprise plans.

https://docs.github.com/en/copilot/concepts/agents/code-review

Additional context

Should be effective, but succinct, while allowing leads in projects to extend or add specific project level guidance as needed

[mejiaj]

@scottqueen-bixal thanks for creating this! It'll be a big help for all teams—especially solo devs.

Is user elicitation is asking more questions to get more context from the user? 👀

My experience

My initial approach has been using GitHub's Code reviewer custom instructions¹ and used that as a basis for a Drupal / PHP code reviewing agent. Based on your links, I realize this probably isn't the best approach because I've been running into issues.

Issues I've noticed

1. Feedback isn't in plain language
2. Formatting on responses can be hard to read/scan
3. Line numbers are often incorrect
4. Accessibility (section 508 / WCAG 2.2 AA) isn't considered
5. Some times it flags security issues when there are none
6. Often times, it'll rewrite the entire file for me when I'm only looking for a review on a specific piece.
   1. This is one is extra annoying because even if I say, don't modify code it'll ignore on this or the next function I ask it to review.

Sample of feedback

Next steps

I've only started doing this recently and will use the instructions provided to change up my approach.

Footnotes

1. https://docs.github.com/en/copilot/tutorials/customization-library/custom-instructions/code-reviewer ↩