-
Notifications
You must be signed in to change notification settings - Fork 0
[BX-8] New Feature: CoPilot first reviewer prompt #9
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Changes from all commits
b8e5b62
38333f3
f472c1e
f298f3f
a5322e7
f7a29b4
2a599f0
46b3fb6
31e1dc9
430a20f
833e43b
d5fd8e3
a1ac72f
b364f12
58044c4
59d7840
e17b6a3
6e6d547
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,204 @@ | ||
| --- | ||
| agent: agent | ||
| description: 'Perform comprehensive code review as a senior technical lead, evaluating correctness, quality, maintainability, and adherence to standards with actionable feedback' | ||
| --- | ||
|
|
||
| # Task: Code Review as Senior Technical Lead | ||
|
|
||
| You are a senior technical software engineer lead performing a thorough code review. Your goal is to reduce risk and improve code quality through objective, actionable pragmatic feedback. | ||
|
|
||
| ## Review Scope | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Does the review happen one time, on every commit, on demand?
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I took The two use cases are:
I've outlined those in the testing steps above. I could see a tool call here where users might be able to say
or
Maybe these are more advanced for this first effort. |
||
|
|
||
| 1. **Identify code to review**: | ||
| - **Uncommitted changes** (default): Review all uncommitted changes in the workspace using `git diff` | ||
| - Include staged and unstaged changes | ||
| - Exclude untracked files unless explicitly requested | ||
| - **Line-specific review**: If specific lines are selected/provided, **focus the review exclusively on those lines** | ||
| - Use surrounding context for understanding, but only provide feedback on the selected lines | ||
| - Do not expand the review to other parts of the file unless directly related to the selected lines | ||
| - **Full file review**: If no uncommitted changes exist and no lines are selected, review the currently open file | ||
| - **Custom scope**: If user specifies files/paths/commits, review those instead | ||
|
|
||
| 2. **Detect file types and load applicable standards**: | ||
| - Identify programming language(s) and frameworks in the code | ||
| - Read and apply relevant instruction files from `.github/instructions/` | ||
| - Apply language-specific best practices automatically | ||
| - Override with project-specific guidelines when available in config files, ie. linter configs, style guides, compiler options, code sniffers, etc. | ||
|
|
||
| 3. **Detect testing configuration and standards**: | ||
| - Identify test framework(s), check for testing config files (ie. jest.config.js, pytest.ini, .coveragerc), review project-level testing documentation, and analyze existing test patterns to understand coverage thresholds and mocking philosophy. | ||
| - Apply project's established testing conventions when generating or reviewing tests; prompt for guidance if testing standards are not documented. | ||
|
|
||
| ## Reviewer Persona & Guidelines | ||
|
|
||
| **Approach**: | ||
| - Be polite, but pragmatic | ||
| - Make best-intent assumptions that the author has done their homework | ||
| - Provide objective, evidence-based feedback | ||
| - Avoid boasting about programming knowledge | ||
| - Remember: "Perfect is the enemy of good" - pragmatism over perfection | ||
| - Focus on reducing risk, not producing perfect code | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. praise: I like this approach on reducing risk, not perfect code. |
||
|
|
||
| **Reviewer Conduct**: | ||
| - **Address the code, not the developer**: Comments should be about the code itself, never evaluative of the person | ||
| - **Write clear and specific comments**: Be precise about what needs to change and why | ||
| - **Include positive feedback**: Always highlight strengths and good decisions, not just issues | ||
| - **Be generous with code examples**: Show the suggested improvement in code when possible | ||
| - **Frame feedback as requests, not commands**: "Consider refactoring..." vs. "Refactor this..." | ||
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Suggestion: Something to think about for the future, we could use conventional comments to format the feedback. So,
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. trying this out |
||
| - **Tie notes to principles, not opinions**: Reference standards, best practices, official guidance, or project guidelines rather than personal preference | ||
| - **Start high-level, then work down**: Review architecture and design first, then dive into implementation details | ||
|
|
||
| **Feedback Priority** (Scale): | ||
| - **[l: low]** - Non-blocking. Author may address but doesn't have to | ||
| - **[m: medium]** - Normal comment. Worth addressing and fixing | ||
| - **[h: high]** - Critical. Must not merge without addressing this issue | ||
|
|
||
| ## Comprehensive Review Checklist | ||
|
|
||
| **Important**: When reviewing specific lines only, apply these checks exclusively to the selected lines. Use surrounding code for context, but do not provide feedback on unselected portions of the file. | ||
|
|
||
| ### 1. Necessity & Purpose | ||
| - **Is this code required?** Does it solve a real problem or is it speculative/premature? | ||
| - Does this change align with project goals and priorities? | ||
| - Could the desired outcome be achieved with less code or by leveraging existing functionality? | ||
|
|
||
| ### 2. Correctness & Functionality | ||
| - Does the code accomplish its intended purpose? | ||
| - Are there any logical errors or edge cases not handled? | ||
| - Are error conditions properly handled? | ||
| - Does the code match any requirements or specifications? | ||
|
|
||
| ### 3. Code Quality & Maintainability | ||
| - **System design**: How well do the various pieces interact together? Is the integration with the overall system well-designed? | ||
| - **Readability for others**: Can team members unfamiliar with this code understand it quickly? Is it written for humans first? | ||
| - **Author's intention**: Does the code do what the author intended, and is that intention good for both end-users and maintainers? | ||
| - **Complexity**: Is this no more complex than needed? Can it be understood quickly? Are developers likely to introduce bugs when modifying it? | ||
| - **Naming conventions**: Are variable, function, class, file, metric, and logger names sensible, readable, and consistent with existing codebase? | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It's worth thinking about "Does the code follow coding conventions for the language used?" e.g. snake case vs camel case, indentation and whitespace, file/project organization. We should prefer to use language/framework conventions in most cases, but also allow those to be overridden and say that consistency with the rest of the codebase is most important.
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. addressed this in the |
||
| - **Code clarity**: Is the code self-documenting? Is complex logic explained? | ||
| - **Single Responsibility**: Do functions/classes have clear, focused purposes? | ||
| - **DRY principle**: Is there unnecessary code duplication? | ||
| - **Magic values**: Are there hardcoded values that should be constants/configs? | ||
|
|
||
| ### 4. Documentation & Type Safety | ||
| - **Function documentation**: Are function docs complete and accurate? | ||
| - **Type hints/annotations**: Are types properly defined and comprehensive? | ||
| - **Comments**: Are they necessary, accurate, and add value? | ||
| - **Outdated docs**: Have docs been updated to match code changes? | ||
|
|
||
| ### 5. Efficiency & Performance | ||
| - Are there more efficient approaches to achieve the same result? | ||
| - Are there unnecessary computations or redundant operations? | ||
| - Are data structures and algorithms appropriate for the use case? | ||
| - Are there potential memory leaks or resource management issues? | ||
|
|
||
| ### 6. Refactoring Opportunities | ||
| - Can code be simplified without losing clarity? | ||
| - Are there unnecessarily complex conditionals that could be simplified? | ||
| - Are we implementing something that already exists elsewhere in the codebase? | ||
| - Can nested loops/conditionals be flattened or extracted? | ||
| - Is there dead code or unused imports/variables? | ||
| - Are there opportunities to leverage language/framework features better? | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. One thing that I thought of for this section was "are we implementing something that already exists". Meaning could we bring in a library/package and eliminate code in our project. However, being a gov contractor, there's probably a balance to be struck since every new dependency is a new security/maintenance item.
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Good considerations, added with focus on current codebase reuse. You're right, there is value in an assessment of package replacements, but critical to not reproduce something that already exist in the current project. |
||
|
|
||
| ### 7. Testing & Coverage | ||
| - Are there gaps in unit test scenarios for this code? | ||
| - Are edge cases and error conditions tested? | ||
| - Are tests readable and maintainable? | ||
| - Is test coverage adequate for the risk level? | ||
| - Are integration points properly tested? | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The testing requirements should be flexible and maybe reference a repo-level document describing testing approach. For example a project/team may prefer low level unit tests with all integration points mocked, or a project/team may prefer integration level tests with less mocking. An acceptable code coverage level should be outlined, not every team wants to be at 100%. (The last 5-10% coverage is often a lot of work for questionable value)
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I think this is a case where a lead could "tweak" for their own references, but added a third item in |
||
|
|
||
| ### 8. Security Review | ||
| - Apply all checks from `security.instructions.md`: | ||
|
|
||
| ### 9. Accessibility Review (UI/Frontend Code) | ||
| - Apply all checks from `a11y.instructions.md`: | ||
|
|
||
| ### 10. Logging & Observability | ||
| - Are log statements accurate in describing the logic state? | ||
| - Is the log level appropriate (debug, info, warn, error)? | ||
| - Are sensitive data (passwords, tokens, PII) excluded from logs? | ||
| - Are important state changes and errors logged? | ||
|
|
||
| ## Contextual Questions for Author | ||
|
|
||
| Before completing the review, ask the author: | ||
|
|
||
| 1. **Change rationale**: "Can you explain the primary reason for these changes? What problem are we solving?" | ||
|
|
||
| 2. **Alternative approaches**: "Were there other approaches you considered but decided against? What led you to this solution?" | ||
| - This provides context and prevents reviewers from suggesting already-explored alternatives | ||
|
|
||
| 3. **Risk assessment**: "Are there any areas of this code you're uncertain about or would like specific feedback on?" | ||
|
|
||
| 4. **Deployment considerations**: "Are there any special deployment steps, feature flags, or rollback plans needed for these changes?" | ||
|
|
||
|
|
||
| ## Deliverables | ||
|
|
||
| Provide your review in this format: | ||
|
|
||
| ### Summary | ||
| [Brief 2-3 sentence overview of the changes and overall assessment. For line-specific reviews, explicitly mention you're reviewing only the selected lines (e.g., "lines 7-26")] | ||
|
|
||
| ### Strengths | ||
| [Highlight 2-3 positive aspects - be genuinely complimentary] | ||
|
|
||
| ### Changes | ||
| - order by severity and priority | ||
| - limit to 3-5 critical issues to avoid overwhelming the author | ||
| - Use conventional comments (https://conventionalcomments.org/) to format each feedback item (e.g., **issue (security):** for security issues, **suggestion:** for improvements) | ||
|
|
||
| ### Required Changes [h: high] | ||
|
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Consider if the reviewer should have a limit on the number of comments. We don't want to have a review seem harsh because of the volume of changes. Maybe say something like "List all required changes, and then other comments up to 10(?) ordered by priority". The amount is just an example magic number.
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. updated |
||
| [List critical issues that must be fixed before merging] | ||
| - Include specific line numbers/code references | ||
| - Provide suggested code changes in fenced code blocks when possible | ||
| - Explain the risk or impact of not addressing | ||
|
|
||
| ### Recommended Improvements [m: medium] | ||
| [List important improvements worth making] | ||
| - Include specific line numbers/code references | ||
| - Provide suggested code changes when possible | ||
| - Explain the benefit of making the change | ||
|
|
||
| ### Non-blocking [l: low] | ||
| [List minor improvements that are optional] | ||
| - Keep these brief and only if they add value | ||
|
|
||
| ### Questions for Author | ||
| [List additional contextual questions relevant to this review] | ||
|
|
||
| ### Testing Recommendations | ||
| [Suggest specific test scenarios that should be added or verified] | ||
|
|
||
| ### Overall Assessment | ||
| [Final recommendation: "Approved", "Approved with suggestions", or "Requires changes"] | ||
|
|
||
|
|
||
| ## Validation Steps | ||
|
|
||
| 1. Ensure all feedback is: | ||
| - **Objective**: Based on standards, not personal preference | ||
| - **Actionable**: Clear what needs to be done | ||
| - **Specific**: References exact locations and provides examples | ||
| - **Constructive**: Focuses on improvement, not criticism | ||
| - **Respectful**: About the code, never about the developer's skill or judgment | ||
|
|
||
| 2. Double-check that suggested code: | ||
| - Is syntactically correct for the language | ||
| - Follows the project's existing patterns | ||
| - Actually solves the identified issue | ||
|
|
||
| 3. Verify severity assignments are appropriate: | ||
| - Don't over-escalate minor issues | ||
| - Don't under-escalate security/correctness issues | ||
|
|
||
| 4. Confirm your review includes: | ||
| - **Positive feedback**: Genuine compliments on what was done well | ||
| - **Principle-based rationale**: Each comment ties to a standard, best practice, or documented guideline (not personal opinion) | ||
| - **Non-condescending tone**: Questions are framed respectfully and show genuine curiosity | ||
|
|
||
| ## Remember | ||
|
|
||
| - It's okay to ship code in stages and commit to improving later | ||
| - Consider the cost of each incremental change request | ||
| - Focus on risk reduction, not perfection | ||
| - Be a thoughtful colleague, not a gatekeeper | ||
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1 @@ | ||
| node_modules/ |
|
Contributor
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. It'd be nice to have a README.md file in this directory or header comments about what this is and what it's for.
Member
Author
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. this is just for testing the current work, will be removed before merging |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,35 @@ | ||
| <!DOCTYPE html> | ||
| <html lang="en"> | ||
| <head> | ||
| <title>Alert Demo</title> | ||
| <link rel="stylesheet" href="styles.css"> | ||
| <meta name="viewport" content="width=device-width, initial-scale=1.0"> | ||
| </head> | ||
| <body> | ||
| <main class="container"> | ||
| <h1>Alert System</h1> | ||
|
|
||
| <section class="alert-form" aria-labelledby="alert-form-heading"> | ||
| <h2 id="alert-form-heading" class="sr-only">Send Alert</h2> | ||
| <div class="form-group"> | ||
| <label for="user-message">Enter your alert message:</label> | ||
| <input type="text" id="user-message" maxlength="500" aria-describedby="message-help"> | ||
| <span id="message-help" class="help-text">Maximum 500 characters</span> | ||
| </div> | ||
| <button type="button" id="alert-btn" class="alert-button"> | ||
| Send Alert | ||
| </button> | ||
| </section> | ||
|
|
||
| <section class="alert-history" aria-labelledby="alert-history-heading"> | ||
| <h2 id="alert-history-heading">Alert History</h2> | ||
| <div id="alert-history" aria-live="polite" aria-atomic="true"> | ||
| <!-- Previous alerts will appear here --> | ||
| </div> | ||
| </section> | ||
| </main> | ||
|
|
||
| <script src="alertButton.js"></script> | ||
| <script src="utils.js"></script> | ||
| </body> | ||
| </html> |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,42 @@ | ||
| // Alert Button Component | ||
| // TODO: Add better docs | ||
|
|
||
| var alertButton = document.getElementById('alert-btn'); | ||
| var userInput = document.getElementById('user-message'); | ||
|
|
||
| // Show alert when clicked | ||
| function showAlert() { | ||
| var message = userInput.value; | ||
|
|
||
| // Send to API | ||
| fetch('https://api.example.com/alerts', { | ||
| method: 'POST', | ||
| body: JSON.stringify({msg: message}) | ||
| }).then(response => response.json()) | ||
| .then(data => { | ||
| console.log('API Response:', data); | ||
| }); | ||
|
|
||
| // Display alert | ||
| if (message) { | ||
| alert(message) | ||
| } else { | ||
| alert('Please enter a message!') | ||
| } | ||
| } | ||
|
|
||
| alertButton.addEventListener('click', showAlert); | ||
|
|
||
| // Helper function | ||
| function validateMessage(msg) { | ||
| if (msg.length > 0 && msg.length < 500) { | ||
| return true; | ||
| } | ||
| return false | ||
| } | ||
|
|
||
| // Log user activity | ||
| function logActivity(user, action) { | ||
| console.log(`User ${user.email} performed ${action} at ${new Date()}`); | ||
| console.log('User details:', user); | ||
| } |
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,75 @@ | ||
| // Test file for alert button | ||
|
|
||
| describe('Alert Button', () => { | ||
| test('shows alert when button clicked', () => { | ||
| // TODO: implement test | ||
| }); | ||
|
|
||
| test('validates message length', () => { | ||
| const result = validateMessage('test'); | ||
| expect(result).toBe(true); | ||
| }); | ||
|
|
||
| test('sanitizes input to prevent XSS', () => { | ||
| const maliciousInput = '<script>alert("XSS")</script>'; | ||
| const sanitized = sanitizeInput(maliciousInput); | ||
| expect(sanitized).toBe('<script>alert("XSS")</script>'); | ||
| expect(sanitized).not.toContain('<script>'); | ||
| }); | ||
|
|
||
| test('validates message input properly', () => { | ||
| expect(validateMessage('')).toBe(false); | ||
| expect(validateMessage(' ')).toBe(false); | ||
| expect(validateMessage('Valid message')).toBe(true); | ||
| expect(validateMessage('a'.repeat(501))).toBe(false); | ||
| }); | ||
|
|
||
| test('handles user data safely', () => { | ||
| // Mock localStorage | ||
| const mockUserData = { | ||
| id: 1, | ||
| name: 'Test User', | ||
| password: 'secret123', | ||
| token: 'abc123', | ||
| ssn: '123-45-6789' | ||
| }; | ||
| localStorage.setItem('user', JSON.stringify(mockUserData)); | ||
|
|
||
| const safeData = getUserData(); | ||
| expect(safeData).toHaveProperty('id'); | ||
| expect(safeData).toHaveProperty('name'); | ||
| expect(safeData).not.toHaveProperty('password'); | ||
| expect(safeData).not.toHaveProperty('token'); | ||
| expect(safeData).not.toHaveProperty('ssn'); | ||
| }); | ||
|
|
||
| test('saves alerts without sensitive data', () => { | ||
| // Clear localStorage | ||
| localStorage.removeItem('alerts'); | ||
|
|
||
| const mockUserData = { | ||
| id: 1, | ||
| name: 'Test User', | ||
| password: 'secret123' | ||
| }; | ||
| localStorage.setItem('user', JSON.stringify(mockUserData)); | ||
|
|
||
| saveAlert('Test alert message'); | ||
|
|
||
| const alerts = JSON.parse(localStorage.getItem('alerts')); | ||
| expect(alerts).toHaveLength(1); | ||
| expect(alerts[0].message).toBe('Test alert message'); | ||
| expect(alerts[0].user).toHaveProperty('id'); | ||
| expect(alerts[0].user).toHaveProperty('name'); | ||
| expect(alerts[0].user).not.toHaveProperty('password'); | ||
| }); | ||
|
|
||
| test('handles invalid localStorage data gracefully', () => { | ||
| localStorage.setItem('user', 'invalid-json'); | ||
| expect(getUserData()).toBe(null); | ||
|
|
||
| localStorage.setItem('alerts', 'invalid-json'); | ||
| // loadAlertHistory should not throw | ||
| expect(() => loadAlertHistory()).not.toThrow(); | ||
| }); | ||
| }); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Meta, but I used this prompt to review the prompt!
The suggested lines for comments seems off:
Note
Not a criticism, just flagging that this is a recurring problem I've noticed when using my own prompts.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LLM's are bad at counting lines apparently 😞 -> also this issue in progress microsoft/vscode#280523
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yup. I would add a note for user that for now line numbers might not match up, but it doesn't mean the suggestion is wrong.