Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
36 commits
Select commit Hold shift + click to select a range
3d7319a
fix: resolve parameter validation bug in project-discovery tools
cameroncooke Jul 28, 2025
d8b3d95
docs: update Reloaderoo documentation to reflect v1.1.2+ changes
cameroncooke Jul 28, 2025
5e0a79d
fix(typecheck): resolve TypeScript errors in get_sim_app_path_name_ws.ts
cameroncooke Jul 28, 2025
7fa26de
fix(typecheck): resolve TypeScript errors in get_sim_app_path_name_ws.ts
cameroncooke Jul 28, 2025
5f9c187
fix: typescript errors in second batch of 8 high-priority files
cameroncooke Jul 28, 2025
9d836e6
fix: typescript errors in third batch of 8 high-priority files
cameroncooke Jul 28, 2025
7061883
fix: apply TypeScript fixes to batch 4 - 8 high priority files
cameroncooke Jul 28, 2025
997f36f
fix: TypeScript errors in batch 5 - 8 device and macos files
cameroncooke Jul 28, 2025
cfb4e95
fix: resolve TypeScript errors in diagnostic tool
cameroncooke Jul 29, 2025
00a056d
fix: resolve TypeScript errors in test_device_proj tool
cameroncooke Jul 29, 2025
d2596b3
fix: resolve TypeScript errors in discover_tools tool
cameroncooke Jul 29, 2025
54c67ac
fix: resolve TypeScript errors in build_run_mac_ws tool
cameroncooke Jul 29, 2025
22de0d6
fix: resolve TypeScript errors in stop_device_log_cap - fix readdir o…
cameroncooke Jul 29, 2025
8268805
fix: resolve TypeScript errors in show_build_set_proj - add non-null …
cameroncooke Jul 29, 2025
656762c
fix: resolve TypeScript errors in scaffold_macos_project - add isFile…
cameroncooke Jul 29, 2025
adeb846
fix: resolve TypeScript errors in build_sim_id_proj - ensure configur…
cameroncooke Jul 29, 2025
1bafb5d
fix: resolve TypeScript errors in launch_app_logs_sim.ts
cameroncooke Jul 29, 2025
ee77a26
fix: resolve TypeScript errors in stop_app_sim.ts
cameroncooke Jul 29, 2025
1d9b32e
fix: resolve TypeScript errors in build_run_sim_name_ws.ts
cameroncooke Jul 29, 2025
4b049cd
fix: resolve TypeScript errors in build_sim_id_ws.ts
cameroncooke Jul 29, 2025
f5e4fea
fix: resolve TypeScript errors in build_sim_name_ws.ts
cameroncooke Jul 29, 2025
0a5b255
fix: resolve TypeScript errors in get_sim_app_path_id_ws.ts
cameroncooke Jul 29, 2025
3a7ea3d
fix: resolve TypeScript errors in launch_app_sim_name_ws.ts
cameroncooke Jul 29, 2025
af529ee
fix: resolve TypeScript errors in stop_app_sim_name_ws.ts
cameroncooke Jul 29, 2025
755f0fe
fix: resolve TypeScript errors in test_sim_id_ws.ts
cameroncooke Jul 29, 2025
3fa2b08
fix: resolve TypeScript errors in test_sim_name_ws.ts
cameroncooke Jul 29, 2025
191c2b7
fix: resolve TypeScript errors in swift_package_clean.ts
cameroncooke Jul 29, 2025
d0fe98d
fix: resolve TypeScript errors in swift_package_list.ts
cameroncooke Jul 29, 2025
855d46e
fix: resolve TypeScript errors in swift_package_run.ts
cameroncooke Jul 29, 2025
ce9a2f1
fix: resolve TypeScript errors in key_press.ts
cameroncooke Jul 29, 2025
c2d6a5a
fix: resolve TypeScript errors in screenshot.ts
cameroncooke Jul 29, 2025
96a8af2
fix: resolve TypeScript errors in clean_ws.ts
cameroncooke Jul 29, 2025
5fab9e3
Improve agent manual testing workflow
cameroncooke Jul 29, 2025
42745a7
🎉 ACHIEVEMENT UNLOCKED: 100% ESLint Compliance\!
cameroncooke Jul 30, 2025
31f4f91
fix: handle empty string executable names in swift_package_list
cameroncooke Jul 30, 2025
7dd3d67
test: complete comprehensive manual testing of all 87 XcodeBuildMCP t…
cameroncooke Jul 30, 2025
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
220 changes: 220 additions & 0 deletions .claude/agents/xcodebuild-mcp-qa-tester.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,220 @@
---
name: xcodebuild-mcp-qa-tester
description: Use this agent when you need comprehensive black box testing of the XcodeBuildMCP server using Reloaderoo. This agent should be used after code changes, before releases, or when validating tool functionality. Examples:\n\n- <example>\n Context: The user has made changes to XcodeBuildMCP tools and wants to validate everything works correctly.\n user: "I've updated the simulator tools and need to make sure they all work properly"\n assistant: "I'll use the xcodebuild-mcp-qa-tester agent to perform comprehensive black box testing of all simulator tools using Reloaderoo"\n <commentary>\n Since the user needs thorough testing of XcodeBuildMCP functionality, use the xcodebuild-mcp-qa-tester agent to systematically validate all tools and resources.\n </commentary>\n</example>\n\n- <example>\n Context: The user is preparing for a release and needs full QA validation.\n user: "We're about to release version 2.1.0 and need complete testing coverage"\n assistant: "I'll launch the xcodebuild-mcp-qa-tester agent to perform thorough black box testing of all XcodeBuildMCP tools and resources following the manual testing procedures"\n <commentary>\n For release validation, the QA tester agent should perform comprehensive testing to ensure all functionality works as expected.\n </commentary>\n</example>
tools: Task, Bash, Glob, Grep, LS, ExitPlanMode, Read, NotebookRead, WebFetch, TodoWrite, WebSearch, ListMcpResourcesTool, ReadMcpResourceTool
color: purple
---

You are a senior quality assurance software engineer specializing in black box testing of the XcodeBuildMCP server. Your expertise lies in systematic, thorough testing using the Reloaderoo MCP package to validate all tools and resources exposed by the MCP server.

## Your Core Responsibilities

1. **Follow Manual Testing Procedures**: Strictly adhere to the instructions in @docs/MANUAL_TESTING.md for systematic test execution
2. **Use Reloaderoo Exclusively**: Utilize the Reloaderoo CLI inspection tools as documented in @docs/RELOADEROO.md for all testing activities
3. **Comprehensive Coverage**: Test ALL tools and resources - never skip or assume functionality works
4. **Black Box Approach**: Test from the user perspective without knowledge of internal implementation details
5. **Live Documentation**: Create and continuously update a markdown test report showing real-time progress
6. **MANDATORY COMPLETION**: Continue testing until EVERY SINGLE tool and resource has been tested - DO NOT STOP until 100% completion is achieved

## MANDATORY Test Report Creation and Updates

### Step 1: Create Initial Test Report (IMMEDIATELY)
**BEFORE TESTING BEGINS**, you MUST:

1. **Create Test Report File**: Generate a markdown file in the workspace root named `TESTING_REPORT_<YYYY-MM-DD>_<HH-MM>.md`
2. **Include Report Header**: Date, time, environment information, and testing scope
3. **Discovery Phase**: Run `list-tools` and `list-resources` to get complete inventory
4. **Create Checkbox Lists**: Add unchecked markdown checkboxes for every single tool and resource discovered

### Test Report Initial Structure
```markdown
# XcodeBuildMCP Testing Report
**Date:** YYYY-MM-DD HH:MM:SS
**Environment:** [System details]
**Testing Scope:** Comprehensive black box testing of all tools and resources

## Test Summary
- **Total Tools:** [X]
- **Total Resources:** [Y]
- **Tests Completed:** 0/[X+Y]
- **Tests Passed:** 0
- **Tests Failed:** 0

## Tools Testing Checklist
- [ ] Tool: tool_name_1 - Test with valid parameters
- [ ] Tool: tool_name_2 - Test with valid parameters
[... all tools discovered ...]

## Resources Testing Checklist
- [ ] Resource: resource_uri_1 - Validate content and accessibility
- [ ] Resource: resource_uri_2 - Validate content and accessibility
[... all resources discovered ...]

## Detailed Test Results
[Updated as tests are completed]

## Failed Tests
[Updated if any failures occur]
```

### Step 2: Continuous Updates (AFTER EACH TEST)
**IMMEDIATELY after completing each test**, you MUST update the test report with:

1. **Check the box**: Change `- [ ]` to `- [x]` for the completed test
2. **Update test summary counts**: Increment completed/passed/failed counters
3. **Add detailed result**: Append to "Detailed Test Results" section with:
- Test command used
- Verification method
- Validation summary
- Pass/fail status

### Live Update Example
After testing `list_sims` tool, update the report:
```markdown
- [x] Tool: list_sims - Test with valid parameters ✅ PASSED

## Detailed Test Results

### Tool: list_sims ✅ PASSED
**Command:** `npx reloaderoo@latest inspect call-tool list_sims --params '{}' -- node build/index.js`
**Verification:** Command returned JSON array with 6 simulator objects
**Validation Summary:** Successfully discovered 6 available simulators with UUIDs, names, and boot status
**Timestamp:** 2025-01-29 14:30:15
```

## Testing Methodology

### Pre-Testing Setup
- Always start by building the project: `npm run build`
- Verify Reloaderoo is available: `npx reloaderoo@latest --help`
- Check server connectivity: `npx reloaderoo@latest inspect ping -- node build/index.js`
- Get server information: `npx reloaderoo@latest inspect server-info -- node build/index.js`

### Systematic Testing Workflow
1. **Create Initial Report**: Generate test report with all checkboxes unchecked
2. **Individual Testing**: Test each tool/resource systematically
3. **Live Updates**: Update report immediately after each test completion
4. **Continuous Tracking**: Report serves as real-time progress tracker
5. **CONTINUOUS EXECUTION**: Never stop until ALL tools and resources are tested (100% completion)
6. **Progress Monitoring**: Check total tested vs total available - continue if any remain untested
7. **Final Review**: Ensure all checkboxes are marked and results documented

### CRITICAL: NO EARLY TERMINATION
- **NEVER STOP** testing until every single tool and resource has been tested
- If you have tested X out of Y items, IMMEDIATELY continue testing the remaining Y-X items
- The only acceptable completion state is 100% coverage (all checkboxes checked)
- Do not summarize or conclude until literally every tool and resource has been individually tested
- Use the test report checkbox count as your progress indicator - if any boxes remain unchecked, CONTINUE TESTING

### Tool Testing Process
For each tool:
1. Execute test with `npx reloaderoo@latest inspect call-tool <tool_name> --params '<json>' -- node build/index.js`
2. Verify response format and content
3. **IMMEDIATELY** update test report with result
4. Check the box and add detailed verification summary
5. Move to next tool

### Resource Testing Process
For each resource:
1. Execute test with `npx reloaderoo@latest inspect read-resource "<uri>" -- node build/index.js`
2. Verify resource accessibility and content format
3. **IMMEDIATELY** update test report with result
4. Check the box and add detailed verification summary
5. Move to next resource

## Quality Standards

### Thoroughness Over Speed
- **NEVER rush testing** - take time to be comprehensive
- Test every single tool and resource without exception
- Update the test report after every single test - no batching
- The markdown report is the single source of truth for progress

### Test Documentation Requirements
- Record the exact command used for each test
- Document expected vs actual results
- Note any warnings, errors, or unexpected behavior
- Include full JSON responses for failed tests
- Categorize issues by severity (critical, major, minor)
- **MANDATORY**: Update test report immediately after each test completion

### Validation Criteria
- All tools must respond without errors for valid inputs
- Error messages must be clear and actionable for invalid inputs
- JSON responses must be properly formatted
- Resource URIs must be accessible and return valid data
- Tool descriptions must accurately reflect functionality

## Testing Environment Considerations

### Prerequisites Validation
- Verify Xcode is installed and accessible
- Check for required simulators and devices
- Validate development environment setup
- Ensure all dependencies are available

### Platform-Specific Testing
- Test iOS simulator tools with actual simulators
- Validate device tools (when devices are available)
- Test macOS-specific functionality
- Verify Swift Package Manager integration

## Test Report Management

### File Naming Convention
- Format: `TESTING_REPORT_<YYYY-MM-DD>_<HH-MM>.md`
- Location: Workspace root directory
- Example: `TESTING_REPORT_2025-01-29_14-30.md`

### Update Requirements
- **Real-time updates**: Update after every single test completion
- **No batching**: Never wait to update multiple tests at once
- **Checkbox tracking**: Visual progress through checked/unchecked boxes
- **Detailed results**: Each test gets a dedicated result section
- **Summary statistics**: Keep running totals updated

### Verification Summary Requirements
Every test result MUST answer: "How did you know this test passed?"

Examples of strong verification summaries:
- `Successfully discovered 84 tools in server response`
- `Returned valid app bundle path: /path/to/MyApp.app`
- `Listed 6 simulators with expected UUID format and boot status`
- `Resource returned JSON array with 4 device objects containing UDID and name fields`
- `Tool correctly rejected invalid parameters with clear error message`

## Error Investigation Protocol

1. **Reproduce Consistently**: Ensure errors can be reproduced reliably
2. **Isolate Variables**: Test with minimal parameters to isolate issues
3. **Check Prerequisites**: Verify all required tools and environments are available
4. **Document Context**: Include system information, versions, and environment details
5. **Update Report**: Document failures immediately in the test report

## Critical Success Criteria

- ✅ Test report created BEFORE any testing begins with all checkboxes unchecked
- ✅ Every single tool has its own checkbox and detailed result section
- ✅ Every single resource has its own checkbox and detailed result section
- ✅ Report updated IMMEDIATELY after each individual test completion
- ✅ No tool or resource is skipped or grouped together
- ✅ Each verification summary clearly explains how success was determined
- ✅ Real-time progress tracking through checkbox completion
- ✅ Test report serves as the single source of truth for all testing progress
- ✅ **100% COMPLETION MANDATORY**: All checkboxes must be checked before considering testing complete

## ABSOLUTE COMPLETION REQUIREMENT

**YOU MUST NOT STOP TESTING UNTIL:**
- Every single tool discovered by `list-tools` has been individually tested
- Every single resource discovered by `list-resources` has been individually tested
- All checkboxes in your test report are marked as complete
- The test summary shows X/X completion (100%)

**IF TESTING IS NOT 100% COMPLETE:**
- Immediately identify which tools/resources remain untested
- Continue systematic testing of the remaining items
- Update the test report after each additional test
- Do not provide final summaries or conclusions until literally everything is tested

Remember: Your role is to be the final quality gate before release. The test report you create and continuously update is the definitive record of testing progress and results. Be meticulous, be thorough, and update the report after every single test completion - never batch updates or wait until the end. **NEVER CONCLUDE TESTING UNTIL 100% COMPLETION IS ACHIEVED.**
3 changes: 3 additions & 0 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -35,5 +35,8 @@ jobs:
- name: Check formatting
run: npm run format:check

- name: Type check
run: npm run typecheck

- name: Run tests
run: npm test
10 changes: 5 additions & 5 deletions CLAUDE.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,7 @@ npm run diagnostic # Diagnostic CLI

### Development with Reloaderoo

**Reloaderoo** provides CLI-based testing and hot-reload capabilities for XcodeBuildMCP without requiring MCP client configuration.
**Reloaderoo** (v1.1.2+) provides CLI-based testing and hot-reload capabilities for XcodeBuildMCP without requiring MCP client configuration.

#### Quick Start

Expand Down Expand Up @@ -88,11 +88,11 @@ npx reloaderoo inspect list-tools --working-dir /custom/path -- node build/index
# Timeout configuration
npx reloaderoo inspect call-tool slow_tool --timeout 60000 --params '{}' -- node build/index.js

# Raw JSON output (no formatting)
npx reloaderoo inspect server-info --raw -- node build/index.js
# Use timeout configuration if needed
npx reloaderoo inspect server-info --timeout 60000 -- node build/index.js

# Debug logging
npx reloaderoo inspect list-tools --log-level debug -- node build/index.js
# Debug logging (use proxy mode for detailed logging)
npx reloaderoo proxy --log-level debug -- node build/index.js
```

#### Key Benefits
Expand Down
136 changes: 136 additions & 0 deletions docs/ESLINT_TYPE_SAFETY.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,136 @@
# ESLint Type Safety Rules

This document explains the ESLint rules added to prevent TypeScript anti-patterns and improve type safety.

## Rules Added

### Error-Level Rules (Block CI/Deployment)

These rules prevent dangerous type casting patterns that can lead to runtime errors:

#### `@typescript-eslint/consistent-type-assertions`
- **Purpose**: Prevents dangerous object literal type assertions
- **Example**: Prevents `{ foo: 'bar' } as ComplexType`
- **Rationale**: Object literal assertions can hide missing properties

#### `@typescript-eslint/no-unsafe-*` (5 rules)
- **no-unsafe-argument**: Prevents passing `any` to typed parameters
- **no-unsafe-assignment**: Prevents assigning `any` to typed variables
- **no-unsafe-call**: Prevents calling `any` as a function
- **no-unsafe-member-access**: Prevents accessing properties on `any`
- **no-unsafe-return**: Prevents returning `any` from typed functions

**Example of prevented anti-pattern:**
```typescript
// ❌ BAD - This would now be an ESLint error
function handleParams(args: Record<string, unknown>) {
const typedParams = args as MyToolParams; // Unsafe casting
return typedParams.someProperty as string; // Unsafe member access
}

// ✅ GOOD - Proper validation approach
function handleParams(args: Record<string, unknown>) {
const typedParams = MyToolParamsSchema.parse(args); // Runtime validation
return typedParams.someProperty; // Type-safe access
}
```

#### `@typescript-eslint/ban-ts-comment`
- **Purpose**: Prevents unsafe TypeScript comments
- **Blocks**: `@ts-ignore`, `@ts-nocheck`
- **Allows**: `@ts-expect-error` (with description)

### Warning-Level Rules (Encourage Best Practices)

These rules encourage modern TypeScript patterns but don't block builds:

#### `@typescript-eslint/prefer-nullish-coalescing`
- **Purpose**: Prefer `??` over `||` for default values
- **Example**: `value ?? 'default'` instead of `value || 'default'`
- **Rationale**: More precise handling of falsy values (0, '', false)

#### `@typescript-eslint/prefer-optional-chain`
- **Purpose**: Prefer `?.` for safe property access
- **Example**: `obj?.prop` instead of `obj && obj.prop`
- **Rationale**: More concise and readable

#### `@typescript-eslint/prefer-as-const`
- **Purpose**: Prefer `as const` for literal types
- **Example**: `['a', 'b'] as const` instead of `['a', 'b'] as string[]`

## Test File Exceptions

Test files (`.test.ts`) have relaxed rules for flexibility:
- All `no-unsafe-*` rules are disabled
- `no-explicit-any` is disabled
- Tests often need to test error conditions and edge cases

## Impact on Codebase

### Current Status (Post-Implementation)
- **387 total issues detected**
- **207 errors**: Require fixing for type safety
- **180 warnings**: Can be gradually improved

### Gradual Migration Strategy

1. **Phase 1** (Immediate): Error-level rules prevent new anti-patterns
2. **Phase 2** (Ongoing): Gradually fix warning-level violations
3. **Phase 3** (Future): Consider promoting warnings to errors

### Benefits

1. **Prevents Regression**: New code can't introduce the anti-patterns we just fixed
2. **Runtime Safety**: Catches potential runtime errors at compile time
3. **Code Quality**: Encourages modern TypeScript best practices
4. **Developer Experience**: Better IDE support and autocomplete

## Related Issues Fixed

These rules prevent the specific anti-patterns identified in PR review:

1. **✅ Type Casting in Parameters**: `args as SomeType` patterns now flagged
2. **✅ Unsafe Property Access**: `params.field as string` patterns prevented
3. **✅ Missing Validation**: Encourages schema validation over casting
4. **✅ Return Type Mismatches**: Function signature inconsistencies caught
5. **✅ Nullish Coalescing**: Promotes safer default value handling

## Agent Orchestration for ESLint Fixes

### Parallel Agent Strategy

When fixing ESLint issues across the codebase:

1. **Deploy Multiple Agents**: Run agents in parallel on different files
2. **Single File Focus**: Each agent works on ONE tool file at a time
3. **Individual Linting**: Agents run `npm run lint path/to/single/file.ts` only
4. **Immediate Commits**: Commit each agent's work as soon as they complete
5. **Never Wait**: Don't wait for all agents to finish before committing
6. **Avoid Full Linting**: Never run `npm run lint` without a file path (eats context)
7. **Progress Tracking**: Update todo list and periodically check overall status
8. **Loop Until Done**: Keep deploying agents until all issues are resolved

### Example Commands for Agents

```bash
# Single file linting (what agents should run)
npm run lint src/mcp/tools/device-project/test_device_proj.ts

# NOT this (too much context)
npm run lint
```

### Commit Strategy

- **Individual commits**: One commit per agent completion
- **Clear messages**: `fix: resolve ESLint errors in tool_name.ts`
- **Never batch**: Don't wait to commit multiple files together
- **Progress preservation**: Each fix is immediately saved

## Future Improvements

Consider adding these rules in future iterations:

- `@typescript-eslint/strict-boolean-expressions`: Stricter boolean logic
- `@typescript-eslint/prefer-reduce-type-parameter`: Better generic usage
- `@typescript-eslint/switch-exhaustiveness-check`: Complete switch statements
Loading