fix: handle iOS 26.0 simulators missing from list_sims output #118
Conversation
Work around Apple simctl JSON bug where duplicate runtime IDs (multiple iOS 26.0 betas with same com.apple.CoreSimulator.SimRuntime.iOS-26-0 identifier) cause JSON output collision, leaving one runtime's devices as empty array. Solution: Hybrid parsing approach that combines JSON (primary) and text (fallback) outputs, merging results to capture all available simulators across all runtimes.
WalkthroughAttempts JSON simulator listing first and falls back to parsing plain-text output on parse/error. Adds Changes
Sequence Diagram(s)sequenceDiagram
participant User
participant ListLogic as list_simsLogic
participant Executor as "simctl Executor"
participant JSONParser as "JSON parse"
participant TextParser as "parseTextOutput"
participant Formatter
User->>ListLogic: request list simulators
ListLogic->>Executor: xcrun simctl list devices --json
Executor-->>ListLogic: stdout (JSON or invalid) / error
alt JSON parse succeeds
ListLogic->>JSONParser: parse JSON
JSONParser-->>ListLogic: devices(JSON)
else JSON parse fails or Executor error
ListLogic->>Executor: xcrun simctl list devices
Executor-->>ListLogic: stdout (text) / error
ListLogic->>TextParser: parse text
TextParser-->>ListLogic: devices(text)
end
note right of ListLogic: Merge devices (prefer JSON, de-dup by UUID)
ListLogic->>Formatter: group by runtime, filter unavailable, mark Booted/Shutdown
Formatter-->>User: formatted device list + "Next Steps"
Estimated code review effort🎯 3 (Moderate) | ⏱️ ~25 minutes Possibly related PRs
Poem
Pre-merge checks and finishing touches❌ Failed checks (1 warning)
✅ Passed checks (2 passed)
✨ Finishing touches
🧪 Generate unit tests
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
|
Cursor review |
|
Skipping Bugbot: Unable to authenticate your request. Please make sure Bugbot is properly installed and configured for this repository. |
|
@cameroncooke please have a look |
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
There was a problem hiding this comment.
Actionable comments posted: 0
🧹 Nitpick comments (2)
src/mcp/tools/simulator/list_sims.ts (1)
28-63: Verify indentation stability and consider more robust parsing.The text parser makes several assumptions that could be fragile:
- Line 44-46: The device regex requires exactly 4 spaces (
^\s{4}). If Apple changes simctl's indentation, parsing will break silently.- Line 47: Devices can be added to the list even if
currentRuntimeis empty (before any runtime header is seen). This could result in devices withruntime: ''being added.- Line 49: The unavailable check uses
line.includes('unavailable')which could theoretically match device names containing "unavailable" (though unlikely in practice).Consider these improvements:
function parseTextOutput(textOutput: string): SimulatorDevice[] { const devices: SimulatorDevice[] = []; const lines = textOutput.split('\n'); let currentRuntime = ''; for (const line of lines) { // Match runtime headers like "-- iOS 26.0 --" or "-- iOS 18.6 --" const runtimeMatch = line.match(/^-- ([\w\s.]+) --$/); if (runtimeMatch) { currentRuntime = runtimeMatch[1]; continue; } // Match device lines like " iPhone 17 Pro (UUID) (Booted)" // UUID pattern is flexible to handle test UUIDs like "test-uuid-123" const deviceMatch = line.match( - /^\s{4}(.+?)\s+\(([^)]+)\)\s+\((Booted|Shutdown|Booting|Shutting Down)\)(?:\s+\(unavailable.*\))?$/i, + /^\s+(.+?)\s+\(([^)]+)\)\s+\((Booted|Shutdown|Booting|Shutting Down)\)(\s+\(unavailable.*\))?$/i, ); - if (deviceMatch && currentRuntime) { + if (deviceMatch && currentRuntime) { const [, name, udid, state] = deviceMatch; - const isUnavailable = line.includes('unavailable'); + const isUnavailable = Boolean(deviceMatch[4]); if (!isUnavailable) { devices.push({ name: name.trim(), udid, state, isAvailable: true, runtime: currentRuntime, }); } } } return devices; }Changes:
- Replace
\s{4}with\s+to accept any indentation- Capture unavailable suffix in group 4 and check it directly instead of
line.includes()- The
currentRuntimeguard on line 47 already prevents devices without runtimesrc/mcp/tools/simulator/__tests__/list_sims.test.ts (1)
51-131: LGTM, but consider adding a merge scenario test.The test correctly verifies the dual-path execution and updated output format. However, this test has identical devices in both JSON and text outputs, so it doesn't exercise the merge logic that adds text-only devices (the core bug fix).
Consider adding a test where the text output contains devices NOT present in the JSON output to verify the merge logic:
it('should merge devices from text that are missing from JSON', async () => { const mockJsonOutput = JSON.stringify({ devices: { 'iOS 18.6': [ { name: 'iPhone 15', udid: 'json-uuid-123', isAvailable: true, state: 'Shutdown', }, ], }, }); const mockTextOutput = `== Devices == -- iOS 18.6 -- iPhone 15 (json-uuid-123) (Shutdown) -- iOS 26.0 -- iPhone 17 Pro (text-uuid-456) (Shutdown)`; const mockExecutor = async (command: string[]) => { if (command.includes('--json')) { return { success: true, output: mockJsonOutput, error: undefined, process: { pid: 12345 } }; } return { success: true, output: mockTextOutput, error: undefined, process: { pid: 12345 } }; }; const result = await list_simsLogic({ enabled: true }, mockExecutor); // Should contain both iOS 18.6 from JSON and iOS 26.0 from text expect(result.content[0].text).toContain('iOS 18.6:'); expect(result.content[0].text).toContain('iPhone 15 (json-uuid-123)'); expect(result.content[0].text).toContain('iOS 26.0:'); expect(result.content[0].text).toContain('iPhone 17 Pro (text-uuid-456)'); });
📜 Review details
Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro
📒 Files selected for processing (2)
src/mcp/tools/simulator/__tests__/list_sims.test.ts(5 hunks)src/mcp/tools/simulator/list_sims.ts(2 hunks)
🧰 Additional context used
📓 Path-based instructions (2)
**/*.{test,spec}.ts
📄 CodeRabbit inference engine (AGENTS.md)
**/*.{test,spec}.ts: Do not use Vitest mocking APIs in tests (no vi.mock, vi.fn, vi.spyOn, etc.)
In tests, import the core ...Logic function from the tool and use createMockExecutor/createMockFileSystemExecutor to simulate outcomes
Files:
src/mcp/tools/simulator/__tests__/list_sims.test.ts
**/*.ts
📄 CodeRabbit inference engine (AGENTS.md)
**/*.ts: Use .ts extensions for all internal relative imports (e.g., import { tool } from './tool.ts')
Use .ts extensions in re-exports for internal files (e.g., export { default } from '../shared/tool.ts')
Use .js extension only for external package entry points (e.g., import ... from '@scope/pkg/file.js')
Never import internal project files using .js extensions
Files:
src/mcp/tools/simulator/__tests__/list_sims.test.tssrc/mcp/tools/simulator/list_sims.ts
🧠 Learnings (1)
📚 Learning: 2025-09-21T19:27:22.795Z
Learnt from: CR
PR: cameroncooke/XcodeBuildMCP#0
File: AGENTS.md:0-0
Timestamp: 2025-09-21T19:27:22.795Z
Learning: Applies to **/*.{test,spec}.ts : In tests, import the core ...Logic function from the tool and use createMockExecutor/createMockFileSystemExecutor to simulate outcomes
Applied to files:
src/mcp/tools/simulator/__tests__/list_sims.test.ts
🧬 Code graph analysis (2)
src/mcp/tools/simulator/__tests__/list_sims.test.ts (1)
src/mcp/tools/simulator/list_sims.ts (1)
list_simsLogic(102-214)
src/mcp/tools/simulator/list_sims.ts (1)
src/utils/logger.ts (1)
log(142-162)
🔇 Additional comments (11)
src/mcp/tools/simulator/list_sims.ts (6)
21-21: LGTM: Optional runtime field.The optional
runtimefield appropriately supports text-parsed devices while maintaining backward compatibility with JSON-only parsing.
109-122: LGTM: Early return on JSON command failure.Properly exits early when the JSON command fails, avoiding unnecessary text parsing in true error scenarios.
124-133: LGTM: Graceful JSON parse failure handling.The try-catch correctly handles invalid JSON from simctl (the core bug this PR addresses) and falls back to text parsing with appropriate logging.
135-167: LGTM: Hybrid merge strategy correctly implemented.The merge logic properly:
- Collects UUIDs from available JSON devices to avoid duplicates
- Adds text-parsed devices missing from JSON (the Apple bug workaround)
- Logs when devices are added from text parsing for observability
- Handles missing runtime with sensible fallback
This directly addresses the iOS 26.0 duplicate runtime ID bug described in the PR objectives.
169-193: LGTM: Enhanced output formatting.The output formatting improvements are well-implemented:
- Groups devices by runtime for clarity
- Shows boot state with
[Booted]indicator- Includes helpful "Next Steps" section for user guidance
- Filters to only available devices
202-213: LGTM: Proper exception handling.The catch block correctly handles both Error objects and other thrown values, logs appropriately, and returns a well-formatted error response.
src/mcp/tools/simulator/__tests__/list_sims.test.ts (5)
1-48: LGTM: Test setup and schema validation.The test structure correctly follows the coding guidelines by importing the core
list_simsLogicfunction and using mock executors. Schema validation tests remain comprehensive and appropriate.Based on learnings.
133-187: LGTM: Booted state test.Correctly verifies that booted simulators display the
[Booted]indicator in the output.
189-207: LGTM: Command failure test.Properly verifies error handling when the simulator command fails, using the mock executor helper as specified in coding guidelines.
Based on learnings.
209-254: LGTM: JSON parse failure fallback test.This test correctly exercises the core workaround for the Apple simctl bug: when JSON parsing fails, the implementation falls back to text parsing and produces valid output. This is essential coverage for the bug described in the PR objectives.
256-284: LGTM: Exception handling tests.Both tests appropriately verify that exceptions (Error objects and string errors) are handled correctly and produce proper error messages.
- Replace fixed-width indentation (\s{4}) with flexible indentation (\s+)
- Capture unavailable suffix in regex group instead of line.includes()
- Add test for merge logic with text-only devices (core bug fix scenario)
- Update resource test to handle new fallback behavior
Addresses CodeRabbit review feedback on PR getsentry#118
|
Addressed CodeRabbit review feedback:
All quality checks pass (typecheck, lint, 1105 tests). |
|
@cursor review |
commit: |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
| -- iOS 17.0 -- | ||
| iPhone 15 (test-uuid-123) (Booted)`; | ||
|
|
||
| const mockExecutor = async (command: string[]) => { |
There was a problem hiding this comment.
Bug: Mock Executor Signature Mismatch
The mock executors in list_simsLogic and simulatorsResourceLogic tests have a signature mismatch. They're defined to only accept command: string[], but the functions under test call the executor with additional parameters like logPrefix, useShell, and env. This divergence from the CommandExecutor interface could lead to unexpected test behavior or runtime errors.
Additional Locations (2)
|
@andreyz Thanks for this. My gut feeling is that this isn't a bug at all, but rather a data issue. JSON should have unique keys; hence, the last value associated with a pre-existing key will prevail. I'm trying to understand the use-case here, but having two simulators with the same identifier seems like an edge case based on the use of multiple beta builds. My preference is to avoid adding additonal complexity here. Could you give me an example of why it's important for XcodeBuildMCP to be able to work with multiple beta builds of the same iOS release and why would you need to test against older beta builds? |
|
@cameroncooke the issue that I ran into was that I had three iOS 26 runtimes installed (two betas and one release) and one 18.6. Given that list_sims would see only the iOS 18.6 there was no way for me to run a build on any of the iOS 26 sims. |
|
I know it's an edge case issue caused by JSON output of xcrun simctl tool, generating incomplete data, not something that's at fault with xcodemcp. I defer to you the decision to integrate or ignore. The patch would make your tool more resilient to faulty output. But the intense beta period won't repeat until next year, so probably no one else will hit that issue. |
|
Forgot to answer: testing regressions between beta builds is valuable when keeping track of issues resolving and popping back during the pre-release period. |
cameroncooke
left a comment
cameroncooke
left a comment
There was a problem hiding this comment.
Ok, I'm going to approve as the changes are fairly small. Thanks for the contribution @andreyz
|
Thank you for creating and maintaining this tool! |
2 participants
Summary
list_simstool to show iOS 26.0 simulators by working around Apple simctl JSON bugBackground/Details
Root Cause Analysis
Apple's
simctl list devices --jsonhas a bug when multiple runtime versions share the same identifier. In the case of iOS 26.0, there are two beta builds (23A339 and 23A343) that both use the identifiercom.apple.CoreSimulator.SimRuntime.iOS-26-0.When
simctlgenerates JSON output with duplicate object keys, the second key's value overwrites the first, leaving one runtime's device array empty. The text output correctly shows both sections with all devices, but the JSON output loses devices from one of the duplicate runtimes.This is an Apple bug in
simctl, not an issue with XcodeBuildMCP's implementation.Solution
Implemented a robust hybrid parsing strategy:
The text parser uses regex to extract device information from the plain text output format, handling all simulator states (Booted, Shutdown, etc.) and filtering out unavailable devices.
Testing
Test Coverage
Manual Testing
Verified on system with iOS 26.0 beta simulators that were previously hidden:
Validation Method
Notes
Summary by CodeRabbit
New Features
Bug Fixes
Tests
Note
Implement JSON-first with text fallback and merge in
list_sims, adding text parsing and updating tests to cover fallback, merging, and dual-command execution.list_sims.ts: callsimctl ... --jsonfirst, then text command, parse text viaparseTextOutput, and merge devices byudid(preferring JSON).runtimetoSimulatorDevice; filter unavailable devices; preserve booted marker; keep “Next Steps” output.List Simulators (JSON)andList Simulators (Text); improved error handling when JSON parse fails.list_simstests to expect two executor calls (JSON + text), verify booted display, JSON-parse fallback, and merging devices missing from JSON.Written by Cursor Bugbot for commit 1f8edcf. This will update automatically on new commits. Configure here.