[Bug]: MCP Sampling Prompt Issue in discover_tools

[codeman9]

Bug Description

Issue Summary

The discover_tools MCP sampling function incorrectly selects workflows for listing tasks, causing list_sims to not be available when users request to list iOS simulators.

Root Cause

The sampling prompt in src/mcp/tools/discovery/discover_tools.ts prioritizes project type and platform over task type, leading to incorrect workflow selection:

• User request: "List my iOS simulators"
• Current behavior: Selects simulator-environment workflow (only has appearance/location tools)
• Expected behavior: Should select simulator-project or simulator-workspace workflow (includes list_sims)

Evidence

Testing with Amazon Q CLI's MCP sampling implementation confirmed:

1. ✅ MCP sampling mechanism works correctly
2. ❌ Original prompt: Enables only environment tools (set_sim_appearance, sim_statusbar, etc.)
3. ✅ Improved prompt: Correctly enables listing tools (list_sims, build_run_sim_*, etc.)

Solution

Update the sampling prompt to prioritize task type over project type:

typescript
// Current prompt focuses on project type first
Project Type Selection Guide:

• If working with .xcworkspace files...

// Should focus on task type first
For LISTING/DISCOVERY tasks (list simulators, show devices):

• "list simulators" → Choose simulator-project
• "list devices" → Choose device-project

Impact

• Users cannot list simulators through MCP tools
• Forces fallback to execute_bash instead of native MCP integration
• Affects discoverability of core iOS development functionality

Test Case

Failing prompt: "List my iOS simulators"
Working prompt: "I need to build and run my iOS app on a simulator. First, I need to list all available iOS simulators for my .xcodeproj project."

The working prompt succeeds because it frames listing as part of a development workflow rather than a standalone task.

Debug Output

N/A

Editor/Client

https://github.com/aws/amazon-q-developer-cli

MCP Server Version

Release v1.11.0

LLM

Claude 4.0 Sonnet

MCP Configuration

"XcodeBuildMCP": {
      "command": "/opt/homebrew/opt/node@22/bin/npx",
      "args": ["-y", "xcodebuildmcp@latest"],
      "env": {
        "SENTRY_DISABLED": "true",
        "XCODEBUILDMCP_DYNAMIC_TOOLS": "true"
      },
      "timeout": 600000,
      "disabled": false,
      "sampling_enabled": true
    },

Steps to Reproduce

Start the chat cli (I'm on a branch that is adding MCP sampling to the client) and run the prompt: List my iOS simulators. The available tools that get loaded are:

XcodeBuildMCP (MCP):
- discover_projs                             * not trusted
- discover_tools                             * trusted
- reset_simulator_location                   * not trusted
- set_sim_appearance                         * not trusted
- set_simulator_location                     * not trusted
- sim_statusbar                              * not trusted

If I use the prompt: "I need to build and run my iOS app on a simulator. First, I need to list all available iOS simulators to choose which one to use for testing my .xcodeproj project." then the following tools are loaded:

[not-ios] > XcodeBuildMCP (MCP):
- boot_sim                                   * not trusted
- build_run_sim_id_proj                      * not trusted
- build_run_sim_name_proj                    * not trusted
- build_sim_id_proj                          * not trusted
- build_sim_name_proj                        * not trusted
- clean_proj                                 * not trusted
- describe_ui                                * not trusted
- discover_projs                             * not trusted
- discover_tools                             * trusted
- get_app_bundle_id                          * not trusted
- get_sim_app_path_id_proj                   * not trusted
- get_sim_app_path_name_proj                 * not trusted
- install_app_sim                            * not trusted
- launch_app_logs_sim                        * not trusted
- launch_app_sim                             * not trusted
- list_schems_proj                           * not trusted
- list_sims                                  * not trusted
- open_sim                                   * not trusted
- screenshot                                 * not trusted
- show_build_set_proj                        * not trusted
- stop_app_sim                               * not trusted
- test_sim_id_proj                           * not trusted
- test_sim_name_proj                         * not trusted

Expected Behavior

I expect the tools from the simulator-environment workflow to be loaded and to include list_sims.

Actual Behavior

XcodeBuildMCP (MCP):
- discover_projs                             * not trusted
- discover_tools                             * trusted
- reset_simulator_location                   * not trusted
- set_sim_appearance                         * not trusted
- set_simulator_location                     * not trusted
- sim_statusbar                              * not trusted

Error Messages

[cameroncooke]

@codeman9 thanks for raising. I'm on the fence on this one, the prompt doesn't indicate whether your project uses a workspace or a project, so the LLM wouldn't know which simulator-* workflow to enable. It's picked the simulator-environment workflow, which kind of makes sense, but that workflow doesn't include list_sims.

I think it might be better to just add a workflow for generic simulator management that would then likely be picked by the LLM.

[cameroncooke]

@codeman9 this has been improved in v1.11.1. Can you try again using the latest version and let me know so I can close this issue?

[cameroncooke]

Going to close as fixed for now, feel free to re-open if you still have issues.

[codeman9]

Thanks for taking a look at this. It seems this has regressed even farther now as the original working prompt no longer works either. One of the things that I tend to do is run my llm outside of an iOS project/workspace directory, but still want to do things like mess with my simulators. It looks like the discover_tools is then trying to use discover_xcodeprojs but there aren't any of those where I have launched my llm.

Also, it seems like this model is just deciding to use the discover_tools and discover_projs and then just crafting xcodebuild commands with this latest version. I can't seem to get it to actually use the XcodeBuildMCP.

[cameroncooke]

Can you tell me what model your client is using for Sampling and also what prompt it’s providing and I’ll try and recreate but in my testing it was working perfectly. It’s worth nothing that the protocol says that clients can modify the prompt and the response before sending to the LLM and back to the MCP server so I can’t rule out Amazon Q doing something weird here. If you could send a video as well that would be amazing! Cameron.

…

On Fri, 8 Aug 2025 at 15:35, Cody Vandermyn ***@***.***> wrote: *codeman9* left a comment (getsentry/XcodeBuildMCP#87) <#87 (comment)> Thanks for taking a look at this. It seems this has regressed even farther now as the original working prompt no longer works either. One of the things that I tend to do is run my llm outside of an iOS project/workspace directory, but still want to do things like mess with my simulators. It looks like the discover_tools is then trying to use discover_xcodeprojs but there aren't any of those where I have launched my llm. Also, it seems like this model is just deciding to use the discover_tools and discover_projs and then just crafting xcodebuild commands with this latest version. I can't seem to get it to actually use the XcodeBuildMCP. — Reply to this email directly, view it on GitHub <#87 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAEZ6SNVQF6VPUN2R3S6GV33MSYS3AVCNFSM6AAAAACDMB2BFOVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZTCNRYGE3DGMRQGA> . You are receiving this because you modified the open/close state.Message ID: ***@***.***>