-
-
Notifications
You must be signed in to change notification settings - Fork 195
Fix bug where AXe isn't bundled for Smithery installs, also improve AXe detection logic #164
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
6 commits
Select commit
Hold shift + click to select a range
File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Large diffs are not rendered by default.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,68 @@ | ||
| # Investigation: UI automation tools unavailable with Smithery install (issue #163) | ||
|
|
||
| ## Summary | ||
| Smithery installs ship only the compiled entrypoint, while the server hard-requires a bundled `bundled/axe` path derived from `process.argv[1]`. This makes UI automation (and simulator video capture) fail even when system `axe` exists on PATH, and Doctor can report contradictory statuses. | ||
|
|
||
| ## Symptoms | ||
| - UI automation tools (`describe_ui`, `tap`, `swipe`, etc.) fail with "Bundled axe tool not found. UI automation features are not available." | ||
| - `doctor` reports system axe present, but UI automation unavailable due to missing bundled binary. | ||
| - Smithery cache lacks `bundled/axe` directory; only `index.cjs`, `manifest.json`, `.metadata.json` present. | ||
|
|
||
| ## Investigation Log | ||
|
|
||
| ### 2026-01-06 - Initial Assessment | ||
| **Hypothesis:** Smithery packaging omits bundled binaries and server does not fallback to system axe. | ||
| **Findings:** Issue report indicates bundled path is computed relative to `process.argv[1]` and Smithery cache lacks `bundled/`. | ||
| **Evidence:** GitHub issue #163 body (Smithery cache contents; bundled path logic). | ||
| **Conclusion:** Needs code and packaging investigation. | ||
|
|
||
| ### 2026-01-06 - AXe path resolution and bundled-only assumption | ||
| **Hypothesis:** AXe resolution is bundled-only, so missing `bundled/axe` disables tools regardless of PATH. | ||
| **Findings:** `getAxePath()` computes `bundledAxePath` from `process.argv[1]` and returns it only if it exists; otherwise `null`. No PATH or env override. | ||
| **Evidence:** `src/utils/axe-helpers.ts:15-36` | ||
| **Conclusion:** Confirmed. Smithery layout lacking `bundled/` will always return null. | ||
|
|
||
| ### 2026-01-06 - UI automation and video capture gating | ||
| **Hypothesis:** UI tools and video capture preflight fail when `getAxePath()` returns null. | ||
| **Findings:** UI tools call `getAxePath()` and throw `DependencyError` if absent; `record_sim_video` preflights `areAxeToolsAvailable()` and `isAxeAtLeastVersion()`; `startSimulatorVideoCapture` returns error if `getAxePath()` is null. | ||
| **Evidence:** `src/mcp/tools/ui-testing/describe_ui.ts:150-164`, `src/mcp/tools/simulator/record_sim_video.ts:80-88`, `src/utils/video_capture.ts:92-99` | ||
| **Conclusion:** Confirmed. Missing bundled binary blocks all UI automation and simulator video capture. | ||
|
|
||
| ### 2026-01-06 - Doctor output inconsistency | ||
| **Hypothesis:** Doctor uses different checks for dependency presence vs feature availability. | ||
| **Findings:** Doctor uses `areAxeToolsAvailable()` (bundled-only) for UI automation feature status, while dependency check can succeed via `which axe` when bundled is missing. | ||
| **Evidence:** `src/mcp/tools/doctor/doctor.ts:49-68`, `src/mcp/tools/doctor/lib/doctor.deps.ts:100-132` | ||
| **Conclusion:** Confirmed. Doctor can report `axe` dependency present but UI automation unsupported. | ||
|
|
||
| ### 2026-01-06 - Packaging/Smithery artifact mismatch | ||
| **Hypothesis:** NPM releases include `bundled/`, Smithery builds do not. | ||
| **Findings:** `bundle:axe` creates `bundled/` and npm packaging includes it, but Smithery config has no asset inclusion hints. Release workflow bundles AXe before publish. | ||
| **Evidence:** `package.json:21-44`, `.github/workflows/release.yml:48-55`, `smithery.yaml:1-3`, `smithery.config.js:1-6` | ||
| **Conclusion:** Confirmed. Smithery build output likely omits bundled artifacts unless explicitly configured. | ||
|
|
||
| ### 2026-01-06 - Smithery local server deployment flow | ||
| **Hypothesis:** Smithery deploys local servers from GitHub pushes and expects build-time packaging to include assets. | ||
| **Findings:** README install flow uses Smithery CLI; `smithery.yaml` targets `local`. `bundled/` is gitignored, so it must be produced during Smithery’s deployment build. Current `npm run build` does not run `bundle:axe`. | ||
| **Evidence:** `README.md:11-74`, `smithery.yaml:1-3`, `.github/workflows/release.yml:48-62`, `.gitignore:66-68` | ||
| **Conclusion:** Confirmed. Smithery deploy must run `bundle:axe` and explicitly include `bundled/` in the produced bundle. | ||
|
|
||
| ### 2026-01-06 - Smithery config constraints and bundling workaround | ||
| **Hypothesis:** Adding esbuild plugins in `smithery.config.js` overrides Smithery’s bootstrap plugin. | ||
| **Findings:** Smithery CLI merges config via spread and replaces `plugins`, causing `virtual:bootstrap` resolution to fail when custom plugins are supplied. Side-effect bundling in `smithery.config.js` avoids plugin override and can copy `bundled/` into `.smithery/`. | ||
| **Evidence:** `node_modules/@smithery/cli/dist/index.js:~2716600-2717500`, `smithery.config.js:1-47` | ||
| **Conclusion:** Confirmed. Bundling must run outside esbuild plugins; Linux builders must skip binary verification. | ||
|
|
||
| ## Root Cause | ||
| Two coupled assumptions break Smithery installs: | ||
| 1) `getAxePath()` is bundled-only and derives the path from `process.argv[1]`, which points into Smithery’s cache (missing `bundled/axe`), so it always returns null. | ||
| 2) Smithery packaging does not include the `bundled/` directory, so the bundled-only resolver can never succeed under Smithery even if AXe is installed system-wide. | ||
|
|
||
| ## Recommendations | ||
| 1. Add a robust AXe resolver: allow explicit env override and PATH fallback; keep bundled as preferred but not exclusive. | ||
| 2. Distinguish bundled vs system AXe in UI tools and video capture; only apply bundled-specific env when the bundled binary is used. | ||
| 3. Align Doctor output: show both bundled availability and PATH availability, and use that in the UI automation supported status. | ||
| 4. Update Smithery build to run `bundle:axe` and copy `bundled/` into the Smithery bundle output; skip binary verification on non-mac builders to avoid build failures. | ||
|
|
||
| ## Preventive Measures | ||
| - Add tests for AXe resolution precedence (bundled, env override, PATH) and for Doctor output consistency. | ||
| - Document Smithery-specific install requirements and verify `bundled/` presence in Smithery artifacts during CI. | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,35 @@ | ||
| #!/usr/bin/env bash | ||
|
|
||
| set -euo pipefail | ||
|
|
||
| PROJECT_ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)" | ||
| BUNDLE_DIR="$PROJECT_ROOT/.smithery/bundled" | ||
| AXE_BIN="$BUNDLE_DIR/axe" | ||
| FRAMEWORK_DIR="$BUNDLE_DIR/Frameworks" | ||
|
|
||
| if [ ! -f "$AXE_BIN" ]; then | ||
| echo "❌ Missing AXe binary at $AXE_BIN" | ||
| if [ -d "$PROJECT_ROOT/.smithery" ]; then | ||
| echo "🔍 .smithery contents:" | ||
| ls -la "$PROJECT_ROOT/.smithery" | ||
| fi | ||
| exit 1 | ||
| fi | ||
|
|
||
| if [ ! -d "$FRAMEWORK_DIR" ]; then | ||
| echo "❌ Missing Frameworks directory at $FRAMEWORK_DIR" | ||
| if [ -d "$BUNDLE_DIR" ]; then | ||
| echo "🔍 bundled contents:" | ||
| ls -la "$BUNDLE_DIR" | ||
| fi | ||
| exit 1 | ||
| fi | ||
|
|
||
| FRAMEWORK_COUNT="$(find "$FRAMEWORK_DIR" -maxdepth 2 -type d -name "*.framework" | wc -l | tr -d ' ')" | ||
| if [ "$FRAMEWORK_COUNT" -eq 0 ]; then | ||
| echo "❌ No frameworks found in $FRAMEWORK_DIR" | ||
| find "$FRAMEWORK_DIR" -maxdepth 2 -type d | head -n 50 | ||
| exit 1 | ||
| fi | ||
|
|
||
| echo "✅ Smithery bundle includes AXe binary and $FRAMEWORK_COUNT frameworks" |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Minor grammar issue: "fallback" should be "fall back".
The word "fallback" is a noun, but here it's used as a verb. Consider changing to "does not fall back to system axe".
🔎 Suggested fix
📝 Committable suggestion
🧰 Tools
🪛 LanguageTool
[grammar] ~14-~14: The word “fallback” is a noun. The verb is spelled with a space.
Context: ...ts bundled binaries and server does not fallback to system axe. Findings: Issue repo...
(NOUN_VERB_CONFUSION)
🤖 Prompt for AI Agents