Skip to content

Fix shared snapshot_ui warning state #185

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
cameroncooke merged 1 commit into from
Jan 26, 2026
Merged

Fix shared snapshot_ui warning state #185

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@
- Update UI automation guard guidance to point at `debug_continue` when paused.
- Fix tool loading bugs in static tool registration.
- Fix xcodemake command argument corruption when project directory path appears as substring in non-path arguments.
- Fix snapshot_ui warning state being isolated per UI automation tool, causing false warnings.

## [1.16.0] - 2025-12-30
- Remove dynamic tool discovery (`discover_tools`) and `XCODEBUILDMCP_DYNAMIC_TOOLS`. Use `XCODEBUILDMCP_ENABLED_WORKFLOWS` to limit startup tool registration.
Expand Down
27 changes: 2 additions & 25 deletions src/mcp/tools/ui-automation/long_press.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ import {
createSessionAwareTool,
getSessionAwareToolSchemaShape,
} from '../../../utils/typed-tool-factory.ts';
import { getSnapshotUiWarning } from './shared/snapshot-ui-state.ts';

// Define schema as ZodObject
const longPressSchema = z.object({
Expand Down Expand Up @@ -99,7 +100,7 @@ export async function long_pressLogic(
await executeAxeCommand(commandArgs, simulatorId, 'touch', executor, axeHelpers);
log('info', `${LOG_PREFIX}/${toolName}: Success for ${simulatorId}`);

const coordinateWarning = getCoordinateWarning(simulatorId);
const coordinateWarning = getSnapshotUiWarning(simulatorId);
const message = `Long press at (${x}, ${y}) for ${duration}ms simulated successfully.`;
const warnings = [guard.warningText, coordinateWarning].filter(Boolean).join('\n\n');

Expand Down Expand Up @@ -153,30 +154,6 @@ export default {
}),
};

// Session tracking for snapshot_ui warnings
interface DescribeUISession {
timestamp: number;
simulatorId: string;
}

const snapshotUiTimestamps = new Map<string, DescribeUISession>();
const SNAPSHOT_UI_WARNING_TIMEOUT = 60000; // 60 seconds

function getCoordinateWarning(simulatorId: string): string | null {
const session = snapshotUiTimestamps.get(simulatorId);
if (!session) {
return 'Warning: snapshot_ui has not been called yet. Consider using snapshot_ui for precise coordinates instead of guessing from screenshots.';
}

const timeSinceDescribe = Date.now() - session.timestamp;
if (timeSinceDescribe > SNAPSHOT_UI_WARNING_TIMEOUT) {
const secondsAgo = Math.round(timeSinceDescribe / 1000);
return `Warning: snapshot_ui was last called ${secondsAgo} seconds ago. Consider refreshing UI coordinates with snapshot_ui instead of using potentially stale coordinates.`;
}

return null;
}

// Helper function for executing axe commands (inlined from src/tools/axe/index.ts)
async function executeAxeCommand(
commandArgs: string[],
Expand Down
22 changes: 22 additions & 0 deletions src/mcp/tools/ui-automation/shared/snapshot-ui-state.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
const SNAPSHOT_UI_WARNING_TIMEOUT_MS = 60000; // 60 seconds

const snapshotUiTimestamps = new Map<string, number>();

export function recordSnapshotUiCall(simulatorId: string): void {
snapshotUiTimestamps.set(simulatorId, Date.now());
}

export function getSnapshotUiWarning(simulatorId: string): string | null {
const timestamp = snapshotUiTimestamps.get(simulatorId);
if (!timestamp) {
return 'Warning: snapshot_ui has not been called yet. Consider using snapshot_ui for precise coordinates instead of guessing from screenshots.';
}

const timeSinceDescribe = Date.now() - timestamp;
if (timeSinceDescribe > SNAPSHOT_UI_WARNING_TIMEOUT_MS) {
const secondsAgo = Math.round(timeSinceDescribe / 1000);
return `Warning: snapshot_ui was last called ${secondsAgo} seconds ago. Consider refreshing UI coordinates with snapshot_ui instead of using potentially stale coordinates.`;
}

return null;
}
Copy link
Contributor

@cursor cursor bot Jan 26, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Shared directory naming violates project conventions

Low Severity · Bugbot Rules

The new shared/ directory doesn't follow the project's documented naming convention. According to docs/dev/CODE_QUALITY.md, common code should be shared via -shared directories (e.g., ui-automation-shared). The review rules also specify that docs/TOOLS.md excludes *-shared directories - since shared doesn't match this pattern, it may not be properly excluded by documentation tooling. The directory should be renamed to follow the *-shared convention.

Fix in Cursor Fix in Web

13 changes: 2 additions & 11 deletions src/mcp/tools/ui-automation/snapshot_ui.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ import {
createSessionAwareTool,
getSessionAwareToolSchemaShape,
} from '../../../utils/typed-tool-factory.ts';
import { recordSnapshotUiCall } from './shared/snapshot-ui-state.ts';

// Define schema as ZodObject
const snapshotUiSchema = z.object({
Expand All @@ -34,16 +35,6 @@ export interface AxeHelpers {

const LOG_PREFIX = '[AXe]';

// Session tracking for snapshot_ui warnings (shared across UI tools)
const snapshotUiTimestamps = new Map<string, { timestamp: number; simulatorId: string }>();

function recordSnapshotUICall(simulatorId: string): void {
snapshotUiTimestamps.set(simulatorId, {
timestamp: Date.now(),
simulatorId,
});
}

/**
* Core business logic for snapshot_ui functionality
*/
Expand Down Expand Up @@ -80,7 +71,7 @@ export async function snapshot_uiLogic(
);

// Record the snapshot_ui call for warning system
recordSnapshotUICall(simulatorId);
recordSnapshotUiCall(simulatorId);

log('info', `${LOG_PREFIX}/${toolName}: Success for ${simulatorId}`);
const response: ToolResponse = {
Expand Down
27 changes: 2 additions & 25 deletions src/mcp/tools/ui-automation/swipe.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,7 @@ import {
createSessionAwareTool,
getSessionAwareToolSchemaShape,
} from '../../../utils/typed-tool-factory.ts';
import { getSnapshotUiWarning } from './shared/snapshot-ui-state.ts';

// Define schema as ZodObject
const swipeSchema = z.object({
Expand Down Expand Up @@ -119,7 +120,7 @@ export async function swipeLogic(
await executeAxeCommand(commandArgs, simulatorId, 'swipe', executor, axeHelpers);
log('info', `${LOG_PREFIX}/${toolName}: Success for ${simulatorId}`);

const coordinateWarning = getCoordinateWarning(simulatorId);
const coordinateWarning = getSnapshotUiWarning(simulatorId);
const message = `Swipe from (${x1}, ${y1}) to (${x2}, ${y2})${optionsText} simulated successfully.`;
const warnings = [guard.warningText, coordinateWarning].filter(Boolean).join('\n\n');

Expand Down Expand Up @@ -170,30 +171,6 @@ export default {
}),
};

// Session tracking for snapshot_ui warnings
interface DescribeUISession {
timestamp: number;
simulatorId: string;
}

const snapshotUiTimestamps = new Map<string, DescribeUISession>();
const SNAPSHOT_UI_WARNING_TIMEOUT = 60000; // 60 seconds

function getCoordinateWarning(simulatorId: string): string | null {
const session = snapshotUiTimestamps.get(simulatorId);
if (!session) {
return 'Warning: snapshot_ui has not been called yet. Consider using snapshot_ui for precise coordinates instead of guessing from screenshots.';
}

const timeSinceDescribe = Date.now() - session.timestamp;
if (timeSinceDescribe > SNAPSHOT_UI_WARNING_TIMEOUT) {
const secondsAgo = Math.round(timeSinceDescribe / 1000);
return `Warning: snapshot_ui was last called ${secondsAgo} seconds ago. Consider refreshing UI coordinates with snapshot_ui instead of using potentially stale coordinates.`;
}

return null;
}

// Helper function for executing axe commands (inlined from src/tools/axe/index.ts)
async function executeAxeCommand(
commandArgs: string[],
Expand Down
22 changes: 2 additions & 20 deletions src/mcp/tools/ui-automation/tap.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,6 +17,7 @@ import {
createSessionAwareTool,
getSessionAwareToolSchemaShape,
} from '../../../utils/typed-tool-factory.ts';
import { getSnapshotUiWarning } from './shared/snapshot-ui-state.ts';

export interface AxeHelpers {
getAxePath: () => string | null;
Expand Down Expand Up @@ -90,25 +91,6 @@ const publicSchemaObject = z.strictObject(baseTapSchema.omit({ simulatorId: true

const LOG_PREFIX = '[AXe]';

// Session tracking for snapshot_ui warnings (shared across UI tools)
const snapshotUiTimestamps = new Map<string, { timestamp: number }>();
const SNAPSHOT_UI_WARNING_TIMEOUT = 60000; // 60 seconds

function getCoordinateWarning(simulatorId: string): string | null {
const session = snapshotUiTimestamps.get(simulatorId);
if (!session) {
return 'Warning: snapshot_ui has not been called yet. Consider using snapshot_ui for precise coordinates instead of guessing from screenshots.';
}

const timeSinceDescribe = Date.now() - session.timestamp;
if (timeSinceDescribe > SNAPSHOT_UI_WARNING_TIMEOUT) {
const secondsAgo = Math.round(timeSinceDescribe / 1000);
return `Warning: snapshot_ui was last called ${secondsAgo} seconds ago. Consider refreshing UI coordinates with snapshot_ui instead of using potentially stale coordinates.`;
}

return null;
}

export async function tapLogic(
params: TapParams,
executor: CommandExecutor,
Expand Down Expand Up @@ -167,7 +149,7 @@ export async function tapLogic(
await executeAxeCommand(commandArgs, simulatorId, 'tap', executor, axeHelpers);
log('info', `${LOG_PREFIX}/${toolName}: Success for ${simulatorId}`);

const coordinateWarning = usesCoordinates ? getCoordinateWarning(simulatorId) : null;
const coordinateWarning = usesCoordinates ? getSnapshotUiWarning(simulatorId) : null;
const message = `${actionDescription} simulated successfully.`;
const warnings = [guard.warningText, coordinateWarning].filter(Boolean).join('\n\n');

Expand Down
27 changes: 2 additions & 25 deletions src/mcp/tools/ui-automation/touch.ts
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ import {
createSessionAwareTool,
getSessionAwareToolSchemaShape,
} from '../../../utils/typed-tool-factory.ts';
import { getSnapshotUiWarning } from './shared/snapshot-ui-state.ts';

// Define schema as ZodObject
const touchSchema = z.object({
Expand Down Expand Up @@ -95,7 +96,7 @@ export async function touchLogic(
await executeAxeCommand(commandArgs, simulatorId, 'touch', executor, axeHelpers);
log('info', `${LOG_PREFIX}/${toolName}: Success for ${simulatorId}`);

const coordinateWarning = getCoordinateWarning(simulatorId);
const coordinateWarning = getSnapshotUiWarning(simulatorId);
const message = `Touch event (${actionText}) at (${x}, ${y}) executed successfully.`;
const warnings = [guard.warningText, coordinateWarning].filter(Boolean).join('\n\n');

Expand Down Expand Up @@ -147,30 +148,6 @@ export default {
}),
};

// Session tracking for snapshot_ui warnings
interface DescribeUISession {
timestamp: number;
simulatorId: string;
}

const snapshotUiTimestamps = new Map<string, DescribeUISession>();
const SNAPSHOT_UI_WARNING_TIMEOUT = 60000; // 60 seconds

function getCoordinateWarning(simulatorId: string): string | null {
const session = snapshotUiTimestamps.get(simulatorId);
if (!session) {
return 'Warning: snapshot_ui has not been called yet. Consider using snapshot_ui for precise coordinates instead of guessing from screenshots.';
}

const timeSinceDescribe = Date.now() - session.timestamp;
if (timeSinceDescribe > SNAPSHOT_UI_WARNING_TIMEOUT) {
const secondsAgo = Math.round(timeSinceDescribe / 1000);
return `Warning: snapshot_ui was last called ${secondsAgo} seconds ago. Consider refreshing UI coordinates with snapshot_ui instead of using potentially stale coordinates.`;
}

return null;
}

// Helper function for executing axe commands (inlined from src/tools/axe/index.ts)
async function executeAxeCommand(
commandArgs: string[],
Expand Down
Loading