Open
Conversation
Three bugs prevented browser events from being captured and parsed:
1. background.js only relayed DOM_EVENT messages but the content script
sends USER_EVENT — events were silently dropped.
2. background.js handleSetMode only read message.payload?.mode but the
recorder sends flat {mode: "record"} — mode was never set to "record"
so the content script never attached record listeners.
3. The BrowserEventType enum used "browser.click" prefix format but the
content script sends raw DOM event names ("click", "keydown", etc.).
This was an artificial convention introduced during the port from
legacy OpenAdapt that was never tested end-to-end. Legacy used raw
names throughout.
Changes:
- background.js: add USER_EVENT relay, fix SET_MODE format handling
- browser_events.py: change enum values to raw DOM names matching the
content script and legacy OpenAdapt, add BrowserMouseMoveEvent
- capture.py: add _parse_element_ref() and rewrite _convert_browser_event()
to handle actual content-script message format including the recorder's
{"message": <raw>} wrapper, add browser_events() and browser_event_count
to CaptureSession
- cli.py: add --browser-events flag to record, show browser event breakdown
in info command
- tests: add 15 e2e tests covering both DB roundtrip and raw content-script
format parsing
Verified with live recording: 84/84 events captured and parsed from
Chrome extension on Hacker News.
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
- Replace bare except with debug logging in _convert_browser_event - Move lazy imports to module level (BoundingBox, ElementState, etc.) - Remove unused imports (pytest, Recording) from test file - Update test class names to reflect structure tested, not removed format - Fix stale docstring in _parse_element_ref Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
USER_EVENTbut background only relayedDOM_EVENT— events silently droppedhandleSetModeonly readmessage.payload?.modebut recorder sends flat{mode: "record"}— mode never set, listeners never attached"browser.click"prefix but content script sends raw DOM names"click"— artificial convention from port that was never tested; changed to match legacy OpenAdapt and the actual content scriptBrowserMouseMoveEventtype,CaptureSession.browser_events()API,--browser-eventsCLI flag, and 15 e2e testsTest plan
🤖 Generated with Claude Code