Add On Screen and Physical Controller Support #307

Mondyro · 2025-11-28T08:23:04Z

This commit introduces a complete physical controller configuration system and in-game control editor to provide users with flexible input options.

New Features

Physical Controller Configuration

Configure Xbox, PlayStation, and generic gamepads with custom button mappings
Access controller settings via in-game menu → "Edit Physical Controller"
Map controller buttons/axes to keyboard keys, mouse buttons, or gamepad inputs
Auto-detection of connected controllers with real-time device scanning when opening the menu

In-Game Control Editor

Edit on-screen controls without leaving your game
Access via in-game menu → "Edit Controls"
Drag elements to reposition, tap to select and edit
Edit button bindings, icons, size, and appearance
Add new control elements or delete unwanted ones
Reset controls to default layout anytime both on screen or physical.
Changes save to game-specific profile

Auto-Hide Controls

New container setting: "Auto-hide on-screen controls when physical controller is connected"
New container setting: "Start With On-Screen Controls Hidden (Overrides the above setting)"
Found in Container Config → Controls tab
When enabled, on-screen controls hide automatically when controller detected
Controls reappear when controller disconnected

Per-Game Control Profiles

Each container/game will have its own custom control profile
Profiles automatically created when using "Edit Controls" for first time
Named as "[Game Name] - Controls" for easy identification

How to Use

Configure Physical Controller:
- Launch your game
- Open in-game menu (≡)
- Select "Edit Physical Controller"
- Map buttons using the dropdown menus
- Test your configuration in-game
Edit On-Screen Controls:
- Launch your game
- Open in-game menu (≡)
- Select "Edit Controls"
- Drag elements to move them
- Tap an element to select it, then tap "Edit" to modify bindings/appearance
- Use "Add" to create new elements
- Use "Edit" to change the bindings or type of Element
- Use "Delete" to remove selected element
- Tap "✓" to save or "✗" to cancel changes

Migration Notes

Existing containers will continue to work without changes
Old keyboard/mouse emulation settings have been removed and replaced with the new flexible binding system

Technical Changes

Added PhysicalControllerHandler for independent controller input processing
Implemented ControllerBindingDialog for controller configuration UI
Added ElementEditorDialog for editing individual control elements
Improved profile loading to support per-container custom profiles
Enhanced controller detection using ControllerManager device scanning
Removed old keyboard/mouse emulation system in favor of flexible binding system
Added edit mode with visual toolbar and element manipulation
Implemented position snapshotting for cancel functionality

…ting This commit introduces a complete physical controller configuration system and in-game control editor to provide users with flexible input options. ## New Features ### Physical Controller Configuration - Configure Xbox, PlayStation, and generic gamepads with custom button mappings - Access controller settings via in-game menu → "Edit Physical Controller" - Map controller buttons/axes to keyboard keys, mouse buttons, or gamepad inputs - Supports multiple controller profiles per game - Auto-detection of connected controllers with real-time device scanning ### In-Game Control Editor - Edit on-screen controls without leaving your game - Access via in-game menu → "Edit Controls" - Automatically creates game-specific control profile on first use - Drag elements to reposition, tap to select and edit - Edit button bindings, icons, size, and appearance - Add new control elements or delete unwanted ones - Reset controls to default layout anytime - Changes save to game-specific profile ### Auto-Hide Controls - New container setting: "Auto-hide on-screen controls when physical controller is connected" - Found in Container Config → Controls tab - When enabled, on-screen controls hide automatically when controller detected - Controls reappear when controller disconnected ### Per-Game Control Profiles - Each container/game can have its own custom control profile - Profiles automatically created when using "Edit Controls" for first time - Named as "[Game Name] - Controls" for easy identification - Fallback to "Profile 0" (Physical Controller Default) if no custom profile exists ## How to Use 1. **Configure Physical Controller:** - Launch your game - Open in-game menu (≡) - Select "Edit Physical Controller" - Choose a controller slot or scan for devices - Map buttons using the dropdown menus - Test your configuration in-game 2. **Edit On-Screen Controls:** - Launch your game - Open in-game menu (≡) - Select "Edit Controls" - Drag elements to move them - Tap an element to select it, then tap "Edit" to modify bindings/appearance - Use "Add" to create new elements - Use "Delete" to remove selected element - Tap "✓" to save or "✗" to cancel changes 3. **Auto-Hide Controls:** - Open Container Settings - Go to Controls tab - Enable "Auto-hide on-screen controls when physical controller is connected" - Save container settings ## Technical Changes - Added PhysicalControllerHandler for independent controller input processing - Implemented ControllerBindingDialog for controller configuration UI - Added ElementEditorDialog for editing individual control elements - Improved profile loading to support per-container custom profiles - Enhanced controller detection using ControllerManager device scanning - Removed old keyboard/mouse emulation system in favor of flexible binding system - Added edit mode with visual toolbar and element manipulation - Implemented position snapshotting for cancel functionality 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>

Key Changes: - Fixed bug where physical controller configuration dialog wouldn't open when on-screen controls were hidden - Changed dialog to load profile from container settings via InputControlsManager instead of relying on InputControlsView.profile - Added safety check to only update InputControlsView profile on save if controls are visible - Improved physical controller configuration UI with categorized two-column layout - Categories: Face Buttons, Shoulder Buttons, Menu Buttons, Thumbstick Buttons, Left/Right Sticks, D-Pad - Cleaner navigation and better use of screen space - Enhanced binding dialog with collapsible search and cross-category filtering - Search now shows match counts per category - Category filtering works in combination with search - Code quality improvements - Better null safety in XServerScreen element editing - Cleanup of InputControlsView and ControlsProfile deprecated methods - Added DisposableEffect cleanup for physical controller handler 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>

- Fix virtual control bindings not persisting when changing element types by adding setTypeWithoutReset() method - Update default bindings for sticks/dpads to use gamepad bindings instead of WASD - Add quick preset buttons for physical controller stick and dpad bindings - Make quick presets more compact for smaller screens - Add mouse movement support for physical controller sticks with analog deflection - Add 8% deadzone to mouse movement timer to save CPU cycles when idle - Support dynamic cursor speed updates when profile changes - Clean up mouse movement timer when profile is set to null 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>

coderabbitai · 2025-11-28T08:23:11Z

Important

Review skipped

Draft detected.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

✨ Finishing touches

🧪 Generate unit tests (beta)

Create PR with unit tests
Post copyable unit tests in a comment

Tip

📝 Customizable high-level summaries are now available in beta!

You can now customize how CodeRabbit generates the high-level summary in your pull requests — including its content, structure, tone, and formatting.

Provide your own instructions using the high_level_summary_instructions setting.
Format the summary however you like (bullet lists, tables, multi-section layouts, contributor stats, etc.).
Use high_level_summary_in_walkthrough to move the summary from the description to the walkthrough section.

Example instruction:

"Divide the high-level summary into five sections:

📝 Description — Summarize the main change in 50–60 words, explaining what was done.

📓 References — List relevant issues, discussions, documentation, or related PRs.

📦 Dependencies & Requirements — Mention any new/updated dependencies, environment variable changes, or configuration updates.

📊 Contributor Summary — Include a Markdown table showing contributions:
| Contributor | Lines Added | Lines Removed | Files Changed |

✔️ Additional Notes — Add any extra reviewer context.
Keep each section concise (under 200 words) and use bullet or numbered lists for clarity."

Note: This feature is currently in beta for Pro-tier users, and pricing will be announced later.

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

utkarshdalal · 2025-11-28T15:32:03Z

app/src/main/java/app/gamenative/ui/screen/xserver/XServerScreen.kt

-                }
-                // handled = ExternalController.onKeyEvent(xServer.winHandler, it.event)
+                // Use standalone handler first (works independently of view visibility)
+                handled = physicalControllerHandler?.onKeyEvent(it.event) == true


Not sure I like the idea of creating a physicalControllerHandler - why can't we extend inputControlsView?

When we hide the on screen controls it also stops the physical controller bindings from taking effect. I thought it was better to keep them seperate as that way one set of controls does not effect the other.

utkarshdalal · 2025-11-28T15:32:28Z

app/src/main/java/app/gamenative/ui/screen/xserver/XServerScreen.kt

+                    android.util.Log.d("gncontrol", "=== Profile Loading Start ===")
+                    android.util.Log.d("gncontrol", "Container: ${container.name}, ProfileID from extra: $profileId")


please remove these logs, or use Timber for logging.

Sorry these slipped though I thought I had removed them all

utkarshdalal · 2025-11-28T15:33:28Z

app/src/main/java/app/gamenative/ui/screen/xserver/XServerScreen.kt

+            icView.post {
+                android.util.Log.d("gncontrol", "Auto-show logic running - view dimensions: ${icView.width}x${icView.height}")
+                loadedProfile?.let { profile ->
+                    // Load elements if not already loaded (view has dimensions now)
+                    if (!profile.isElementsLoaded) {
+                        android.util.Log.d("gncontrol", "Loading profile elements for auto-show")
+                        profile.loadElements(icView)
+                    }
+
+                    // Only auto-show if profile has on-screen elements
+                    android.util.Log.d("gncontrol", "Profile has ${profile.elements.size} elements loaded")
+                    if (profile.elements.isNotEmpty()) {
+                        // Read control visibility settings
+                        val startWithControlsHidden = container.getExtra("startWithControlsHidden", "false").toBoolean()
+                        val hideWithController = container.getExtra("hideControlsWithController", "false").toBoolean()
+
+                        // Check for ACTUAL physically connected controllers, not just saved bindings
+                        val controllerManager = ControllerManager.getInstance()
+                        controllerManager.scanForDevices()
+                        val hasPhysicalController = controllerManager.getDetectedDevices().isNotEmpty()
+
+                        android.util.Log.d("gncontrol", "Control visibility settings: startWithControlsHidden=$startWithControlsHidden, hideWithController=$hideWithController, hasPhysicalController=$hasPhysicalController")
+


Not sure this is necessary, we already don't show the onscreen controller if a connected controller is detected.

This allows users to bind on-screen controls to work alongside standard controller input. It offers a flexible way to add keyboard-style commands directly on screen, which is especially helpful in RPGs and strategy games that often require more actions than a controller can provide. It also enhances accessibility by allowing users with disabilities to place additional on-screen buttons where they’re most useful. For example a quicksave button.

This would only come into play in specific situations.

utkarshdalal · 2025-11-28T15:37:17Z

app/src/main/java/com/winlator/container/ContainerData.kt

+                    hideControlsWithController = (savedMap["hideControlsWithController"] as? Boolean) ?: false,
+                    startWithControlsHidden = (savedMap["startWithControlsHidden"] as? Boolean) ?: false,


i don't think either of these flags or the logic are necessary, we can remove them.

if a controller is connected, we already don't show the onscreen one.

and we can simply not show the onscreen controller by default. the reason we were doing this was that previously, if a controller (including onscreen) was not detected when the game was launched, the input wouldn't work. That bug is now solved. So we can simply remove the line in XServerScreen that shows the onscreen controller. In short, just delete this else block:

My focus was mainly on RPGs and strategy games, but the idea is more about laying a foundation for the future. Eventually you could have specific profiles load automatically when a controller is plugged in, instead of always relying on the standard layout.

This feature lets users bind on-screen controls to work alongside normal controller input, giving them a flexible way to add keyboard-style actions directly on screen. It’s especially useful in RPGs and strategy titles that need more inputs than a controller can offer, and it also improves accessibility by allowing users to place extra on-screen buttons wherever they need them—for example, a dedicated quicksave button.

I am happy to remove it though

utkarshdalal · 2025-11-28T15:38:27Z

app/src/main/java/com/winlator/inputcontrols/ControlsProfile.java

None of the changes to this file seem necessary? Or is there something important I'm missing?

utkarshdalal · 2025-11-28T15:38:55Z

app/src/main/java/com/winlator/widget/InputControlsView.java

                }
-                case MotionEvent.ACTION_UP: {
+                case MotionEvent.ACTION_UP:
+                case MotionEvent.ACTION_CANCEL: {


when is ACTION_CANCEL called?

utkarshdalal · 2025-11-28T15:43:18Z

app/src/main/java/app/gamenative/ui/screen/xserver/PhysicalControllerHandler.kt

I feel like this could go into WinHandler? or no?

Looking at it, I agree that this could be moved into Winhandler. That said, keeping it separate isn’t necessarily a bad idea, as I imagine we may want to control deadzones, sensitivity, and trigger pressures per stick in the future. This could get more complex down the line.

Mondyro and others added 4 commits November 28, 2025 15:26

Fix Quick Presets and Mouse controls

5ce5dc3

utkarshdalal reviewed Nov 28, 2025

View reviewed changes

		android.util.Log.d("gncontrol", "=== Profile Loading Start ===")
		android.util.Log.d("gncontrol", "Container: ${container.name}, ProfileID from extra: $profileId")

		hideControlsWithController = (savedMap["hideControlsWithController"] as? Boolean) ?: false,
		startWithControlsHidden = (savedMap["startWithControlsHidden"] as? Boolean) ?: false,

Add On Screen and Physical Controller Support #307

Are you sure you want to change the base?

Add On Screen and Physical Controller Support #307

Conversation

Mondyro commented Nov 28, 2025

New Features

Physical Controller Configuration

In-Game Control Editor

Auto-Hide Controls

Per-Game Control Profiles

How to Use

Migration Notes

Technical Changes

Uh oh!

coderabbitai bot commented Nov 28, 2025

Review skipped

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Mondyro Nov 28, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Mondyro Nov 28, 2025 •

edited

Loading