Skip to content

Use G1 WebRTC camera in headless setup#1553

Open
kaiknower wants to merge 1 commit intodimensionalOS:mainfrom
kaiknower:fix-g1-headless-camera
Open

Use G1 WebRTC camera in headless setup#1553
kaiknower wants to merge 1 commit intodimensionalOS:mainfrom
kaiknower:fix-g1-headless-camera

Conversation

@kaiknower
Copy link

Summary

  • publish G1 camera image and camera info directly from the WebRTC connection
  • remove the local webcam dependency from the headless G1 primitive blueprint
  • keep the existing G1 camera TF publication so perception modules still receive camera frames in the expected coordinate system

Motivation

unitree-g1 was trying to open /dev/video0 on the deployment machine via CameraModule, which breaks headless setups where the camera is actually on the G1 robot.

Validation

  • verified Python syntax with python -m py_compile on the modified files
  • manually checked that the blueprint no longer instantiates the local webcam path

@greptile-apps
Copy link
Contributor

greptile-apps bot commented Mar 14, 2026

Greptile Summary

This PR moves camera image and metadata publishing from the blueprint layer (via CameraModule + local webcam) into the G1Connection class itself, sourcing video frames directly from the robot's WebRTC stream. This fixes headless deployments where /dev/video0 doesn't exist on the deployment machine.

  • connection.py: G1Connection now publishes color_image from the WebRTC video stream and runs a 1Hz background thread for camera_info + TF transforms (camera_link and camera_optical), matching the pattern already established by G1SimConnection.
  • uintree_g1_primitive_no_nav.py: Removes _create_webcam(), _camera variable, and all related imports (Webcam, camera_module, zed, geometry types). The blueprint already had color_image and camera_info wired via LCM transports, so downstream consumers are unaffected.
  • The stop() method was also improved to guard against self.connection being None, making it more robust than the previous assert-based approach.

Confidence Score: 4/5

  • This PR is safe to merge — it applies a well-established pattern from G1SimConnection to the real hardware connection.
  • The changes are focused and well-motivated. The new camera publishing in G1Connection closely follows the existing G1SimConnection pattern. The blueprint cleanup is clean with no orphaned references. One minor style concern about a class-level mutable default, but it's non-blocking since the object is never mutated in place.
  • dimos/robot/unitree/g1/connection.py — verify the camera_info_static class-level default is intentional

Important Files Changed

Filename Overview
dimos/robot/unitree/g1/connection.py Adds WebRTC camera image publishing and a 1Hz camera_info/TF publishing loop to G1Connection. The implementation mirrors the existing G1SimConnection pattern and the old CameraModule behavior. One minor style note about a class-level mutable default.
dimos/robot/unitree/g1/blueprints/primitive/uintree_g1_primitive_no_nav.py Removes local webcam/CameraModule setup and related imports. Camera streams now come from G1Connection via LCM transports, which were already wired. Clean removal with no orphaned references.

Sequence Diagram

sequenceDiagram
    participant G1 as G1 Robot (WebRTC)
    participant Conn as G1Connection
    participant LCM as LCM Transport
    participant BP as Blueprint / Consumers

    Note over Conn: start() called
    Conn->>G1: Open WebRTC connection
    G1-->>Conn: video_stream() Observable

    loop Every video frame
        G1->>Conn: Video frame (Image)
        Conn->>Conn: color_image.publish(image)
        Conn->>LCM: Broadcast /color_image
        LCM->>BP: Image delivered
    end

    loop Every 1 second (background thread)
        Conn->>Conn: camera_info.publish(static_info.with_ts)
        Conn->>LCM: Broadcast /camera_info
        Conn->>Conn: tf.publish(camera_link, camera_optical)
        LCM->>BP: CameraInfo + TF delivered
    end

    Note over Conn: stop() called
    Conn->>Conn: _stop_event.set()
    Conn->>G1: connection.stop()
    Conn->>Conn: Join camera_info thread
Loading

Last reviewed commit: d620ace

ip: str | None
connection_type: str | None = None
_global_config: GlobalConfig
camera_info_static: CameraInfo = _camera_info_static()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Class-level mutable default is shared across instances

_camera_info_static() is called once at class-definition time, so every G1Connection instance shares the same CameraInfo object. This is safe today because with_ts() returns a new copy and the static object is never mutated, but it's a subtle footgun if the object is ever modified in-place later. For comparison, G1SimConnection reads camera info at runtime from self.connection.camera_info_static.

Consider initializing this in __init__ instead:

Suggested change
camera_info_static: CameraInfo = _camera_info_static()
camera_info_static: CameraInfo

And then in __init__, add self.camera_info_static = _camera_info_static().

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant