Skip to content

Refactor rtde parsing#419

Open
urfeex wants to merge 38 commits intoUniversalRobots:masterfrom
urfeex:refactor_rtde_parsing
Open

Refactor rtde parsing#419
urfeex wants to merge 38 commits intoUniversalRobots:masterfrom
urfeex:refactor_rtde_parsing

Conversation

@urfeex
Copy link
Member

@urfeex urfeex commented Jan 21, 2026

This is a refactor of RTDE communication effectively improving two things:

  • No more heap memory allocations during communication once everything is setup
  • Reduced parsing and access times of RTDE data that reduces the time of read and send operations

There's currently some documentation and testing missing, hence the draft status.


Note

High Risk
Touches core RTDE networking/parsing/IO concurrency and changes public APIs with deprecations; regressions could impact real-time data integrity, reconnection behavior, and write semantics under load.

Overview
RTDE read path is refactored to avoid per-cycle heap allocations and reduce overhead. RTDEClient no longer relies on the Pipeline for reads; it uses URProducer directly plus a double-buffered background read thread, and introduces start(bool read_packages_in_background) with new getDataPackage(DataPackage&/unique_ptr&, timeout) and getDataPackageBlocking(...) APIs while deprecating the allocating getDataPackage(timeout) overload.

RTDE parsing and packages are updated to support reuse and stricter validation. Parser/IProducer add single-product parse/tryGet overloads, RTDEParser is split into a new rtde_parser.cpp with a prealloc-friendly parse path (vector-based parse deprecated), RTDEPackage exposes getType(), and DataPackage switches from an unordered_map to an ordered vector store, initializes empty data on construction, validates recipe keys (throwing new RTDEInvalidKeyException), adds type checking on setData, and improves toString() formatting.

RTDE write path is refactored for fewer allocations and batching. RTDEWriter replaces the moodycamel queue with double-buffered storage + condition variable, adds sendPackage() for sending a fully-prepared package, preallocates register key strings, and resets mask fields after sending; it now throws on init()/setInputRecipe() while running.

API consumers, docs, and tests are migrated to the new allocation-free patterns. Examples and documentation now show preallocating DataPackage and using the new read modes; ExampleRobotWrapper removes its custom RTDE consumer thread and forwards the background-read choice to UrDriver. Tests are expanded/updated for background vs blocking read, reconnection, invalid keys/conflicts, DataPackage parsing/toString, and parser single-product mode.

CI and tooling tweaks: workflow disables URSim port forwarding, gates coverage uploads on successful build, shortens example sleep between runs, and improves fake RTDE server thread shutdown safety.

Written by Cursor Bugbot for commit 4e23395. This will update automatically on new commits. Configure here.

@codecov
Copy link

codecov bot commented Jan 21, 2026

Codecov Report

❌ Patch coverage is 79.59184% with 80 lines in your changes missing coverage. Please review.
✅ Project coverage is 75.01%. Comparing base (08c33c3) to head (4e23395).

Files with missing lines Patch % Lines
src/rtde/rtde_client.cpp 76.72% 11 Missing and 16 partials ⚠️
src/rtde/rtde_parser.cpp 70.00% 20 Missing and 7 partials ⚠️
include/ur_client_library/comm/producer.h 45.83% 9 Missing and 4 partials ⚠️
include/ur_client_library/rtde/data_package.h 68.75% 1 Missing and 4 partials ⚠️
include/ur_client_library/exceptions.h 60.00% 4 Missing ⚠️
include/ur_client_library/primary/primary_parser.h 75.00% 0 Missing and 2 partials ⚠️
src/rtde/rtde_writer.cpp 98.01% 0 Missing and 2 partials ⚠️
Additional details and impacted files
@@            Coverage Diff             @@
##           master     #419      +/-   ##
==========================================
- Coverage   75.33%   75.01%   -0.32%     
==========================================
  Files         102      103       +1     
  Lines        5120     5328     +208     
  Branches      544      553       +9     
==========================================
+ Hits         3857     3997     +140     
- Misses        980     1065      +85     
+ Partials      283      266      -17     
Flag Coverage Δ
start_ursim 83.27% <100.00%> (-1.35%) ⬇️
ur20-latest 70.51% <75.51%> (?)
ur5-3.14.3 ?
ur5e-10.7.0 65.95% <77.26%> (+0.64%) ⬆️
ur5e-5.9.4 74.30% <79.01%> (+0.77%) ⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

@urfeex urfeex force-pushed the refactor_rtde_parsing branch 7 times, most recently from 6529df0 to 24aa5ab Compare January 26, 2026 16:21
@urfeex urfeex marked this pull request as ready for review January 26, 2026 16:21
@urfeex urfeex requested a review from urrsk January 26, 2026 16:21
@urfeex urfeex force-pushed the refactor_rtde_parsing branch 3 times, most recently from 7cb3675 to b70a72c Compare January 27, 2026 15:27
@urfeex urfeex added the enhancement New feature or request label Feb 2, 2026
if (result == nullptr || result->getType() != PackageType::RTDE_DATA_PACKAGE)
{
result = std::make_unique<DataPackage>(recipe_, protocol_version_);
URCL_LOG_WARN("The passed result pointer is empty or does not contain a DataPackage. A new DataPackage will "
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Let's us ensure that this warning only applies when there is something unexpected happening

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

With the latest changes I could not encounter this during normal operation. I've added a more descriptive output in that case so if it ever happened, we know a bit better what happened.

@urrsk
Copy link
Member

urrsk commented Feb 5, 2026

bugbot run

@cursor
Copy link

cursor bot commented Feb 5, 2026

Skipping Bugbot: Bugbot is disabled for this repository

@urrsk
Copy link
Member

urrsk commented Feb 5, 2026

bugbot run

@urfeex urfeex force-pushed the refactor_rtde_parsing branch from 3c6efbe to 798efbb Compare February 5, 2026 12:09
@urfeex urfeex requested a review from urrsk February 6, 2026 13:53
Comment on lines 107 to 122
if (result == nullptr || result->getType() != PackageType::RTDE_DATA_PACKAGE)
{
if (result == nullptr)
{
URCL_LOG_WARN("The passed result pointer is empty. A new DataPackage will "
"have to be allocated. Please pass a pre-allocated DataPackage if you expect a DataPackage "
"would be sent.");
}
else
{
URCL_LOG_WARN("Passed a pre-allocated RTDE package of type %u while a DataPackage was received. A new "
"DataPackage will have to be allocated. Please pass a pre-allocated DataPackage if you expect "
"a DataPackage would be sent.",
result->getType());
}
result = std::make_unique<DataPackage>(recipe_, protocol_version_);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • Consider make one if elseif case instead of an if else in an if

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The result = std::make_unique<DataPackage>(recipe_, protocol_version_); is common for both cases, which is why I catch those cases in the outer if. Depending on what's the cause for a not correctly allocated data package the output is different, hence the inner if.

Merging both layers into a single if...else if would require duplicating the important line potentially increasing the error missing one case when updating the code in the future. My opinion on this isn't that strong, if you think that the following structure is better, we can go with that:

      if (result == nullptr)
      {
        URCL_LOG_WARN("The passed result pointer is empty. A new DataPackage will "
                      "have to be allocated. Please pass a pre-allocated DataPackage if you expect a DataPackage "
                      "would be sent.");
        result = std::make_unique<DataPackage>(recipe_, protocol_version_);
      }
      else if (result->getType() != PackageType::RTDE_DATA_PACKAGE)
      {
        URCL_LOG_WARN("Passed a pre-allocated RTDE package of type %u while a DataPackage was received. A new "
                      "DataPackage will have to be allocated. Please pass a pre-allocated DataPackage if you expect "
                      "a DataPackage would be sent.",
                      result->getType());
        result = std::make_unique<DataPackage>(recipe_, protocol_version_);
      }

Copy link
Member

@urrsk urrsk left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Look very good. Just a small comment

urfeex and others added 14 commits February 11, 2026 16:08
Co-authored-by: Rune Søe-Knudsen <41109954+urrsk@users.noreply.github.com>
… unknown key

Initializing a robot with keys that the specific version doesn't know is
unaffected by this. This affects only configurations where users specify
a key that is unknown to the client library.
When no package can be retreived in the given timeout, don't log an
error. That hasn't been there before, as well.
Otherwise there is a race condition when checking whether the thread is
joinable.
@urfeex urfeex force-pushed the refactor_rtde_parsing branch from 0486b21 to 45eb65f Compare February 12, 2026 07:18
That test was not adding a real scenario and now that we have the real
reconnection tests using the fake server, we can remove that test.
@urfeex urfeex force-pushed the refactor_rtde_parsing branch from 45eb65f to bd1d00e Compare February 12, 2026 07:42
@urfeex urfeex requested a review from urrsk February 12, 2026 07:51
Copy link

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 1 potential issue.

Bugbot Autofix is OFF. To automatically fix reported issues with Cloud Agents, enable Autofix in the Cursor dashboard.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants