[3/9] Route ROS 2 publishers and subscribers through the middleware abstraction#9809
Draft
JArmandoAnaya wants to merge 7 commits into
Draft
Conversation
The carla-ros2-native ExternalProject configures a separate CMake process, so it never inherited CMake/Common.cmake's CMAKE_CXX_STANDARD and compiled at the Clang default (C++17) while the rest of the project is C++20. Set the standard explicitly in the inner project and pass it through the ExternalProject CMAKE_ARGS so every consumer of the ROS 2 headers (carla-server, libcarla-ros2-native.so, the LibCarla test suites) builds at the same language level. Vendor ExternalProjects (fastdds, foonathan_memory) keep their own standard; only our code is raised. Validated: all TUs of the shared lib compile clean at -std=gnu++20 with 0 errors and 0 warnings under UE's libc++, and a full package-development build runs to BUILD SUCCESSFUL. FastDDS 2.11.2 and Fast-CDR 1.1 headers are C++20-clean.
Introduce the middleware-neutral type layer of the ROS 2 middleware decoupling series, ported from ue4-dev: - types/msg/*.h: 31 plain C++ structs, one per ROS 2 message type, no DDS dependency, standard library headers only, all members value-initialized (upstream ue4-dev carla-simulator#9612). - types/CdrSerialization.h: serialize_to_cdr(), deserialize_from_cdr() and cdr_serialized_size() for all msg::* types using Fast-CDR (classic CDR, little-endian, DDS encapsulation header). The buffers are wire-compatible with every ROS 2 distribution and can be handed directly to FastDDS write() paths or CycloneDDS dds_writecdr(), removing the need for per-vendor generated type files. A kMaxCdrSequenceElements cap rejects hostile sequence lengths during deserialization (upstream ue4-dev carla-simulator#9643). - types/CdrTopicInfo.h: per-type type_name(), REP-2011 RIHS01 type hash and max_serialized_size(); the hashes let ROS 2 Iron and newer RMWs parse the type hash CARLA advertises via USER_DATA (upstream ue4-dev carla-simulator#9681). - types/UserDataFormat.h: build_user_data() / build_user_data_for<T>() helpers producing the REP-2016 "typehash=RIHS01_<hex>;" key-value payload (upstream ue4-dev carla-simulator#9681). UE5 adaptation: ue5-dev pins FastDDS 2.11.2 with bundled Fast-CDR 1.x, so CdrSerialization.h keeps the Fast-CDR 1.x spellings (eprosima::fastcdr::Cdr::DDS_CDR, getSerializedDataLength()) instead of the Fast-CDR 2.x forms the ue4-dev tip carries since its Fast-DDS 2.14.6 upgrade (ue4-dev carla-simulator#9789). Five lines differ; the wire format is identical either way and is pinned by the golden-bytes test added in the follow-up test commit. The FastDDSConversions.h / FastDDSTypeMap.h files from carla-simulator#9612 are deliberately not ported; they were superseded by unified CDR upstream. The new headers are not referenced by any build target yet; they start compiling when the middleware abstraction lands in the next PR of the series. (adapted from ue4-dev 542959a) (adapted from ue4-dev f53144c) (adapted from ue4-dev c64e8f4) (adapted from ue4-dev b865088)
Add LibCarla/source/test/server/test_ros2_serialization.cpp with the cdr_topic_info (2 cases) and cdr_serialization (22 cases) GTest groups from the ue4-dev middleware suite: POD round-trips for every message family, a golden-bytes guard that pins the classic CDR little-endian encapsulation so any drift to XCDRv2 or big-endian is caught deterministically, truncated / corrupt-encapsulation / hostile-sequence-length rejection paths exercising the kMaxCdrSequenceElements cap, and the contract that cdr_serialized_size(msg) equals serialize_to_cdr(msg).size(). The tests exercise only Fast-CDR plus the new foundation headers; no DDS participant is created and libfastrtps is not linked. Build wiring, gated on ENABLE_ROS2: - libcarla_test_server gains the Ros2Native install include dir as a SYSTEM include and links libfastcdr.so from the fastdds ExternalProject install tree; add_dependencies orders the test compile and link after the fastdds install step. - The fastdds ExternalProject declares libfastcdr.so via INSTALL_BYPRODUCTS so Ninja accepts the full-path link input on a clean tree. - Without ENABLE_ROS2 the test source is removed from the server test glob and the suite builds exactly as before. (adapted from ue4-dev f53144c) (adapted from ue4-dev b865088)
Introduce the vendor-neutral middleware strategy layer of the ROS 2 middleware decoupling series and its FastDDS implementation, compiled into libcarla-ros2-native.so. Ported from ue4-dev: - middleware/Middleware.h: the Middleware enum plus the string and ROS 2 type-name helpers; the CycloneDDS value and its availability branches are present but stay compiled out until the CycloneDDS middleware lands (upstream ue4-dev carla-simulator#9608). - middleware/IPublisherMiddleware.h, ISubscriberMiddleware.h: the type-erased publisher/subscriber strategy interfaces. Subscribers write received samples straight into caller-owned storage to avoid a copy (upstream ue4-dev carla-simulator#9608). - middleware/MiddlewareFactory.h: creates the active middleware for a traits type; each vendor arm is double-gated on its CARLA_ROS2_MIDDLEWARE_* macro and CARLA_ROS2_MIDDLEWARE_TESTING so the suite exercises the availability logic without linking DDS (upstream ue4-dev carla-simulator#9608). - middleware/ActiveMiddleware.{h,cpp}: a DDS-free bridge (SetActiveMiddleware) so ROS2.cpp, the only ROS 2 translation unit in carla-server, can select the middleware without any DDS header crossing the shared-library boundary. Nothing calls it until the cutover; the definition ships now so the shared lib has a translation unit that compiles MiddlewareFactory.h and the FastDDS headers with the real vendor macros. - middleware/fastdds/GenericCdrPubSubType.h: one FastDDS TopicDataType that serializes every carla::ros2::msg::* struct through the unified CdrSerialization.h path, replacing the generated per-type PubSubType classes; getSerializedSizeProvider reports the actual instance size so variable-length payloads (camera frames, point clouds) are not bounded by the static max size (upstream ue4-dev carla-simulator#9643). - middleware/fastdds/FastDDS{Publisher,Subscriber}Middleware.h: the FastDDS strategy implementations. Each endpoint advertises the REP-2016 "typehash=RIHS01_<hex>;" USER_DATA so Jazzy RMWs match on the REP-2011 type hash (upstream ue4-dev carla-simulator#9681). - middleware/fastdds/FastDDSSharedParticipant.{h,cpp}: a refcounted process-wide DomainParticipant shared across all FastDDS endpoints, avoiding the discovery storm that destroying N participants back to back caused on shutdown (upstream ue4-dev carla-simulator#9681). UE5 adaptation: ue5-dev pins FastDDS 2.11.2 with bundled Fast-CDR 1.x, so GenericCdrPubSubType.h keeps the Fast-CDR 1.x spellings (eprosima::fastcdr::Cdr::DDS_CDR, getSerializedDataLength()) matching CdrSerialization.h from the previous PR of the series. The wire format is classic CDR little-endian and is unchanged. The existing publishers keep using the generated FastDDS types; the cutover to this abstraction and the PublisherImpl/SubscriberImpl rewrite land in the next PR of the series. No behavior change. (adapted from ue4-dev 82c28e2) (adapted from ue4-dev f53144c) (adapted from ue4-dev 02a83ef) (adapted from ue4-dev c64e8f4)
Extend the ROS 2 serialization suite with the middleware abstraction cases and rename it to test_ros2_middleware.cpp to match the layer it now covers. 32 new cases, all pure logic with no DDS participant: - middleware_to_string / middleware_from_string / middleware_available / middleware_type_name: the Middleware enum string and type-name helpers, and compile-time availability reporting. - MiddlewareFactoryFixture: set/get, strict resolution, and that creating an unavailable vendor middleware returns nullptr. - generic_cdr_pubsubtype: GenericCdrPubSubType round-trips a fixed-size and a string-bearing message through a real FastDDS SerializedPayload_t, name matches CdrTopicInfo, createData/deleteData and getKey behave. - generic_cdr_pubsubtype_large_payload: getSerializedSizeProvider reports the actual instance size so a full camera frame and a large point cloud serialize past the static max size. The file defines CARLA_ROS2_MIDDLEWARE_FASTDDS and CARLA_ROS2_MIDDLEWARE_TESTING so the factory availability branches are active while the real DDS entities stay compiled out; only GenericCdrPubSubType pulls actual Fast-DDS/Fast-CDR headers. When ENABLE_ROS2 is ON, libcarla_test_server now also links libfastrtps.so (added as an INSTALL_BYPRODUCTS on the fastdds ExternalProject) for the TopicDataType base and SerializedPayload_t, with -rpath-link / -rpath / --disable-new-dtags so its private foonathan_memory dependency resolves at link and run time. When OFF, the renamed source is removed from the glob and the suite builds exactly as before. (adapted from ue4-dev 82c28e2) (adapted from ue4-dev f53144c) (adapted from ue4-dev c64e8f4)
…bstraction Cut the ROS 2 publishers and subscribers over to the middleware strategy layer added earlier in the series. PublisherImpl and SubscriberImpl stop talking to FastDDS directly and delegate to an IPublisherMiddleware / ISubscriberMiddleware created by MiddlewareFactory, so the transport is chosen at runtime with no vendor header reaching these templates. Ported from ue4-dev: - publishers/PublisherImpl.h, subscribers/SubscriberImpl.h: rewritten from raw FastDDS plumbing (participant/writer/reader/listener) to hold a unique_ptr<I*Middleware> from MiddlewareFactory plus one owned message. The LIBCARLA_WITH_GTEST seams (SetMiddlewareForTesting, and the subscriber's SimulateMessageReceiptForTesting) let the suite inject fakes without a live DDS participant. The traits contract is reduced to a single msg_type typedef. - publishers/Carla*Publisher.cpp, subscribers/*Subscriber.cpp: every publisher and subscriber traits now binds a carla::ros2::msg::* POD instead of the generated sensor_msgs::msg::X + XPubSubType pair, and the message bodies use direct member access in place of the generated field accessors. The DDS type name, hash and size come from the CdrTopicInfo<msg_type> specialization consumed inside the middleware. - ROS2.cpp: ROS2::Enable selects the middleware through the DDS-free SetActiveMiddleware(Middleware::FastDDS) bridge before creating any publisher or subscriber. FastDDS is the only middleware today; the runtime --rmw= selection lands later in the series. The developer BasicPublisher / BasicSubscriber / BasicListener demo path (WITH_ROS2_DEMO) also moves off the generated std_msgs::msg::String: BasicPublisher through its traits, and the raw-FastDDS BasicSubscriber and BasicListener by registering GenericCdrPubSubType<msg::String>, so the generated type files can be removed cleanly in the next PR. FastDDS keeps publishing exactly as before; the wire format is unchanged. (adapted from ue4-dev f2c0760)
Add the PublisherImpl and SubscriberImpl cases to the ROS 2 middleware suite, 14 new cases covering the cutover. Ported from ue4-dev: - MockPublisherMiddleware / MockSubscriberMiddleware: minimal IPublisherMiddleware / ISubscriberMiddleware fakes that record the calls the impls make. - publisher_impl (7): GetMessage returns the owned buffer, Init and Publish delegate to the injected middleware, Publish before Init fails, IsAlive and GetTopicName delegate, and the owned message pointer flows through Publish unchanged. - subscriber_impl (7): the new-message flag starts false, Init threads the storage and flag pointers to the middleware and propagates its result, GetMessage clears the flag, and SimulateMessageReceipt drives the receive path without a live participant. The impls are injected through the LIBCARLA_WITH_GTEST seams, so no DDS participant is created. The cases reuse the TestMsg traits already defined for the middleware groups. (adapted from ue4-dev f2c0760)
|
Thanks for opening this pull request! The maintainers of this repository would appreciate it if you would update our CHANGELOG.md based on your changes. |
JArmandoAnaya
changed the title
This was referenced Jul 8, 2026
youtalk
reviewed
Jul 13, 2026
youtalk
left a comment
youtalk
left a comment
Contributor
There was a problem hiding this comment.
Thank you for the quick PR work! One correctness note inline below.
| std::mutex _message_mutex; | ||
| std::unique_ptr<ISubscriberMiddleware> _middleware; | ||
| msg_type _message{}; | ||
| bool _new_message{false}; |
Contributor
There was a problem hiding this comment.
Restore the mutex + atomic around _message/_new_message. _message is written on the DDS receive thread while the game thread reads it in GetMessage(), and _new_message is now a plain bool. The pre-refactor code guarded _message with a mutex and used std::atomic<bool> (release/acquire) for the flag. As-is this is a data race: torn reads of multi-field control messages and unspecified flag visibility. Suggest having the middleware copy into a staging buffer and publish it under a lock, as before.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Third PR of the ROS 2 middleware decoupling series for ue5-dev (tracking issue #9762): the cutover that routes every publisher and subscriber through the middleware abstraction added in the previous PR. Ported from the merged ue4-dev series (#9619).
PublisherImplandSubscriberImplstop talking to FastDDS directly and delegate to anIPublisherMiddleware/ISubscriberMiddlewarecreated byMiddlewareFactory, so the transport is chosen at runtime with no vendor header reaching these templates. Every publisher and subscriber traits now binds acarla::ros2::msg::*POD instead of the generatedsensor_msgs::msg::X+XPubSubTypepair, and the message bodies use direct member access; the DDS type name, hash and size come from theCdrTopicInfo<msg_type>specialization consumed inside the middleware.ROS2::Enableselects the middleware through the DDS-freeSetActiveMiddleware(Middleware::FastDDS)bridge before any endpoint is created.FastDDS keeps publishing exactly as before; the wire format is unchanged. After this PR no production code references the generated
types/*PubSubTypes.*, so the next PR removes them. Runtime--rmw=selection lands later in the series.Depends on #9808; review after it merges. The diff below includes that PR's commits until then.
What this PR changes
publishers/PublisherImpl.handsubscribers/SubscriberImpl.hfrom raw FastDDS plumbing to aunique_ptr<I*Middleware>obtained fromMiddlewareFactory, with theLIBCARLA_WITH_GTESTseams (SetMiddlewareForTesting,SimulateMessageReceiptForTesting). Traits reduce to a singlemsg_typetypedef.Carla*Publisher.cpp) and subscriber (*Subscriber.cpp) traits to thecarla::ros2::msg::*PODs, converting the generated field accessors to direct member access.ROS2.cpp:ROS2::EnablecallsSetActiveMiddleware(Middleware::FastDDS)before creating any endpoint.WITH_ROS2_DEMOBasic*demo path off the generatedstd_msgs::msg::String:BasicPublisherthrough its traits, and the raw-FastDDSBasicSubscriber/BasicListenerby registeringGenericCdrPubSubType<msg::String>, so the generated types delete cleanly next.publisher_impl(7) andsubscriber_impl(7) cases withMock*Middlewarefakes totest_ros2_middleware.cpp. No CMake change: theRos2Nativeglobs and the test link already cover the touched directories.PR series
types/msg/*POD structs,CdrSerialization/CdrTopicInfo/UserDataFormat, CDR round-trip GTestsGenericCdrPubSubType, shared participant,ActiveMiddlewarebridgePublisherImpl/SubscriberImplto delegate through the factory, migrate publisher traits to the POD types,ROS2.cppcallsSetActiveMiddlewaretypes/*set--rmw=selection, plugin integration, examples, docsCarlaSettings/CarlaEnginewiring, launch flag, examples, docs, consolidated CHANGELOG--rmw=zenoh)middleware/zenoh/*, factory branch, zenoh-c build inRos2Native, rviz example, tests--ros-domain-id)MiddlewareConfig, resolution--ros-domain-id>ROS_DOMAIN_ID> 0, honored by all middlewares, docs + example + testsRelated: #9762, #9808
Where has this been tested?
libcarla_test_server210/210 (including the 14 new impl cases);libcarla_test_client193/193, unaffected;package-developmentbuild green.Possible Drawbacks
This change is