[3/9] Route ROS 2 publishers and subscribers through the middleware abstraction#9809
Conversation
The carla-ros2-native ExternalProject configures a separate CMake process, so it never inherited CMake/Common.cmake's CMAKE_CXX_STANDARD and compiled at the Clang default (C++17) while the rest of the project is C++20. Set the standard explicitly in the inner project and pass it through the ExternalProject CMAKE_ARGS so every consumer of the ROS 2 headers (carla-server, libcarla-ros2-native.so, the LibCarla test suites) builds at the same language level. Vendor ExternalProjects (fastdds, foonathan_memory) keep their own standard; only our code is raised. Validated: all TUs of the shared lib compile clean at -std=gnu++20 with 0 errors and 0 warnings under UE's libc++, and a full package-development build runs to BUILD SUCCESSFUL. FastDDS 2.11.2 and Fast-CDR 1.1 headers are C++20-clean.
Introduce the middleware-neutral type layer of the ROS 2 middleware decoupling series, ported from ue4-dev: - types/msg/*.h: 31 plain C++ structs, one per ROS 2 message type, no DDS dependency, standard library headers only, all members value-initialized (upstream ue4-dev carla-simulator#9612). - types/CdrSerialization.h: serialize_to_cdr(), deserialize_from_cdr() and cdr_serialized_size() for all msg::* types using Fast-CDR (classic CDR, little-endian, DDS encapsulation header). The buffers are wire-compatible with every ROS 2 distribution and can be handed directly to FastDDS write() paths or CycloneDDS dds_writecdr(), removing the need for per-vendor generated type files. A kMaxCdrSequenceElements cap rejects hostile sequence lengths during deserialization (upstream ue4-dev carla-simulator#9643). - types/CdrTopicInfo.h: per-type type_name(), REP-2011 RIHS01 type hash and max_serialized_size(); the hashes let ROS 2 Iron and newer RMWs parse the type hash CARLA advertises via USER_DATA (upstream ue4-dev carla-simulator#9681). - types/UserDataFormat.h: build_user_data() / build_user_data_for<T>() helpers producing the REP-2016 "typehash=RIHS01_<hex>;" key-value payload (upstream ue4-dev carla-simulator#9681). UE5 adaptation: ue5-dev pins FastDDS 2.11.2 with bundled Fast-CDR 1.x, so CdrSerialization.h keeps the Fast-CDR 1.x spellings (eprosima::fastcdr::Cdr::DDS_CDR, getSerializedDataLength()) instead of the Fast-CDR 2.x forms the ue4-dev tip carries since its Fast-DDS 2.14.6 upgrade (ue4-dev carla-simulator#9789). Five lines differ; the wire format is identical either way and is pinned by the golden-bytes test added in the follow-up test commit. The FastDDSConversions.h / FastDDSTypeMap.h files from carla-simulator#9612 are deliberately not ported; they were superseded by unified CDR upstream. The new headers are not referenced by any build target yet; they start compiling when the middleware abstraction lands in the next PR of the series. (adapted from ue4-dev 542959a) (adapted from ue4-dev f53144c) (adapted from ue4-dev c64e8f4) (adapted from ue4-dev b865088)
Add LibCarla/source/test/server/test_ros2_serialization.cpp with the cdr_topic_info (2 cases) and cdr_serialization (22 cases) GTest groups from the ue4-dev middleware suite: POD round-trips for every message family, a golden-bytes guard that pins the classic CDR little-endian encapsulation so any drift to XCDRv2 or big-endian is caught deterministically, truncated / corrupt-encapsulation / hostile-sequence-length rejection paths exercising the kMaxCdrSequenceElements cap, and the contract that cdr_serialized_size(msg) equals serialize_to_cdr(msg).size(). The tests exercise only Fast-CDR plus the new foundation headers; no DDS participant is created and libfastrtps is not linked. Build wiring, gated on ENABLE_ROS2: - libcarla_test_server gains the Ros2Native install include dir as a SYSTEM include and links libfastcdr.so from the fastdds ExternalProject install tree; add_dependencies orders the test compile and link after the fastdds install step. - The fastdds ExternalProject declares libfastcdr.so via INSTALL_BYPRODUCTS so Ninja accepts the full-path link input on a clean tree. - Without ENABLE_ROS2 the test source is removed from the server test glob and the suite builds exactly as before. (adapted from ue4-dev f53144c) (adapted from ue4-dev b865088)
Introduce the vendor-neutral middleware strategy layer of the ROS 2 middleware decoupling series and its FastDDS implementation, compiled into libcarla-ros2-native.so. Ported from ue4-dev: - middleware/Middleware.h: the Middleware enum plus the string and ROS 2 type-name helpers; the CycloneDDS value and its availability branches are present but stay compiled out until the CycloneDDS middleware lands (upstream ue4-dev carla-simulator#9608). - middleware/IPublisherMiddleware.h, ISubscriberMiddleware.h: the type-erased publisher/subscriber strategy interfaces. Subscribers write received samples straight into caller-owned storage to avoid a copy (upstream ue4-dev carla-simulator#9608). - middleware/MiddlewareFactory.h: creates the active middleware for a traits type; each vendor arm is double-gated on its CARLA_ROS2_MIDDLEWARE_* macro and CARLA_ROS2_MIDDLEWARE_TESTING so the suite exercises the availability logic without linking DDS (upstream ue4-dev carla-simulator#9608). - middleware/ActiveMiddleware.{h,cpp}: a DDS-free bridge (SetActiveMiddleware) so ROS2.cpp, the only ROS 2 translation unit in carla-server, can select the middleware without any DDS header crossing the shared-library boundary. Nothing calls it until the cutover; the definition ships now so the shared lib has a translation unit that compiles MiddlewareFactory.h and the FastDDS headers with the real vendor macros. - middleware/fastdds/GenericCdrPubSubType.h: one FastDDS TopicDataType that serializes every carla::ros2::msg::* struct through the unified CdrSerialization.h path, replacing the generated per-type PubSubType classes; getSerializedSizeProvider reports the actual instance size so variable-length payloads (camera frames, point clouds) are not bounded by the static max size (upstream ue4-dev carla-simulator#9643). - middleware/fastdds/FastDDS{Publisher,Subscriber}Middleware.h: the FastDDS strategy implementations. Each endpoint advertises the REP-2016 "typehash=RIHS01_<hex>;" USER_DATA so Jazzy RMWs match on the REP-2011 type hash (upstream ue4-dev carla-simulator#9681). - middleware/fastdds/FastDDSSharedParticipant.{h,cpp}: a refcounted process-wide DomainParticipant shared across all FastDDS endpoints, avoiding the discovery storm that destroying N participants back to back caused on shutdown (upstream ue4-dev carla-simulator#9681). UE5 adaptation: ue5-dev pins FastDDS 2.11.2 with bundled Fast-CDR 1.x, so GenericCdrPubSubType.h keeps the Fast-CDR 1.x spellings (eprosima::fastcdr::Cdr::DDS_CDR, getSerializedDataLength()) matching CdrSerialization.h from the previous PR of the series. The wire format is classic CDR little-endian and is unchanged. The existing publishers keep using the generated FastDDS types; the cutover to this abstraction and the PublisherImpl/SubscriberImpl rewrite land in the next PR of the series. No behavior change. (adapted from ue4-dev 82c28e2) (adapted from ue4-dev f53144c) (adapted from ue4-dev 02a83ef) (adapted from ue4-dev c64e8f4)
Extend the ROS 2 serialization suite with the middleware abstraction cases and rename it to test_ros2_middleware.cpp to match the layer it now covers. 32 new cases, all pure logic with no DDS participant: - middleware_to_string / middleware_from_string / middleware_available / middleware_type_name: the Middleware enum string and type-name helpers, and compile-time availability reporting. - MiddlewareFactoryFixture: set/get, strict resolution, and that creating an unavailable vendor middleware returns nullptr. - generic_cdr_pubsubtype: GenericCdrPubSubType round-trips a fixed-size and a string-bearing message through a real FastDDS SerializedPayload_t, name matches CdrTopicInfo, createData/deleteData and getKey behave. - generic_cdr_pubsubtype_large_payload: getSerializedSizeProvider reports the actual instance size so a full camera frame and a large point cloud serialize past the static max size. The file defines CARLA_ROS2_MIDDLEWARE_FASTDDS and CARLA_ROS2_MIDDLEWARE_TESTING so the factory availability branches are active while the real DDS entities stay compiled out; only GenericCdrPubSubType pulls actual Fast-DDS/Fast-CDR headers. When ENABLE_ROS2 is ON, libcarla_test_server now also links libfastrtps.so (added as an INSTALL_BYPRODUCTS on the fastdds ExternalProject) for the TopicDataType base and SerializedPayload_t, with -rpath-link / -rpath / --disable-new-dtags so its private foonathan_memory dependency resolves at link and run time. When OFF, the renamed source is removed from the glob and the suite builds exactly as before. (adapted from ue4-dev 82c28e2) (adapted from ue4-dev f53144c) (adapted from ue4-dev c64e8f4)
…bstraction Cut the ROS 2 publishers and subscribers over to the middleware strategy layer added earlier in the series. PublisherImpl and SubscriberImpl stop talking to FastDDS directly and delegate to an IPublisherMiddleware / ISubscriberMiddleware created by MiddlewareFactory, so the transport is chosen at runtime with no vendor header reaching these templates. Ported from ue4-dev: - publishers/PublisherImpl.h, subscribers/SubscriberImpl.h: rewritten from raw FastDDS plumbing (participant/writer/reader/listener) to hold a unique_ptr<I*Middleware> from MiddlewareFactory plus one owned message. The LIBCARLA_WITH_GTEST seams (SetMiddlewareForTesting, and the subscriber's SimulateMessageReceiptForTesting) let the suite inject fakes without a live DDS participant. The traits contract is reduced to a single msg_type typedef. - publishers/Carla*Publisher.cpp, subscribers/*Subscriber.cpp: every publisher and subscriber traits now binds a carla::ros2::msg::* POD instead of the generated sensor_msgs::msg::X + XPubSubType pair, and the message bodies use direct member access in place of the generated field accessors. The DDS type name, hash and size come from the CdrTopicInfo<msg_type> specialization consumed inside the middleware. - ROS2.cpp: ROS2::Enable selects the middleware through the DDS-free SetActiveMiddleware(Middleware::FastDDS) bridge before creating any publisher or subscriber. FastDDS is the only middleware today; the runtime --rmw= selection lands later in the series. The developer BasicPublisher / BasicSubscriber / BasicListener demo path (WITH_ROS2_DEMO) also moves off the generated std_msgs::msg::String: BasicPublisher through its traits, and the raw-FastDDS BasicSubscriber and BasicListener by registering GenericCdrPubSubType<msg::String>, so the generated type files can be removed cleanly in the next PR. FastDDS keeps publishing exactly as before; the wire format is unchanged. (adapted from ue4-dev f2c0760)
Add the PublisherImpl and SubscriberImpl cases to the ROS 2 middleware suite, 14 new cases covering the cutover. Ported from ue4-dev: - MockPublisherMiddleware / MockSubscriberMiddleware: minimal IPublisherMiddleware / ISubscriberMiddleware fakes that record the calls the impls make. - publisher_impl (7): GetMessage returns the owned buffer, Init and Publish delegate to the injected middleware, Publish before Init fails, IsAlive and GetTopicName delegate, and the owned message pointer flows through Publish unchanged. - subscriber_impl (7): the new-message flag starts false, Init threads the storage and flag pointers to the middleware and propagates its result, GetMessage clears the flag, and SimulateMessageReceipt drives the receive path without a live participant. The impls are injected through the LIBCARLA_WITH_GTEST seams, so no DDS participant is created. The cases reuse the TestMsg traits already defined for the middleware groups. (adapted from ue4-dev f2c0760)
|
Thanks for opening this pull request! The maintainers of this repository would appreciate it if you would update our CHANGELOG.md based on your changes. |
youtalk
left a comment
There was a problem hiding this comment.
Thank you for the quick PR work! One correctness note inline below.
| std::mutex _message_mutex; | ||
| std::unique_ptr<ISubscriberMiddleware> _middleware; | ||
| msg_type _message{}; | ||
| bool _new_message{false}; |
There was a problem hiding this comment.
Restore the mutex + atomic around _message/_new_message. _message is written on the DDS receive thread while the game thread reads it in GetMessage(), and _new_message is now a plain bool. The pre-refactor code guarded _message with a mutex and used std::atomic<bool> (release/acquire) for the flag. As-is this is a data race: torn reads of multi-field control messages and unspecified flag visibility. Suggest having the middleware copy into a staging buffer and publish it under a lock, as before.
Description
Third PR of the ROS 2 middleware decoupling series for ue5-dev (tracking issue #9762): the cutover that routes every publisher and subscriber through the middleware abstraction added in the previous PR. Ported from the merged ue4-dev series (#9619).
PublisherImplandSubscriberImplstop talking to FastDDS directly and delegate to anIPublisherMiddleware/ISubscriberMiddlewarecreated byMiddlewareFactory, so the transport is chosen at runtime with no vendor header reaching these templates. Every publisher and subscriber traits now binds acarla::ros2::msg::*POD instead of the generatedsensor_msgs::msg::X+XPubSubTypepair, and the message bodies use direct member access; the DDS type name, hash and size come from theCdrTopicInfo<msg_type>specialization consumed inside the middleware.ROS2::Enableselects the middleware through the DDS-freeSetActiveMiddleware(Middleware::FastDDS)bridge before any endpoint is created.FastDDS keeps publishing exactly as before; the wire format is unchanged. After this PR no production code references the generated
types/*PubSubTypes.*, so the next PR removes them. Runtime--rmw=selection lands later in the series.Depends on #9808; review after it merges. The diff below includes that PR's commits until then.
What this PR changes
publishers/PublisherImpl.handsubscribers/SubscriberImpl.hfrom raw FastDDS plumbing to aunique_ptr<I*Middleware>obtained fromMiddlewareFactory, with theLIBCARLA_WITH_GTESTseams (SetMiddlewareForTesting,SimulateMessageReceiptForTesting). Traits reduce to a singlemsg_typetypedef.Carla*Publisher.cpp) and subscriber (*Subscriber.cpp) traits to thecarla::ros2::msg::*PODs, converting the generated field accessors to direct member access.ROS2.cpp:ROS2::EnablecallsSetActiveMiddleware(Middleware::FastDDS)before creating any endpoint.WITH_ROS2_DEMOBasic*demo path off the generatedstd_msgs::msg::String:BasicPublisherthrough its traits, and the raw-FastDDSBasicSubscriber/BasicListenerby registeringGenericCdrPubSubType<msg::String>, so the generated types delete cleanly next.publisher_impl(7) andsubscriber_impl(7) cases withMock*Middlewarefakes totest_ros2_middleware.cpp. No CMake change: theRos2Nativeglobs and the test link already cover the touched directories.PR series
types/msg/*POD structs,CdrSerialization/CdrTopicInfo/UserDataFormat, CDR round-trip GTestsGenericCdrPubSubType, shared participant,ActiveMiddlewarebridgePublisherImpl/SubscriberImplto delegate through the factory, migrate publisher traits to the POD types,ROS2.cppcallsSetActiveMiddlewaretypes/*set--rmw=selection, plugin integration, examples, docsCarlaSettings/CarlaEnginewiring, launch flag, examples, docs, consolidated CHANGELOG--rmw=zenoh)middleware/zenoh/*, factory branch, zenoh-c build inRos2Native, rviz example, tests--ros-domain-id)MiddlewareConfig, resolution--ros-domain-id>ROS_DOMAIN_ID> 0, honored by all middlewares, docs + example + testsRelated: #9762, #9808
Where has this been tested?
libcarla_test_server210/210 (including the 14 new impl cases);libcarla_test_client193/193, unaffected;package-developmentbuild green.Possible Drawbacks
This change is