Skip to content

.instructions.md

Latest commit

 

History

History
181 lines (140 loc) · 9.79 KB

.instructions.md

File metadata and controls

181 lines (140 loc) · 9.79 KB
Raw

Room Concept Project Instructions

Build System

  • Always use cbuild to compile - this is the standard command for this workspace
  • Do NOT use make, ninja, or cmake --build directly from terminal
  • CMake Tools extension in VS Code is the primary build interface
  • Build artifacts are generated in build/ directory
  • When configuration changes are needed, use CMake Tools "Delete Cache and Reconfigure" option

RoboComp Overview

RoboComp is an open-source robotics middleware framework designed for building distributed robotic systems. It provides:

  • Code Generation Tool: robocompdsl - generates boilerplate C++ and Python components from declarative interface definitions (.cdsl files)
  • Component Model: RoboComp components are autonomous agents that inherit from GenericWorker and implement domain-specific interfaces
  • Communication: Components communicate via Ice middleware (RPC-based), though in this project we primarily use DSR (see CORTEX below)
  • Configuration: Each component has a config file (INI format) loaded at startup via ConfigLoader
  • Lifecycle: Components follow: initialize() → periodic compute() calls → finalize() cleanup
  • RoboComp codebase is located in https://github.com/robocomp/robocomp_core and the codegenerator in https://github.com/robocomp/robocomp_tools`.

For this project, see room_concept.cdsl which defines the component interface.

CORTEX Description

Memories Memories are distributed, graph-based data structures operated by agents. They are supported by the libdsr libraries that provide a graph (G) editing API defined in, /home/robocomp/components/cortex/api/include/dsr/api/dsr_api.h, and several sub-apis for especific tasks: dsr_inner_eigen_api.h for kinematic transformations in the RT-edges tree and dsr_camera_api.h to access camera atributes. Each memory is a graph of nodes and edges, where each node and edge can have multiple attributes. The DSR API provides functions to create, read, update, and delete nodes, edges, and attributes in the memory. The memory is distributed across multiple agents, which can read and write to it concurrently. The memory is designed to be flexible and extensible, allowing for different types of data to be stored and different types of agents to interact with it.

Agents are Linux processes and RoboComp components that can edit the distributed memories. They are generated using RoboComp's code generation tool, robocompdsl, and can be written in C++ or Python. Agents can have different roles, such as sensors that write data to the memory, controllers that read data from the memory and make decisions, and effectors that read data from the memory and act on the environment. Agents interact with the memories using the DSR API. They do not talk to each other directly.

CORTEX code base in located in https://github.com/robocomp/cortex and the DSR API documentation is available at https://robocomp.github.io/cortex/. ** DO NOT use raw access to the core data structures. Always use the API

Frame Conventions

  • Get frame conventions from ../FRAMES.md - this file defines the standard coordinate frames and rotation conventions used throughout the project
  • Standard Room Frame: Origin at room center, X forward, Y right, Z up
  • Camera Frame: Y is forward (depth direction), following RoboComp standard
  • Qt3D Viewer Rotation: Uses α = π + θ convention (standard) for alignment with room yaw

Key DSR Attributes

Read dsr attributes documentation for details on how to use these in code from /home/robocomp/components/cortex/core/include/dsr/core/types/type_checking/dsr_attr_name.h

Common attributes in this project:

  • room_height (float): Ceiling height in meters, stored on room node
  • cam_rgb (uint8_t vector): RGB image payload from zed camera
  • cam_rgb_width, cam_rgb_height (int): Image dimensions
  • cam_rgb_focalx, cam_rgb_focaly (float): Camera focal lengths
  • delimiting_polygon_x, delimiting_polygon_y (vector): Room floor polygon corners

DSR API Pattern for attributes:

// Reading an attribute
if (auto val = graph_->get_attrib_by_name<room_height_att>(room_node); val.has_value()) {
    float height = val.value();
}

// Writing an attribute
node.attrs()[room_height_str.data()] = DSR::Attribute{height_value, 0, 0};

Project Structure

Source Layout:

  • src/specificworker.{cpp,h}: Main compute loop, room creation, pose synchronization
  • src/camera_visualizer.{cpp,h}: Qt6 dialog for RGB + geometry overlay visualization
  • src/room_concept.{cpp,h}: Room model, corner detection from lidar point clouds
  • src/room_model.{cpp,h}: Geometric representation of room with Eigen-based transforms
  • src/svg_room_loader.{cpp,h}: Loads initial room geometry from SVG files
  • src/viewer_2d.{cpp,h}: Qt scene-based 2D visualization of room + robot pose
  • generated/: Auto-generated Ice interface code (do not edit manually)

Configuration:

  • etc/config: Runtime parameters (room size, default values)
  • room_concept.cdsl: Component interface definition
  • shadow.json: Initial room geometry data

Code Standards

  • Language: C++23 standard (enforced by CMake)
  • Key Libraries: Eigen3, Qt6, RoboComp Cortex DSR, CUDA (optional)
  • Always check optional returns: All DSR API calls return std::optional<T> - verify with .has_value() before accessing
  • Geometry operations: Use Eigen3 (Vector2f, Vector3f, Matrix3f, Affine2f)
  • GUI: Qt6 with auto-moc/uic (do not manually run moc/uic)
  • Logging: Use std::print for logging
  • Logic operations: Use and and or instead of && and || for better readability
  • Indentation: Open brace on next line. 4 spaces per indent level. No tabs.
  • Naming conventions: snake_case for variables and functions, PascalCase for classes and structs
  • Comments: Use // for single line comments, /** */ for multi-line comments. Comments should explain why something is done, not what is being done (the code should be self-explanatory). Avoid redundant comments.

Common Tasks

Working with the DSR Graph

Accessing the graph:

  • Global graph pointer is G (RoboComp convention)
  • Always verify optional returns: if (auto node = G->get_node(name); node.has_value()) { ... }

Getting/Setting attributes:

// Read attribute
if (auto attr = G->get_attrib_by_name<attribute_type_att>(node); attr.has_value()) {
    T value = attr.value();
}

// Write attribute
node.attrs()[attribute_name_str.data()] = DSR::Attribute{value, 0, 0};
G->update_node(node);

Using sub-APIs:

  • CameraAPI: Fetch camera data (RGB, intrinsics) 
    auto cam_node = G->get_node("zed");
if (cam_node.has_value()) {
    auto camera_api = G->get_camera_api(cam_node.value());
}
  • InnerEigenAPI: Transform points between frames 
    auto inner_api = G->get_inner_eigen_api();
auto p_in_cam = inner_api->transform("camera_frame", point_3d, "world_frame");
  • RT_API: Work with rotation/translation edges (kinematic tree)

Debugging Camera Visualization

  1. Verify zed node exists: G->get_node("zed")
  2. Verify RGB data populated: Check cam_rgb attribute has data
  3. Verify room node: G->get_node("room") with room_height attribute
  4. Check frame transforms: Verify room→robot and robot→world RT edges exist
  5. Test projection: Add debug output in project_points_to_image() method
  6. Check camera intrinsics: Verify cam_rgb_focalx/y/cx/cy are reasonable values

Adding a New Feature

  1. Modify the CDSL interface if new RPC methods needed
  2. Regenerate code: cbuild will auto-regenerate from .cdsl
  3. Implement in SpecificWorker: Add logic to compute() or new slots
  4. If using DSR attributes: Add to room_node or appropriate node via node.attrs()
  5. Build and test: cbuild to compile, then test with running system

Fixing Build Issues

  1. If configuration fails: "Delete Cache and Reconfigure" via CMake Tools command palette
  2. If link errors persist: Check that cortex (dsr_api, dsr_core) built successfully
  3. If Qt autogen fails: Clean build folder and reconfigure
  4. Always use cbuild - never try workarounds with direct cmake/ninja commands

Important Notes

DSR Graph Management

  • Thread-safe: DSR graph is thread-safe for concurrent read/write operations
  • Consistency: Always check .has_value() on optional returns - DSR API is defensive
  • Performance: Cache frequently accessed nodes/attributes if reading in tight loops
  • Transactions: Updates are atomic at the node level, not graph-level

Room Geometry

  • Room height is dynamically read from DSR room_height attribute (fallback 2.4m if not present)
  • Room corners are stored as 2D polygon in room node (X, Y coordinates on floor)
  • 3D room geometry is computed from 2D floor polygon + room_height for visualization

Camera Visualization

  • CameraVisualizer uses NaN for invalid/out-of-bounds projections - safe for rendering (skipped when drawing)
  • InnerEigenAPI requires both source AND destination frame names as strings
  • Camera intrinsics are always fetched fresh from DSR attributes, not cached
  • RGB payload can be RGBA8888, RGB888, or Grayscale8 - size checking determines format

Performance Considerations

  • Point cloud processing (corner detection) runs in separate lidar thread
  • Camera visualization refreshes at 10 Hz (configurable via QTimer in camera_visualizer.cpp)
  • DSR graph updates are published to all connected agents - minimize write frequency
  • Lidar and compute operate independently to avoid blocking the GUI

Resources

  • RoboComp Documentation: http://robocomp.readthedocs.io/
  • Cortex DSR API: /home/robocomp/robocomp/api/include/dsr/api/dsr_api.h
  • Component Examples: /home/robocomp/robocomp/components/
  • Frame Conventions: ../FRAMES.md
  • Build Configuration: CMakeLists.txt in this directory