Skip to content

README.md

Latest commit

 

History

History
206 lines (143 loc) · 5.91 KB

README.md

File metadata and controls

206 lines (143 loc) · 5.91 KB
Raw

PySpotObserver

A clean, Pythonic interface for streaming camera data from Boston Dynamics Spot robots.

Features

  • Clean API: Simple, intuitive interface following Python best practices
  • Type-safe: Full type hints for better IDE support and type checking
  • Async Support: Both synchronous and async/await patterns supported
  • Context Managers: Automatic resource cleanup with with statements
  • YAML Configuration: Load settings from config files or pass as parameters
  • Multi-stream: Support for multiple concurrent camera streams
  • Thread-safe: Background streaming with thread-safe image buffering
  • Optional Vision Pipeline: ONNX Runtime inference can be enabled explicitly without making it a base dependency

Installation

Important: torch==2.8.0+cu129 is a CUDA wheel not hosted on PyPI. Running pip install -e . directly will fail. Use requirements.txt instead, it pins the correct PyTorch index URL automatically.

From source (recommended)

cd PySpotObserver
pip install -r requirements.txt

This installs the package in editable mode with all development dependencies and fetches the correct PyTorch CUDA wheel.

With optional vision pipeline support

pip install -e ".[vision]" --extra-index-url https://download.pytorch.org/whl/cu129

Quick Start

Basic Synchronous Usage

from pyspotobserver import SpotConfig, SpotConnection, CameraType

# Create configuration
config = SpotConfig(
    robot_ip="192.168.80.3",
    username="",
    password=""
)

# Connect and stream
with SpotConnection(config) as conn:
    stream = conn.create_cam_stream()

    # Start streaming from front cameras
    stream.start_streaming(CameraType.FRONTLEFT | CameraType.FRONTRIGHT)

    # Get images
    rgb_images, depth_images, body_T_worlds = stream.get_current_images()
    # body_T_worlds follows SDK camera order; virtual cameras are omitted.

    # Process images...

    stream.stop_streaming()

Async Usage

import asyncio
from pyspotobserver import SpotConfig, SpotConnection, CameraType

async def main():
    config = SpotConfig(robot_ip="192.168.80.3")

    async with SpotConnection(config) as conn:
        stream = conn.create_cam_stream()
        stream.start_streaming(CameraType.FRONTLEFT)

        # Async image retrieval
        rgb_images, depth_images, body_T_worlds = await stream.async_get_current_images()
        # body_T_worlds follows SDK camera order; virtual cameras are omitted.

        stream.stop_streaming()

asyncio.run(main())

Using YAML Configuration

from pyspotobserver import SpotConfig, SpotConnection

# Load from YAML file
config = SpotConfig.from_yaml("config.yaml")

with SpotConnection(config) as conn:
    # ...

Example config.yaml:

robot_ip: "192.168.80.3"
username: ""
password: ""
image_buffer_size: 5
image_quality_percent: 100.0
request_timeout_seconds: 10.0

Architecture

SpotConnection

Manages the robot connection lifecycle and authentication:

  • connect() / async_connect(): Establish connection
  • disconnect() / async_disconnect(): Clean shutdown
  • create_cam_stream(): Create a new camera stream
  • remove_cam_stream(): Remove an existing stream

SpotCamStream

Handles camera streaming in a background thread:

  • start_streaming(camera_mask): Begin streaming from specified cameras
  • stop_streaming(): Stop the stream
  • get_current_images() / async_get_current_images(): Retrieve latest frame; RGB/depth are in camera order, while body-to-world transforms are in SDK camera order with virtual cameras omitted
  • get_camera_order(): Get list of cameras being streamed

CameraType

Enum for specifying cameras (use bitwise OR to combine):

  • CameraType.BACK
  • CameraType.FRONTLEFT
  • CameraType.FRONTRIGHT
  • CameraType.LEFT
  • CameraType.RIGHT
  • CameraType.HAND

Image Format

Images are returned as NumPy arrays:

  • RGB: (H, W, 3) float32 in range [0, 1]
  • Depth: (H, W) float32 in meters

Examples

See the examples/ directory for complete working examples:

  • basic_streaming.py: Unified streaming example with sync, async, single-stream, multi-stream, and timing flags
  • config_example.yaml: Example configuration file
  • benchmark_allocation.py: Allocation vs in-place conversion benchmark

Design Principles

This implementation follows modern Python best practices:

  1. Type Safety: Full type hints using Python 3.9+ syntax
  2. Dataclasses: Configuration uses @dataclass for clean structure
  3. Context Managers: Automatic cleanup with with statements
  4. Async Support: Async variants for non-blocking operations
  5. CPU-Only: NumPy arrays for simplicity and portability (no CUDA)
  6. FIFO Buffering: Simple Queue-based buffering (vs. LIFO circular buffer in C++)
  7. Logging: Structured logging throughout for debugging
  8. Error Handling: Custom exception types for different failure modes

Differences from C++ Version

The Python implementation differs in these ways:

  • No CUDA: Uses CPU-only NumPy arrays instead of GPU memory
  • FIFO Queue: Simpler queue.Queue instead of custom circular buffer
  • Threading: Python threading instead of C++ jthread
  • Optional ML Pipeline: Inference is available through an optional ONNX Runtime extra
  • Simplified: Removes Unity plugin and DLL export complexity

Requirements

  • Python 3.9+
  • Boston Dynamics Spot SDK (bosdyn-client)
  • NumPy
  • OpenCV (opencv-python)
  • PyYAML
  • ONNX Runtime GPU (onnxruntime-gpu) only when installing the vision extra

Contributing

When contributing, please follow these guidelines:

  1. Use ruff for code formatting
  2. Use mypy for type checking
  3. Add tests for new features
  4. Update documentation

License

See LICENSE file for details.

Related Projects