Skip to content

Latest commit

 

History

History
498 lines (408 loc) · 19.4 KB

File metadata and controls

498 lines (408 loc) · 19.4 KB

MeshMonitor Architecture

This document provides a detailed overview of the MeshMonitor application architecture, design decisions, and implementation details.

Overview

MeshMonitor is a full-stack web application designed to monitor and interact with Meshtastic mesh networks. The application follows a three-tier architecture with a React frontend, Express.js backend, and flexible database backend supporting SQLite (default), PostgreSQL, and MySQL/MariaDB.

┌─────────────────┐    ┌─────────────────┐    ┌─────────────────────────┐
│   Browser       │    │   Express.js    │    │   Database (Drizzle)    │
│   (React SPA)   │◄──►│   (API Server)  │◄──►│   • SQLite (default)    │
└─────────────────┘    └─────────────────┘    │   • PostgreSQL          │
                                │             │   • MySQL/MariaDB       │
                                ▼             └─────────────────────────┘
                       ┌─────────────────┐
                       │  Meshtastic     │
                       │  Node (HTTP)    │
                       └─────────────────┘

Frontend Architecture

Technology Stack

  • React 18 with TypeScript for type safety
  • Vite as the build tool for fast development and optimized builds
  • CSS3 with Catppuccin Mocha theme for modern, dark UI
  • ES Modules for modern JavaScript features

Key Components

App.tsx - Main Application Component

The core application component manages:

  • Connection status to Meshtastic node
  • Node data fetching and state management
  • Message handling with iPhone Messages-style UI featuring threaded replies and emoji reactions
  • Channel management and filtering
  • Real-time data updates via polling

State Management Strategy

  • React useState hooks for local component state
  • Custom hooks for data fetching logic
  • Map-based state for pending message acknowledgments
  • Real-time polling for backend synchronization

UI/UX Features

  • iPhone Messages-style bubbles with proper alignment
  • Interactive reply and tapback system
    • Hover-based reply button on each message
    • Instant emoji reactions: 👍 👎 ❓ ❗ 😂 😢 💩
    • Reply context display in send box
    • Clickable existing reactions
    • Threaded conversation support
  • Sender identification dots with tooltips showing node names
  • Real-time delivery status (⏳ pending → ✓ delivered)
  • Optimistic UI updates for instant feedback
  • Responsive design for mobile and desktop

Data Flow

User Input → State Update → API Call → Backend Processing → Database → Response → UI Update
     ↓                                                                              ↑
Optimistic UI ←─────────────────────────────────────────────────────────────────────┘

Backend Architecture

Technology Stack

  • Node.js with Express.js framework
  • TypeScript for type safety and better developer experience
  • Drizzle ORM for type-safe, database-agnostic queries
  • Database Drivers: better-sqlite3 (SQLite), pg (PostgreSQL), mysql2 (MySQL)
  • CORS enabled for cross-origin requests

Core Services

MeshtasticManager

Location: src/server/meshtasticManager.ts

The central service for Meshtastic node communication:

  • HTTP API Client for Meshtastic node communication
  • Protobuf Message Parsing for binary data interpretation
  • Real-time Polling for continuous data synchronization
  • Channel Detection with whitelist-based filtering
  • Node Discovery and telemetry data extraction
  • Automatic Traceroute Scheduler running every 3 minutes
  • Route Path Discovery for network topology mapping

Key Methods:

connect(): Promise<boolean>                                           // Connect to Meshtastic node
sendTextMessage(text: string, channel: number, destination?: number,  // Send messages with optional
                replyId?: number, emoji?: number)                     // reply/tapback support
sendTraceroute(destination: string)                                   // Send traceroute request
getAllNodes(): DeviceInfo[]                                           // Get node information
getRecentMessages(limit: number)                                      // Get message history
startTracerouteScheduler()                                            // Start automatic traceroutes
isValidPosition(lat: number, lon: number)                             // Validate position coordinates

Position Validation:

  • All position coordinates are validated before database storage
  • Latitude must be between -90 and 90 degrees
  • Longitude must be between -180 and 180 degrees
  • Values must be valid numbers (not NaN or Infinity)
  • Invalid coordinates are rejected with warning logs
  • Applied in both position messages and node info packets

Channel Management:

  • Whitelist-based filtering prevents invalid channels
  • Known channels: Primary, admin, gauntlet, telemetry, Secondary, LongFast, VeryLong
  • Automatic cleanup of invalid channel entries

DatabaseService

Location: src/services/database.ts

Manages all database operations with support for multiple backends:

  • Multi-database support: SQLite (default), PostgreSQL, MySQL/MariaDB
  • Async-first design: All methods use async/await for database-agnostic operation
  • Drizzle ORM: Type-safe queries that work across all database backends
  • Repository pattern: Domain-specific repositories in src/db/repositories/
  • Node management (create, read, update, delete)
  • Message persistence with deduplication
  • Channel configuration storage
  • Data cleanup utilities
  • Export/import functionality

Database Selection:

# SQLite (default - no configuration needed)
DATABASE_PATH=/data/meshmonitor.db

# PostgreSQL
DATABASE_URL=postgres://user:password@host:5432/meshmonitor

# MySQL/MariaDB
DATABASE_URL=mysql://user:password@host:3306/meshmonitor

Database Schema:

-- Nodes table
CREATE TABLE nodes (
  nodeNum INTEGER PRIMARY KEY,
  nodeId TEXT UNIQUE NOT NULL,
  longName TEXT,
  shortName TEXT,
  hwModel INTEGER,
  role INTEGER,
  hopsAway INTEGER,
  lastHeard INTEGER,
  snr REAL,
  rssi INTEGER,
  isFavorite BOOLEAN DEFAULT 0,  -- Synced from Meshtastic NodeDB
  -- ... additional telemetry fields
);

-- Messages table
CREATE TABLE messages (
  id TEXT PRIMARY KEY,
  fromNodeNum INTEGER NOT NULL,
  toNodeNum INTEGER NOT NULL,
  fromNodeId TEXT NOT NULL,
  toNodeId TEXT NOT NULL,
  text TEXT NOT NULL,
  channel INTEGER NOT NULL DEFAULT 0,
  timestamp INTEGER NOT NULL,
  hopStart INTEGER,
  hopLimit INTEGER,
  replyId INTEGER,  -- Message ID being replied to
  emoji INTEGER,    -- 0=normal message, 1=tapback reaction
  -- ... FOREIGN KEY constraints
);

-- Channels table
CREATE TABLE channels (
  id INTEGER PRIMARY KEY,
  name TEXT,
  psk TEXT,
  uplinkEnabled BOOLEAN DEFAULT 1,
  downlinkEnabled BOOLEAN DEFAULT 1
);

-- Traceroutes table
CREATE TABLE traceroutes (
  id INTEGER PRIMARY KEY AUTOINCREMENT,
  fromNodeNum INTEGER NOT NULL,
  toNodeNum INTEGER NOT NULL,
  fromNodeId TEXT NOT NULL,
  toNodeId TEXT NOT NULL,
  route TEXT,
  routeBack TEXT,
  snrTowards TEXT,
  snrBack TEXT,
  timestamp INTEGER NOT NULL,
  createdAt INTEGER NOT NULL,
  FOREIGN KEY (fromNodeNum) REFERENCES nodes(nodeNum),
  FOREIGN KEY (toNodeNum) REFERENCES nodes(nodeNum)
);

API Design

RESTful Endpoints

Node Management:

  • GET /api/nodes - Retrieve all nodes
  • GET /api/nodes/active?days=7 - Get recently active nodes

Message Operations:

  • GET /api/messages?limit=100 - Get paginated messages
  • POST /api/messages/send - Send message to channel
  • GET /api/messages/channel/:channel - Channel-specific messages

Traceroute Operations:

  • GET /api/traceroutes/recent - Get recent traceroutes with route paths
  • POST /api/traceroutes/send - Send traceroute to specific node

System Management:

  • GET /api/health - Health check endpoint
  • GET /api/connection - Meshtastic connection status
  • POST /api/cleanup/channels - Clean invalid channels

Error Handling

  • Comprehensive try-catch blocks with logging
  • HTTP status codes for different error types
  • Graceful degradation when Meshtastic node is unreachable
  • Database constraint handling

Data Layer

Multi-Database Architecture

MeshMonitor uses Drizzle ORM to provide a consistent API across SQLite, PostgreSQL, and MySQL:

┌─────────────────────────────────────────────────────────────┐
│                    DatabaseService                          │
│              (async facade, caching layer)                  │
└─────────────────────────────────────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────┐
│                     Repositories                             │
│     (nodes, messages, telemetry, auth, settings, etc.)      │
└─────────────────────────────────────────────────────────────┘
                              │
                              ▼
┌─────────────────────────────────────────────────────────────┐
│                      Drizzle ORM                             │
│            (type-safe, database-agnostic queries)           │
└─────────────────────────────────────────────────────────────┘
          │                   │                   │
          ▼                   ▼                   ▼
    ┌──────────┐       ┌──────────┐       ┌──────────┐
    │  SQLite  │       │ PostgreSQL│       │  MySQL   │
    │(default) │       │(enterprise)│       │(optional)│
    └──────────┘       └──────────┘       └──────────┘

Key Files:

  • src/services/database.ts - Main service facade
  • src/db/schema/ - Drizzle schema definitions
  • src/db/repositories/ - Domain-specific data access
  • src/db/drivers/ - Database-specific connection handling

Performance Optimizations

  • Indexes on frequently queried columns (nodeId, timestamp, channel)
  • WAL mode for SQLite concurrent access
  • Connection pooling for PostgreSQL/MySQL
  • Foreign key constraints for data integrity
  • Prepared statements for SQL injection prevention

Data Integrity

  • UPSERT operations for node data to handle duplicates
  • Timestamp-based sorting for chronological message ordering
  • Node relationship validation via foreign keys
  • Channel name normalization and validation
  • BIGINT handling for large node IDs across databases

Meshtastic Integration

Protocol Handling

HTTP API Communication

  • RESTful HTTP calls to Meshtastic node
  • Binary protobuf parsing for message decoding
  • Automatic reconnection on connection failures
  • Configuration requests for device setup

Protobuf Message Processing

// Simplified protobuf parsing flow
Raw Binary Data  Field Extraction  Type Detection  Data Processing  Database Storage

Protobuf Definitions:

  • Official Meshtastic protobufs integrated as git submodule
  • Pinned to specific release versions (e.g., v2.7.11)
  • Located at protobufs/ directory (submodule)
  • Automatically stays in sync with official Meshtastic protocol definitions
  • Update via git submodule update --remote and checkout desired tag

Message Types Handled:

  • Node Information (device details, telemetry, hardware model)
  • Text Messages (user communications with reply threading and emoji tapbacks)
    • Threaded replies via replyId field (Meshtastic protobuf field 7)
    • Instant emoji reactions via emoji flag (Meshtastic protobuf field 8)
    • Supports: 👍 👎 ❓ ❗ 😂 😢 💩
  • Channel Configuration (network settings)
  • Telemetry Data (battery, GPS, signal strength)
  • Position Telemetry (latitude/longitude tracking for mobile detection)

Channel Management Strategy

  • Whitelist approach for known Meshtastic channels
  • Automatic filtering of WiFi SSIDs and random strings
  • Dynamic channel discovery from legitimate sources
  • Channel cleanup utilities for maintenance

Security Considerations

Backend Security

  • Input validation on all API endpoints
  • SQL injection prevention via prepared statements
  • CORS configuration for controlled access
  • Environment-based configuration for sensitive data

Frontend Security

  • TypeScript for compile-time error prevention
  • Output sanitization for user-generated content
  • Secure HTTP connections support via TLS configuration

Deployment Architecture

Docker Containerization

# Multi-stage build for optimized production image
FROM node:20-alpine AS builder
# Build both frontend and backend
RUN npm run build && npm run build:server

FROM node:20-alpine AS production
# Copy only built assets and production dependencies

Production Configuration

  • Volume mounting for persistent database storage
  • Environment variables for configuration management
  • Health checks for container orchestration
  • Graceful shutdown handling for data integrity

Performance Considerations

Frontend Optimizations

  • React.memo for component optimization
  • Efficient re-renders via proper state management
  • Optimistic updates for perceived performance
  • Lazy loading for large node lists

Backend Optimizations

  • Connection pooling for database operations
  • Efficient polling with configurable intervals
  • Memory management for protobuf parsing
  • Caching strategies for frequently accessed data

Database Optimizations

  • Query optimization with proper indexing
  • Batch operations for bulk data processing
  • Regular cleanup of old data
  • Database maintenance utilities

Monitoring and Observability

Logging Strategy

  • Structured logging with different levels
  • Request/response logging for API calls
  • Error tracking with stack traces
  • Performance metrics for optimization

Health Monitoring

  • Connection status monitoring
  • Database health checks
  • Memory and CPU usage tracking
  • Error rate monitoring

Development Workflow

Build Process

# Development
npm run dev:full    # Start both frontend and backend in dev mode

# Production
npm run build       # Build frontend assets
npm run build:server # Compile TypeScript backend
npm start          # Start production server

Code Quality

  • TypeScript for type safety
  • ESLint for code quality
  • Consistent formatting standards
  • Git hooks for pre-commit validation

Mobile Node Detection

Position Tracking and Movement Analysis

MeshMonitor automatically detects mobile nodes based on position telemetry:

Detection Logic:

  • Position data (latitude/longitude) is saved to the telemetry table
  • Historical position data is analyzed for variance
  • Haversine formula calculates distance between positions
  • Nodes with >1km total movement are marked as isMobile: true
  • Mobile status is included in API responses and node display

Benefits:

  • Identify mobile nodes (vehicles, hikers, etc.) vs. stationary nodes
  • Optimize network topology understanding
  • Enable mobile-specific features and visualizations

Traceroute Scheduler

Automatic Network Discovery

The MeshMonitor application includes an intelligent traceroute scheduler that automatically discovers network topology:

Scheduling Logic:

  • Runs every 3 minutes when connected to a Meshtastic node
  • Selects nodes that either have no traceroute data or oldest traceroute
  • Automatically sends traceroute requests to discover routes
  • Stores complete route paths with SNR data for each hop

Data Collection:

  • Route paths (both forward and return routes)
  • SNR values for each hop in the path
  • Timestamp for traceroute completion
  • Node-to-node relationship mapping

Filtering:

  • Traceroute messages are filtered from Primary channel display
  • Prevents clutter while maintaining network discovery
  • Data stored in dedicated traceroutes table

Map Visualization

Route Display Features

Interactive Route Mapping:

  • "Show Routes" checkbox to toggle route display
  • Weighted polylines showing route segments
  • Thickness varies from 2-8px based on segment usage
  • Routes that appear in multiple traceroutes are drawn thicker
  • Purple color scheme matching Catppuccin theme

Node Popup Information:

  • iPhone Messages-style popup with route-popup styling
  • Displays readable node role (e.g., "Router", "Client")
  • Shows formatted hardware model name (e.g., "Station G2", "Heltec Wireless Paper")
  • Hardware names converted from technical format (STATION_G2) to readable format
  • Auto-opens when selecting node from node list
  • Includes SNR, battery level, and last heard timestamp

Segment Weight Calculation:

  • Base weight: 2px
  • Additional weight: +1px per usage occurrence
  • Maximum weight: 8px for heavily used routes
  • Bidirectional segments counted once (normalized)

Hardware Model Display:

  • 116 Meshtastic hardware models supported (models 0-114 plus 255)
  • Technical names (e.g., STATION_G2) formatted for readability
  • Brand name capitalization (Heltec, Lilygo, BetaFPV, NomadStar, etc.)
  • Version number formatting (V2P0 → V2.0)
  • Abbreviations preserved uppercase (LR, TX, RAK, etc.)

Future Enhancements

Planned Improvements

  1. Enhanced Telemetry Parsing - More sophisticated protobuf decoding
  2. Real-time WebSocket Updates - Replace polling with WebSocket connections
  3. Advanced Channel Management - Custom channel creation and management
  4. Message Search - Full-text search across message history
  5. Data Visualization - Charts and graphs for network analytics
  6. Mobile App - React Native companion application
  7. Advanced Traceroute Analytics - Network path optimization and analysis

Scalability Considerations

  • Database flexibility - PostgreSQL/MySQL support already implemented for larger deployments
  • Horizontal scaling with load balancers
  • Caching layer with Redis for improved performance
  • Microservices architecture for complex deployments

This architecture supports MeshMonitor's current capabilities while providing a foundation for future enhancements and scalability improvements.