feat(uas): MAVLink connection + 3-backend video receiver#45
Open
jfuginay wants to merge 9 commits into
Open
Conversation
First chunk of iOS UAS parity with the Android sibling (OmniTAK-Android #56, #62). Closes the receiver-side gap surfaced by Foxbat in the TAK Discord RubyFPV thread. What ships: - VideoSource sealed enum (none / rtsp / rawH264Udp / mpegTsUdp), mirroring the Kotlin sealed class - DroneState struct + MAVAutopilot/MAVType enums - MAVLinkCodec — hand-rolled MAVLink 2 frame parser with the 8 messages we need (HEARTBEAT, GLOBAL_POSITION_INT, ATTITUDE, GPS_RAW_INT, SYS_STATUS, BATTERY_STATUS, COMMAND_LONG, COMMAND_ACK). CRC-16/MCRF4XX + per-message CRC_EXTRA byte. Pure Swift, no dependencies; the survey turned up no actively-maintained Swift MAVLink SPM package (MAVSDK-Swift is a heavy gRPC wrapper, only alternative is modnovolyk/MAVLinkSwift which is Swift 3 / v1 only) - MAVLinkConnection — NWConnection UDP transport, 1 Hz GCS heartbeat, rolling RX buffer + frame extraction - UASManager — @mainactor ObservableObject, owns the link + publishes DroneState; sendCommandLong wrappers for arm / takeoff / RTL - UASVideoPipView — backed by MobileVLCKit (libVLC handles RTSP + raw H264 over UDP + MPEG-TS over UDP via url-scheme dispatch). Graceful fallback view when MobileVLCKit isn't linked, matching the existing VLCPlayerView pattern - UASConnectView — SwiftUI form: host/port/callsign for MAVLink + segmented Video Source picker with dynamic field per mode - Toolbar entry: BarItem "tool.uas" (Vehicles, airplane.departure icon) wired through ToolSheetHost to UASConnectView Not in this PR (separate follow-up): - MapView integration (drone MKAnnotation, HUD card, gated UAS PIP) - arm/disarm/takeoff/RTL buttons - CoT broadcast (telemetry → UASCoTGenerator → TAKService.sendCoT) - MobileVLCKit SPM dependency (today's #if canImport renders fallback) xcodebuild clean (iOS Simulator, Debug, SDK 26.5). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Wires tylerjonesio/vlckit-spm 3.6.0 as a Swift Package Manager dependency on the OmniTAKMobile target. Exposes the upstream MobileVLCKit module (LGPL-2.1+) so the `#if canImport(MobileVLCKit)` branches in VLCPlayerView and UASVideoPipView now compile the real playback path instead of the "Add MobileVLCKit" fallback. Why this wrapper vs. upstream: VideoLAN doesn't publish a first-party SPM package (their issue #302 is still open). Tyler Jones' wrapper ships the official VideoLAN XCFramework with a Package.swift on top and tracks the 3.x line — `import MobileVLCKit` stays as the module name, so existing call sites are unchanged. Build verified clean on iOS Simulator (Debug). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
…pass gyb_detect ESP32 drone detector integration (BLE GATT) + a performance and structural cleanup of the iOS client. gyb detector: - BLE GATT client/parser/manager + in-app discover/connect/status UI - Detections feed the shared RemoteIdTrackStore (dedup with on-device RID) - Drones render as live CoT air contacts (MIL-STD symbol, 3D altitude leader line), federate to servers, but are gated out of the chat participant list (not messageable EUDs) Performance (main-thread invalidation storm on TAKService.shared): - Move CoT regex parse off the main thread (serial queue; ordering kept) - Remove dead @published lastMessage (full XML stored per packet, 0 readers) - Coalesce messagesReceived/bytesReceived publishing to <=4x/sec so the map body no longer re-renders per packet - Dedup cotEvents in the Rust-FFI callback path (was appending unbounded) - Hash-guard Cesium setEntities/setTrails bridge calls (were re-encoding + re-crossing the WKWebView boundary every frame) - Move SettingsView cache-size walk off the main thread UI routing: - Wire previously-dead radial-menu actions: centerMap + draw line/circle/polygon (via a ViewModifier to dodge the type-checker ceiling on ATAKMapView.body), and route createRoute/quickChat/ emergency/getInfo to real screens Dead code: - Delete orphaned MapKit stack (Map3DViewController + 6 Map/Overlays MKOverlay files, superseded by Mapbox+Cesium) and unused ConnectionCoordinator (~3,700 lines) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The drone altitude leader line mutated a LIVE Cesium entity's `.polyline` graphics in place — reassigning `new Cesium.PolylineGraphics(...)` and `= undefined` on each position update. That's the only place in the bridge that mutated graphics on a live entity (every other polyline — drawings, measurements, trails — is a standalone add/remove entity), and it crashed the WebGL globe when a drone appeared. Render the leader as its own `uid:leader` entity in a dedicated `_state.leaders` map, remove-then-add on every upsert (matching trails), and clean it up in removeEntity/removeAll. Kept out of `_state.entities` so setEntities' stale-cleanup doesn't touch it. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The Cesium globe pulls ~5MB of Cesium.js + terrain from the cesium.com CDN on every launch. When the device can't reach it (bad network, dead CDN), the page sat on "Loading 3D world…" forever with no escape. Add a 12s load watchdog in the CesiumMainMap coordinator: if the bridge hasn't signalled omniBridgeReady by then, post .cesiumLoadTimedOut. ATAKMapView (via the RadialMenuExtraObservers modifier, to stay off the already-maxed body type-checker) switches the engine to 2D and shows a brief "switched to 2D" banner. Cancelled the instant the globe loads, so it never fires on a healthy load. Validated in the simulator (which can't run Cesium's WebGL): instead of hanging, it now falls back to the 2D map after 12s. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
ROOT CAUSE of the "Loading 3D world…" hang (sim AND device): the leader
label built its HAE string with `'\n'` written as a single backslash
inside the Cesium HTML, which is a Swift `"""` multiline literal. Swift
turned `\n` into a REAL newline, producing an unterminated JS string
literal — a SyntaxError ("Unexpected EOF") that aborted the ENTIRE inline
Cesium <script>. The viewer never initialized, so the globe spun forever.
Confirmed on-device via a temporary JS-error bridge:
JSERR: SyntaxError: Unexpected EOF @:138 (the labelText line)
typeof Cesium=object (CDN load was fine all along)
viewer=no, ready=n/a (main script never ran)
Fix: write `\\n` so Swift emits a literal `\n` JS newline escape.
node --check now parses the whole inline script clean.
This was never a network/CDN issue or related to the dead-code cleanup —
both were ruled out. The watchdog→2D fallback (prev commit) stays as a
safety net for genuine load failures.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Symptom: the 3D globe worked for a while then went black on device, and only a manual 2D↔3D toggle brought it back. That toggle recreates the WKWebView — confirming the WebContent (WebGL) process had been terminated by iOS under memory pressure (Cesium is GPU-heavy on mobile), leaving a dead black view with no automatic recovery. Set the WKWebView's navigationDelegate to the coordinator and implement webViewWebContentProcessDidTerminate: reset isReady, clear the bridge payload-hash cache (so the fresh page gets a full re-push), reload the Cesium HTML, and restart the load watchdog. omniBridgeReady re-fires on reload and updateUIView re-pushes the entity/drawing/trail snapshots, so the scene restores itself without user intervention. If the reload can't complete, the watchdog still falls back to 2D. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The globe was stacking three heavy network layers on every launch —
World Terrain + World Imagery + Google Photorealistic 3D Tiles. The
photoreal tileset is the heavy hitter: slow to stream/sharpen and a
GPU-memory hog that contributed to the WebGL process being killed
(the "worked then went black" + "takes forever to sharpen" reports).
Now the base is selectable from the Layers panel instead of all-on:
- Default = World Imagery + terrain (fast, sharp, stable).
- "Photoreal 3D" is a new Layers option (globe only), loaded on demand
via setBaseLayer('photoreal') and unloaded when another base is picked.
- When it does load, maximumScreenSpaceError=24 (coarser than the
default 16) so it sharpens faster and uses less GPU.
setBaseLayer now manages the tileset lifecycle; _state.photoreal tracks
it. The always-on createGooglePhotorealistic3DTileset at viewer init is
removed.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Bumps from 2.32.0 (which was never archived — it shipped the globe-load syntax bug). This is the first working build of the gyb-detector + perf/structure release, plus the new selectable globe base layers (photoreal opt-in). Build 26053002 (2026-05-30 #2). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Tracks OmniTAK-iOS#40. First chunk of iOS UAS parity with Android sibling (engindearing-projects/OmniTAK-Android#56, #62).
What ships
Receiver-side parity with Android's UAS Video PIP. iOS now has the full UAS connection + telemetry foundation plus a live video pipe through libVLC.
.none / .rtsp / .rawH264Udp / .mpegTsUdp), mirroring the Kotlin sealed class withvlcURLmapping each active source to a libVLC-compatible URL@MainActor ObservableObject, publishes DroneState, exposesarm/disarm/takeoff/RTLvia COMMAND_LONGMobileVLCKit(libVLC's url-scheme dispatch handles all three transports: RTSP demuxer, raw H264 via--demux=h264, MPEG-TS native)tool.uas, airplane.departure icon, orange tint) routed through ToolSheetHost to UASConnectViewPackage.swifton top and keeps theMobileVLCKitmodule name so existing call sites stay drop-in. LGPL-2.1+, App Store compatible with attributionWhy hand-roll MAVLink
Surveyed the Swift MAVLink package landscape: zero actively-maintained, MAVLink-2-capable SPM packages.
mavlink/MAVSDK-Swift— heavy gRPC wrapper around MAVSDK C++, last commit 2023-10. Wrong shape for an in-process codec.modnovolyk/MAVLinkSwift— only pure-Swift SPM codec that ever shipped, Swift 3 / MAVLink v1 only, last commit 2018-03.ArduPilot/pymavlink's Swift generator — alive (2025 commits) but produces vendored code, not a published package.Given the survey result and the small message subset we need today, hand-rolling kept the dependency footprint at zero and the codec at ~370 lines of readable Swift.
What's NOT in this PR
MKAnnotation, HUD card)UASManager.videoSourcebut the map doesn't render the PIP yetUASManagerhas the commands; UI needs to land alongside the map HUDUASCoTGenerator→TAKService.sendCoT)MARKETING_VERSION/CURRENT_PROJECT_VERSIONbumpBuild
xcodebuild -project OmniTAKMobile.xcodeproj -scheme OmniTAKMobile -sdk iphonesimulator -destination 'generic/platform=iOS Simulator' -configuration Debug build— BUILD SUCCEEDED with VLCKit linkedNavigationView+ 1-argonChange(of:_:))To validate locally
127.0.0.1for SITL on same Mac), port14550, callsignpython3 fake_drone.py(from/tmp/omnitak-verify/)ffmpeg -re -f lavfi -i testsrc -c:v libx264 -profile:v baseline -bsf:v h264_mp4toannexb -an -f h264 'udp://127.0.0.1:5000?pkt_size=1024'... -f mpegts 'udp://127.0.0.1:5010?pkt_size=1316'Once PR-B lands, UAS PIP will appear bottom-right on the map gated on
state.isConnected().🤖 Generated with Claude Code