Skip to content

Commit 594ac14

Browse files
Add webhook configuration support for collector
Add environment variable configuration for forwarding events to external HTTP endpoints. Supports separate webhooks for advertisements, channel messages, and direct messages with configurable timeouts, retries, and authentication via X-Webhook-Secret header. - Add webhook fields to CollectorSettings (config.py) - Add create_webhooks_from_settings() to build webhooks from env vars - Integrate WebhookDispatcher into Subscriber with background processing - Update collector CLI to load and display webhook configuration - Document webhook settings in README, AGENTS.md, .env.example - Update docker-compose.yml.example with webhook env vars 🤖 Generated with [Claude Code](https://claude.com/claude-code) Co-Authored-By: Claude <noreply@anthropic.com>
1 parent 0cbd01f commit 594ac14

8 files changed

Lines changed: 335 additions & 3 deletions

File tree

.env.example

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -74,3 +74,29 @@ NETWORK_CONTACT_DISCORD=
7474

7575
# Path to members JSON file (mounted into container)
7676
MEMBERS_FILE_PATH=./data/members.json
77+
78+
# ===================
79+
# Webhook Settings
80+
# ===================
81+
82+
# Webhook for advertisement events (node discovery)
83+
# Events are sent as POST requests with JSON payload
84+
WEBHOOK_ADVERTISEMENT_URL=
85+
WEBHOOK_ADVERTISEMENT_SECRET=
86+
87+
# Webhook for all message events (channel and direct messages)
88+
# Use this for a single endpoint handling all messages
89+
WEBHOOK_MESSAGE_URL=
90+
WEBHOOK_MESSAGE_SECRET=
91+
92+
# Optional: Separate URLs for channel vs direct messages
93+
# These override WEBHOOK_MESSAGE_URL if set
94+
# WEBHOOK_CHANNEL_MESSAGE_URL=
95+
# WEBHOOK_CHANNEL_MESSAGE_SECRET=
96+
# WEBHOOK_DIRECT_MESSAGE_URL=
97+
# WEBHOOK_DIRECT_MESSAGE_SECRET=
98+
99+
# Webhook behavior settings
100+
WEBHOOK_TIMEOUT=10.0
101+
WEBHOOK_MAX_RETRIES=3
102+
WEBHOOK_RETRY_BACKOFF=2.0

AGENTS.md

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -429,6 +429,31 @@ Key variables:
429429
- `API_READ_KEY`, `API_ADMIN_KEY` - API authentication keys
430430
- `LOG_LEVEL` - Logging verbosity
431431

432+
### Webhook Configuration
433+
434+
The collector supports forwarding events to external HTTP endpoints:
435+
436+
| Variable | Description |
437+
|----------|-------------|
438+
| `WEBHOOK_ADVERTISEMENT_URL` | Webhook for node advertisement events |
439+
| `WEBHOOK_ADVERTISEMENT_SECRET` | Secret sent as `X-Webhook-Secret` header |
440+
| `WEBHOOK_MESSAGE_URL` | Webhook for all message events (channel + direct) |
441+
| `WEBHOOK_MESSAGE_SECRET` | Secret for message webhook |
442+
| `WEBHOOK_CHANNEL_MESSAGE_URL` | Override for channel messages only |
443+
| `WEBHOOK_DIRECT_MESSAGE_URL` | Override for direct messages only |
444+
| `WEBHOOK_TIMEOUT` | Request timeout (default: 10.0s) |
445+
| `WEBHOOK_MAX_RETRIES` | Max retries on failure (default: 3) |
446+
| `WEBHOOK_RETRY_BACKOFF` | Exponential backoff multiplier (default: 2.0) |
447+
448+
Webhook payload structure:
449+
```json
450+
{
451+
"event_type": "advertisement",
452+
"public_key": "abc123...",
453+
"payload": { ... }
454+
}
455+
```
456+
432457
## Troubleshooting
433458

434459
### Common Issues

README.md

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -172,6 +172,31 @@ All components are configured via environment variables. Create a `.env` file or
172172
|----------|---------|-------------|
173173
| `DATABASE_URL` | `sqlite:///./meshcore.db` | SQLAlchemy database URL |
174174

175+
#### Webhook Configuration
176+
177+
The collector can forward events to external HTTP endpoints:
178+
179+
| Variable | Default | Description |
180+
|----------|---------|-------------|
181+
| `WEBHOOK_ADVERTISEMENT_URL` | *(none)* | Webhook URL for advertisement events |
182+
| `WEBHOOK_ADVERTISEMENT_SECRET` | *(none)* | Secret sent as `X-Webhook-Secret` header |
183+
| `WEBHOOK_MESSAGE_URL` | *(none)* | Webhook URL for all message events |
184+
| `WEBHOOK_MESSAGE_SECRET` | *(none)* | Secret for message webhook |
185+
| `WEBHOOK_CHANNEL_MESSAGE_URL` | *(none)* | Override URL for channel messages only |
186+
| `WEBHOOK_DIRECT_MESSAGE_URL` | *(none)* | Override URL for direct messages only |
187+
| `WEBHOOK_TIMEOUT` | `10.0` | Request timeout in seconds |
188+
| `WEBHOOK_MAX_RETRIES` | `3` | Max retry attempts on failure |
189+
| `WEBHOOK_RETRY_BACKOFF` | `2.0` | Exponential backoff multiplier |
190+
191+
Webhook payload format:
192+
```json
193+
{
194+
"event_type": "advertisement",
195+
"public_key": "abc123...",
196+
"payload": { ... event data ... }
197+
}
198+
```
199+
175200
### API Settings
176201

177202
| Variable | Default | Description |

docker-compose.yml.example

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -170,6 +170,18 @@ services:
170170
- MQTT_PASSWORD=${MQTT_PASSWORD:-}
171171
- MQTT_PREFIX=${MQTT_PREFIX:-meshcore}
172172
- DATABASE_URL=sqlite:////data/meshcore.db
173+
# Webhook configuration
174+
- WEBHOOK_ADVERTISEMENT_URL=${WEBHOOK_ADVERTISEMENT_URL:-}
175+
- WEBHOOK_ADVERTISEMENT_SECRET=${WEBHOOK_ADVERTISEMENT_SECRET:-}
176+
- WEBHOOK_MESSAGE_URL=${WEBHOOK_MESSAGE_URL:-}
177+
- WEBHOOK_MESSAGE_SECRET=${WEBHOOK_MESSAGE_SECRET:-}
178+
- WEBHOOK_CHANNEL_MESSAGE_URL=${WEBHOOK_CHANNEL_MESSAGE_URL:-}
179+
- WEBHOOK_CHANNEL_MESSAGE_SECRET=${WEBHOOK_CHANNEL_MESSAGE_SECRET:-}
180+
- WEBHOOK_DIRECT_MESSAGE_URL=${WEBHOOK_DIRECT_MESSAGE_URL:-}
181+
- WEBHOOK_DIRECT_MESSAGE_SECRET=${WEBHOOK_DIRECT_MESSAGE_SECRET:-}
182+
- WEBHOOK_TIMEOUT=${WEBHOOK_TIMEOUT:-10.0}
183+
- WEBHOOK_MAX_RETRIES=${WEBHOOK_MAX_RETRIES:-3}
184+
- WEBHOOK_RETRY_BACKOFF=${WEBHOOK_RETRY_BACKOFF:-2.0}
173185
command: ["collector"]
174186
healthcheck:
175187
test: ["CMD", "meshcore-hub", "health", "collector"]

src/meshcore_hub/collector/cli.py

Lines changed: 25 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -75,13 +75,37 @@ def collector(
7575
- Trace path data
7676
- Telemetry responses
7777
- Informational events (battery, status, etc.)
78+
79+
Webhooks can be configured via environment variables:
80+
- WEBHOOK_ADVERTISEMENT_URL: Webhook for advertisement events
81+
- WEBHOOK_MESSAGE_URL: Webhook for all message events
82+
- WEBHOOK_CHANNEL_MESSAGE_URL: Override for channel messages
83+
- WEBHOOK_DIRECT_MESSAGE_URL: Override for direct messages
7884
"""
7985
configure_logging(level=log_level)
8086

8187
click.echo("Starting MeshCore Collector")
8288
click.echo(f"MQTT: {mqtt_host}:{mqtt_port} (prefix: {prefix})")
8389
click.echo(f"Database: {database_url}")
8490

91+
# Load webhook configuration from settings
92+
from meshcore_hub.common.config import get_collector_settings
93+
from meshcore_hub.collector.webhook import (
94+
WebhookDispatcher,
95+
create_webhooks_from_settings,
96+
)
97+
98+
settings = get_collector_settings()
99+
webhooks = create_webhooks_from_settings(settings)
100+
webhook_dispatcher = WebhookDispatcher(webhooks) if webhooks else None
101+
102+
if webhook_dispatcher and webhook_dispatcher.webhooks:
103+
click.echo(f"Webhooks configured: {len(webhooks)}")
104+
for wh in webhooks:
105+
click.echo(f" - {wh.name}: {wh.url}")
106+
else:
107+
click.echo("Webhooks: None configured")
108+
85109
from meshcore_hub.collector.subscriber import run_collector
86110

87111
run_collector(
@@ -91,4 +115,5 @@ def collector(
91115
mqtt_password=mqtt_password,
92116
mqtt_prefix=prefix,
93117
database_url=database_url,
118+
webhook_dispatcher=webhook_dispatcher,
94119
)

src/meshcore_hub/collector/subscriber.py

Lines changed: 97 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5,18 +5,23 @@
55
2. Subscribes to all event topics
66
3. Routes events to appropriate handlers
77
4. Persists data to database
8+
5. Dispatches events to configured webhooks
89
"""
910

11+
import asyncio
1012
import logging
1113
import signal
1214
import threading
1315
import time
14-
from typing import Any, Callable, Optional
16+
from typing import Any, Callable, Optional, TYPE_CHECKING
1517

1618
from meshcore_hub.common.database import DatabaseManager
1719
from meshcore_hub.common.health import HealthReporter
1820
from meshcore_hub.common.mqtt import MQTTClient, MQTTConfig
1921

22+
if TYPE_CHECKING:
23+
from meshcore_hub.collector.webhook import WebhookDispatcher
24+
2025
logger = logging.getLogger(__name__)
2126

2227

@@ -31,21 +36,28 @@ def __init__(
3136
self,
3237
mqtt_client: MQTTClient,
3338
db_manager: DatabaseManager,
39+
webhook_dispatcher: Optional["WebhookDispatcher"] = None,
3440
):
3541
"""Initialize subscriber.
3642
3743
Args:
3844
mqtt_client: MQTT client instance
3945
db_manager: Database manager instance
46+
webhook_dispatcher: Optional webhook dispatcher for event forwarding
4047
"""
4148
self.mqtt = mqtt_client
4249
self.db = db_manager
50+
self._webhook_dispatcher = webhook_dispatcher
4351
self._running = False
4452
self._shutdown_event = threading.Event()
4553
self._handlers: dict[str, EventHandler] = {}
4654
self._mqtt_connected = False
4755
self._db_connected = False
4856
self._health_reporter: Optional[HealthReporter] = None
57+
# Webhook processing
58+
self._webhook_queue: list[tuple[str, dict[str, Any], str]] = []
59+
self._webhook_lock = threading.Lock()
60+
self._webhook_thread: Optional[threading.Thread] = None
4961

5062
@property
5163
def is_healthy(self) -> bool:
@@ -117,6 +129,78 @@ def _handle_mqtt_message(
117129
except Exception as e:
118130
logger.error(f"Error logging event {event_type}: {e}")
119131

132+
# Queue event for webhook dispatch
133+
if self._webhook_dispatcher and self._webhook_dispatcher.webhooks:
134+
self._queue_webhook_event(event_type, payload, public_key)
135+
136+
def _queue_webhook_event(
137+
self, event_type: str, payload: dict[str, Any], public_key: str
138+
) -> None:
139+
"""Queue an event for webhook dispatch.
140+
141+
Args:
142+
event_type: Event type name
143+
payload: Event payload
144+
public_key: Source node public key
145+
"""
146+
with self._webhook_lock:
147+
self._webhook_queue.append((event_type, payload, public_key))
148+
149+
def _start_webhook_processor(self) -> None:
150+
"""Start background thread for webhook processing."""
151+
if not self._webhook_dispatcher or not self._webhook_dispatcher.webhooks:
152+
return
153+
154+
# Capture dispatcher in local variable for closure (avoids Optional issues)
155+
dispatcher = self._webhook_dispatcher
156+
157+
def run_webhook_loop() -> None:
158+
"""Run async webhook dispatch in background thread."""
159+
loop = asyncio.new_event_loop()
160+
asyncio.set_event_loop(loop)
161+
162+
try:
163+
loop.run_until_complete(dispatcher.start())
164+
logger.info("Webhook processor started")
165+
166+
while self._running:
167+
# Get queued events
168+
events_to_process: list[tuple[str, dict[str, Any], str]] = []
169+
with self._webhook_lock:
170+
if self._webhook_queue:
171+
events_to_process = self._webhook_queue.copy()
172+
self._webhook_queue.clear()
173+
174+
# Process events
175+
for event_type, payload, public_key in events_to_process:
176+
try:
177+
loop.run_until_complete(
178+
dispatcher.dispatch(event_type, payload, public_key)
179+
)
180+
except Exception as e:
181+
logger.error(f"Webhook dispatch error: {e}")
182+
183+
# Small sleep to prevent busy-waiting
184+
time.sleep(0.01)
185+
186+
finally:
187+
loop.run_until_complete(dispatcher.stop())
188+
loop.close()
189+
logger.info("Webhook processor stopped")
190+
191+
self._webhook_thread = threading.Thread(
192+
target=run_webhook_loop, daemon=True, name="webhook-processor"
193+
)
194+
self._webhook_thread.start()
195+
196+
def _stop_webhook_processor(self) -> None:
197+
"""Stop the webhook processor thread."""
198+
if self._webhook_thread and self._webhook_thread.is_alive():
199+
# Thread will exit when self._running becomes False
200+
self._webhook_thread.join(timeout=5.0)
201+
if self._webhook_thread.is_alive():
202+
logger.warning("Webhook processor thread did not stop cleanly")
203+
120204
def start(self) -> None:
121205
"""Start the subscriber."""
122206
logger.info("Starting collector subscriber")
@@ -149,6 +233,9 @@ def start(self) -> None:
149233

150234
self._running = True
151235

236+
# Start webhook processor if configured
237+
self._start_webhook_processor()
238+
152239
# Start health reporter for Docker health checks
153240
self._health_reporter = HealthReporter(
154241
component="collector",
@@ -181,6 +268,9 @@ def stop(self) -> None:
181268
self._running = False
182269
self._shutdown_event.set()
183270

271+
# Stop webhook processor
272+
self._stop_webhook_processor()
273+
184274
# Stop health reporter
185275
if self._health_reporter:
186276
self._health_reporter.stop()
@@ -201,6 +291,7 @@ def create_subscriber(
201291
mqtt_password: Optional[str] = None,
202292
mqtt_prefix: str = "meshcore",
203293
database_url: str = "sqlite:///./meshcore.db",
294+
webhook_dispatcher: Optional["WebhookDispatcher"] = None,
204295
) -> Subscriber:
205296
"""Create a configured subscriber instance.
206297
@@ -211,6 +302,7 @@ def create_subscriber(
211302
mqtt_password: MQTT password
212303
mqtt_prefix: MQTT topic prefix
213304
database_url: Database connection URL
305+
webhook_dispatcher: Optional webhook dispatcher for event forwarding
214306
215307
Returns:
216308
Configured Subscriber instance
@@ -230,7 +322,7 @@ def create_subscriber(
230322
db_manager = DatabaseManager(database_url)
231323

232324
# Create subscriber
233-
subscriber = Subscriber(mqtt_client, db_manager)
325+
subscriber = Subscriber(mqtt_client, db_manager, webhook_dispatcher)
234326

235327
# Register handlers
236328
from meshcore_hub.collector.handlers import register_all_handlers
@@ -247,6 +339,7 @@ def run_collector(
247339
mqtt_password: Optional[str] = None,
248340
mqtt_prefix: str = "meshcore",
249341
database_url: str = "sqlite:///./meshcore.db",
342+
webhook_dispatcher: Optional["WebhookDispatcher"] = None,
250343
) -> None:
251344
"""Run the collector (blocking).
252345
@@ -257,6 +350,7 @@ def run_collector(
257350
mqtt_password: MQTT password
258351
mqtt_prefix: MQTT topic prefix
259352
database_url: Database connection URL
353+
webhook_dispatcher: Optional webhook dispatcher for event forwarding
260354
"""
261355
subscriber = create_subscriber(
262356
mqtt_host=mqtt_host,
@@ -265,6 +359,7 @@ def run_collector(
265359
mqtt_password=mqtt_password,
266360
mqtt_prefix=mqtt_prefix,
267361
database_url=database_url,
362+
webhook_dispatcher=webhook_dispatcher,
268363
)
269364

270365
# Set up signal handlers

0 commit comments

Comments
 (0)