Skip to content

Commit 334c851

Browse files
Tony Liuclaude
andcommitted
feat(ha): Home Assistant integration v1 — wall power, efficiency, rack env
DCGM measures GPU power; node_exporter has none of the rest. The blind spots — real socket-side W per host, tokens/W efficiency joined against LiteLLM throughput, and rack temperature/humidity/AC state — get filled by a tiny Python exporter that polls Home Assistant REST every 15 s. The exporter exposes ha_* Prometheus metrics on :9105, obs Prometheus scrapes them like node_exporter, recording rules fold them into cluster:wall_power_w:sum / cluster:tokens_per_watt, and the FastAPI cluster payload picks them up via _ha_power / _ha_env helpers. Three new tm-cards in the Telemetry section render the values; if the exporter is absent the cards self-hide and nothing else changes. Hearth-OSS friendly: NO node ids or plug IDs are hardcoded. Operators declare the topology in hearth.yaml: ha: base_url: "http://homeassistant.local:8123" rack_sensor_id: "..." rack_ac_plug_id: "..." blocklist: ["2051674991"] # never monitor router plug nodes: - id: spark-01 sources: ha_plug_id: "2029950736" Hard safety: the MT6000 router id is in a default blocklist that the exporter cannot bypass — if any node points at a blocklisted plug, the exporter refuses to start. The exporter only does GET /api/states/*; control endpoints stay explicitly out of v1 scope. Honest degradation: - HA exporter absent → /api/cluster.power / .env return null → the three tm-cards in Telemetry are hidden entirely. - Stale sensor → that field null; siblings still render. - Node without ha_plug_id (e.g. Atlas with no smart plug) → its wall- power slot is "—", not "0 W". Files: - server/integration/ha-exporter.py (NEW, config-driven, ~180 lines) - server/deploy/ha-exporter.service (NEW, systemd unit) - server/deploy/install-ha-exporter.sh (NEW, install + smoke test) - config/hearth.example.yaml (docs new ha: block) - docs/topology.md (new ha: section) - docs/requirements/ha-integration.md (NEW, full requirements + spec handoffs) - docs/requirements/01..05-*.md (NEW, prefixed split for workflow tooling) - server/api/main.py (+_ha_power, _ha_env; /api/cluster adds power, env) - server/prometheus/prometheus.yml (new home_assistant scrape job) - server/prometheus/rules/recording.yml (cluster:wall_power_w:sum, tokens_per_watt, node:power_overhead_ratio) - server/prometheus/rules/alerts.yml (HAExporterDown, RackOverTemp, RackACOff, NodePowerLost, RackSensorLowBattery) - sections-b.jsx (TelemetrySection adds 3 tm-cards) - data.js (mirrors p.cluster.power / .env into live.*) - styles.css (.tm-card) - index.html (cache bump to 20260601a) - CHANGELOG.md (Added entry) - .gitignore (ignore .claude/ workflow hooks) NOT YET done in this commit (left to operator + a follow-up session): - T4 systemd deploy on Atlas (run install-ha-exporter.sh) - T5-T7 Prometheus reload + verify scrape job comes up - T11 Playwright end-to-end UI verification - T12 HA-offline degradation smoke test - T13 verify-no-prod-impact + DeepSeek inference 200 Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
1 parent fe89423 commit 334c851

21 files changed

Lines changed: 994 additions & 8 deletions

.gitignore

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -45,3 +45,4 @@ BASELINES.md
4545
.specify/
4646
config/hearth.yaml
4747
_local/
48+
.claude/

CHANGELOG.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
99

1010
### Added
1111

12+
- **Home Assistant integration (optional) — wall power + rack environment.** Fills the gap DCGM (GPU only) and node_exporter (no socket-side W) can't: real per-host wall power from HA smart plugs, `tokens/W` efficiency (LiteLLM throughput / wall power), and rack temperature/humidity/AC status — surfaced in the existing Telemetry section as three terse `tm-card`s. Architecture: a tiny Python exporter (`server/integration/ha-exporter.py`) polls HA REST every 15 s and exposes Prometheus metrics on `:9105`; obs Prometheus scrapes it the same way it scrapes node_exporter. New `ha:` block in `hearth.yaml` plus per-node `sources.ha_plug_id` declare the topology — operators map plugs to nodes in config, not in code. Hard blocklist (MT6000 router by default) is enforced at exporter startup. Cards self-hide cleanly when no HA is configured, so the integration is invisible to users who don't need it. New `/api/config` field surface unchanged; `/api/cluster` gains `power` and `env` sub-objects (every field `null` when stale). Docs: [`docs/requirements/ha-integration.md`](docs/requirements/ha-integration.md), [`docs/topology.md`](docs/topology.md) `ha:` section.
1213
- **`display.timezone` — pin all timestamps to a specific IANA zone.** Hearth has always rendered the nav clock and request-log times in the browser's own locale, which is fine until the same dashboard is viewed from a server-side browser, a remote support seat in another country, or a Pi whose TZ was never set. Add optional `display.timezone: "Asia/Taipei"` (or any IANA zone) to `hearth.yaml` and the frontend formats every timestamp in that zone regardless of viewer locale. Omit it and Hearth keeps the existing browser-local default. New `/api/config` endpoint exposes the value; data.js fetches it once at boot and routes the two render sites (nav clock, request log) through a `formatTime/formatDate` helper. Backend stays TZ-agnostic — SQL still emits ISO-8601 `Z`.
1314
- **Multi-node (tensor/pipeline parallel) topology** — new optional `model_topology` config block. A TP=N deployment (e.g. vLLM `--tensor-parallel-size 4` over Ray across several hosts) exposes only the head node's endpoint to the gateway, so auto-discovery attributed the model — and its GPU-activity rings — to the head alone, making the worker nodes look idle while they were running TP shards at full GPU. Declaring `model_topology: {<model>: {nodes: [...], parallelism: "TP=4"}}` attributes the model and its derived activity to every node in the group. Docs: [`docs/topology.md`](docs/topology.md).
1415
- **SGLang adapter** (`_scrape_sglang`, `sglang:*` metrics) — tps / TTFT / TPOT / KV% / p50-99 / running / waiting, the full rich set (SGLang exposes e2e + TTFT histograms like vLLM). Frontend renders SGLang models exactly like vLLM. ⚠️ Implemented from SGLang's documented metric names but **not yet verified against a live SGLang instance** (dev cluster runs vLLM + llama.cpp only) — please open an issue if metric names differ in your version.

config/hearth.example.yaml

Lines changed: 22 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -187,3 +187,25 @@ alerts:
187187
# ── Generic JSON POST (escape hatch — wire to anything) ───────────
188188
# - type: webhook
189189
# url_env: "MY_WEBHOOK_URL"
190+
191+
# ── Home Assistant integration (OPTIONAL) ───────────────────────────
192+
# Fills the gap DCGM and node_exporter can't: per-host WALL power (real
193+
# socket-side W, not just GPU), tokens/W efficiency, and rack environment
194+
# (temperature, humidity, AC). The exporter polls Home Assistant REST every
195+
# 15s and produces Prometheus metrics on :9105; obs Prometheus scrapes it
196+
# the same way it scrapes node_exporter. Omit the entire `ha:` block to
197+
# disable — Hearth degrades cleanly (the energy/env cards hide themselves).
198+
#
199+
# Per-node: declare each smart-plug → node mapping in nodes[].sources.ha_plug_id.
200+
#
201+
# Deployment: see server/deploy/ha-exporter.service (systemd unit) and the
202+
# install script in server/deploy/. The HA bearer token must come from a
203+
# root-only systemd drop-in, NEVER from this file.
204+
#
205+
# ha:
206+
# base_url: "http://homeassistant.local:8123" # mDNS — use this, not an IP, so DHCP changes don't break you
207+
# exporter_target: "127.0.0.1:9105" # where Prometheus scrapes the exporter
208+
# rack_sensor_id: "miaomiaoc_cn_blt_3_..." # OPTIONAL temperature/humidity sensor entity id prefix
209+
# rack_ac_plug_id: "2027457700" # OPTIONAL rack AC's smart-plug id
210+
# blocklist: # plugs the exporter must NEVER monitor (refuses to start if seen)
211+
# - "2051674991" # e.g. your main router — turning it off kills the whole LAN

data.js

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -342,6 +342,10 @@
342342
live.cluster.rps = pad(p.cluster.history.rps, HIST);
343343
live.cluster.pow = pad(p.cluster.history.pow, HIST);
344344
live.cluster.temp = pad(p.cluster.history.temp, HIST);
345+
// Home Assistant — optional. Each field may be null (HA exporter
346+
// absent or sensor stale); the UI treats null as "—", not 0.
347+
live.power = p.cluster.power || null;
348+
live.env = p.cluster.env || null;
345349
}
346350
// ── nodes:后端是运行时权威全集。前端动态对齐——后端发什么节点就显
347351
// 什么(id 任意, 不要求匹配前端静态目录)。静态 NODES 仅 mock 模式用;

docs/requirements/01-overview.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
# 01 概述 / Overview
2+
3+
> 详见 [`ha-integration.md`](./ha-integration.md) — 主需求文档(Hearth × Home Assistant 集成)。
4+
> 系统目标:用 HA 智能插座补 DCGM 与 node-exporter 在「整机墙功率」和「机柜环境」上的盲区。

docs/requirements/02-actors.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
# 02 Actors / 参与者
2+
3+
> 详见 [`ha-integration.md` §0](./ha-integration.md) — 完整 Actor 表。
4+
> 核心参与者:集群运维者、ha-exporter 进程、Home Assistant 实例、Prometheus、Hearth FastAPI、Hearth 前端。MT6000 路由器为显式排除项。

docs/requirements/03-scenarios.md

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
# 03 Scenarios / 核心使用场景
2+
3+
> 详见 [`ha-integration.md` §0.5](./ha-integration.md) — 完整 4 个 UC。
4+
>
5+
> - UC-1:整机墙功率监控(Wall Power Monitoring)
6+
> - UC-2:能效比 tokens/W 监控(Efficiency Curve)
7+
> - UC-3:机柜环境告警(Rack Environmental Alerts)
8+
> - UC-4:节点失电检测(Node Power Loss Detection)
Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
# 04 Architecture / 集成架构
2+
3+
> 详见 [`ha-integration.md` §4](./ha-integration.md) — 完整数据流图与组件契合理由。
4+
> 数据流: HA REST(只读 15s 轮询) → ha-exporter (`:9105`/metrics) → Prometheus scrape job → recording rules → FastAPI `/api/cluster` → 前端 Telemetry 区。
Lines changed: 10 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,10 @@
1+
# 05 Spec Handoffs / 交付 Phase 6 spec 的开放点
2+
3+
> 详见 [`ha-integration.md` §9](./ha-integration.md) — 完整 7 个开放设计点(2 CRITICAL + 4 HIGH + 1 MEDIUM)。
4+
>
5+
> 关键开放点(Phase 6 spec 必须解决):
6+
> - 9.1 `hearth.yaml` schema 扩展(`nodes[].sources.ha_plug_id` 等)
7+
> - 9.2 `/api/cluster` payload 的 `power` / `env` 子对象字段定义
8+
> - 9.3 exporter `node` 标签 ⇄ `obs_node_label` 一致性
9+
> - 9.4 HA 离线时 SSE 降级语义(null vs 0 vs stale)
10+
> - 9.5 exporter 服务发现机制

0 commit comments

Comments
 (0)