feat: add Futu OpenD as optional HK realtime and fundamental data source (#2269)

* feat: add Futu OpenD as an optional HK realtime and fundamental data source

Add FutuFetcher and FutuFundamentalAdapter behind FUTU_OPEND_HOST/PORT,
register the settings in Config and config_registry so the Web settings
page can expose host, port and HK realtime priority, and route HK
realtime quotes through a configurable futu/longbridge/akshare/yfinance
order while keeping A-share priority untouched. Include offline tests
for the adapter, config schema and HK routing/fallback, plus docs and
CHANGELOG entries.

* fix: wire Futu fundamentals into HK pipeline and restore quote supplementation

- _fetch_offshore_fundamental_bundle() prefers the Futu fundamental
  adapter for HK when FUTU_OPEND_HOST is configured, and falls back to
  yfinance when Futu is absent or returns no usable content.
- HK realtime priority loop now supplements missing quote fields
  (volume_ratio / turnover_rate / pe/pb / market cap) from later
  configured sources instead of returning after the first non-empty
  quote, matching the US path's _supplement_quote behavior.
- capital_flow / boards blocks are filled from the Futu bundle for HK
  instead of being hard-coded not_supported; status and missing_fields
  aggregation updated accordingly.
- Add regression tests for partial-quote supplementation and Futu
  fundamental bundle routing/fallback.

* test: expect boards block ok when bundle provides belong_boards

The Futu integration made the offshore boards block data-driven instead
of hard-coded not_supported; update the existing US/HK fundamental
context test to match (belong_boards from the bundle now surface as an
ok boards block).

* fix: preserve HK fallback_from metadata and normalize Futu quote timestamps

- HK realtime priority loop now records the failed preferred source token
  and passes it as fallback_from when a later source takes over, so the
  pipeline and analysis context can mark the quote as degraded.
- Futu snapshot update_time is a naive Beijing-time (UTC+8) string; attach
  the +08:00 offset before storing provider_timestamp so stale_seconds /
  is_stale / provider_timestamp freshness semantics are correct instead of
  being parsed as UTC.
- Add regression tests for fallback_from propagation and timestamp
  normalization.

* fix: normalize Futu belong_boards to name/type/code contract

OpenD owner_plate returns plate_code / plate_name / plate_type, but DSA
downstream consumers (notification, extract_board_detail_fields, market
structure) only read name/type/code. Map the fields in
FutuFundamentalAdapter._boards so HK Futu boards are actually consumed
instead of silently dropped, and add regression tests including an
end-to-end check through extract_board_detail_fields.

* fix: merge yfinance bundle when Futu fundamental returns partial blocks

Futu partial success (e.g. statements failed but static info worked) used
to short-circuit the whole bundle, silently dropping the growth/earnings
that the existing yfinance path could still provide. Now, when Futu
returns content but is missing growth or earnings, fetch the yfinance
bundle within the remaining budget and merge the missing blocks
(growth/earnings/institution/capital_flow/belong_boards), keeping
Futu-preferred values where both exist. Add regression test for the
partial-success merge path.

* fix: use field-level checks when deciding Futu-vs-yfinance growth/earnings

The previous merge condition only checked dict truthiness, so a truthy
growth/earnings shell (all-None core values or metadata-only keys such
as report_date/period/currency) would skip the yfinance supplement and
silently downgrade existing HK fundamentals. Add _earnings_block_has_values
(a core numeric field or a populated dividend is required) and reuse the
existing _has_meaningful_payload for growth; both the missing_core check
and the merge loop now use these. Add regression test for the
all-None-shell scenario.

* fix: fill HK fundamental field gaps from yfinance instead of block-level checks

Block-level meaningful checks still skipped the yfinance supplement when
Futu hit only part of the growth/earnings fields (e.g. revenue_yoy but
None net_profit_yoy, or earnings with only basic_eps), silently dropping
fields the main branch used to provide. Replace the missing_core decision
with a per-field gap list (growth: revenue_yoy/net_profit_yoy/gross_margin;
earnings.financial_report: revenue/net_profit_parent/basic_eps/gross_profit)
and make the merge field-level: keep Futu values, fill each missing field
from yfinance. Add regression tests for partial-hit and all-None shells.

* fix: normalize Futu dividends to the repo contract and treat dividend gaps as supplement triggers

Futu OpenD dividend_list carries raw fields (statement/ex_date/record_date)
which the notification/data_processing market-structure consumers do not
read; the repo contract is ttm_cash_dividend_per_share,
ttm_dividend_yield_pct and events[].cash_dividend_per_share /
ex_dividend_date / event_date. Normalize events in
FutuFundamentalAdapter._dividends_and_splits, compute TTM count/cash and
yield from the latest quote, and teach _field_gaps/_merge_bundles to treat
a dividend block that does not satisfy the contract as a gap so yfinance
supplements it. Also dedupe FUTU_OPEND_HOST/PORT in full-guide_EN.

* fix: read dividend yield price from UnifiedRealtimeQuote objects

FutuFetcher.get_realtime_quote returns a UnifiedRealtimeQuote dataclass,
not a dict, so the yield branch in _dividends_and_splits that guarded on
isinstance(quote, dict) never ran on the live Futu path, silently dropping
ttm_dividend_yield_pct while the contract check considered the dividend
block complete. Read price via getattr(quote, 'price', None) and keep the
dict fallback for other fetchers; add a regression test driving the real
UnifiedRealtimeQuote shape.

* fix: treat dividend blocks with TTM cash but no yield as supplement gaps

The repo contract consumes ttm_cash_dividend_per_share and
ttm_dividend_yield_pct together. When the Futu dividend path has events
and TTM cash but the extra realtime price snapshot failed (quote None /
no price), ttm_dividend_yield_pct cannot be computed and the block was
previously treated as complete, so yfinance was never consulted and the
notification rendered the yield as N/A.

_dividend_contract_has_values() now requires the paired yield whenever
TTM cash is present, so _field_gaps() triggers the yfinance supplement
and _merge_bundles() replaces the incomplete dividend block.

Add regression tests for the adapter-level gap shape (quote unavailable
leaves no yield) and the manager-level supplement path (Futu cash
without yield pulls yfinance and fills the yield).

* fix: skip unconfigured Futu in HK realtime routing

When FUTU_OPEND_HOST is not configured, the HK realtime priority loop
used to still attempt the futu source, record it as the failed primary,
and attach fallback_from='futu' to a successful quote from the next
enabled source (longbridge/akshare/yfinance). Consumers then wrongly
treated an enabled source's first success as degraded fallback data,
contradicting the documented contract that Futu only participates when
OpenD is configured.

The HK loop now checks FutuFetcher.has_configured_endpoint() once and
skips the futu token entirely when it is disabled, so no fallback_from
is written. Existing configured-Futu routing tests explicitly patch the
endpoint check; a new regression test asserts an unconfigured Futu is
never called and the enriched quote carries fallback_from=None.

* fix: release cached HK Futu fundamental fetcher in DataFetcherManager.close()

The HK Futu fundamental path lazily creates and caches its own
FutuFetcher (an OpenQuoteContext-backed OpenD connection) on
_futu_fundamental_fetcher, but close() only released the TickFlow
fetcher and the default fetchers snapshot. Explicit close / reload
paths therefore left the OpenD connection hanging.

close() now takes the cached _futu_fundamental_fetcher, clears the
reference and calls its close() best-effort. A regression test injects
an observable fetcher into _futu_fundamental_fetcher and asserts
close() invokes it and clears the attribute.

---------

Co-authored-by: BayMax local review <baymax-local@invalid>

Author: sunkai174634

.env.example

@@ -13,6 +13,7 @@ STOCK_LIST=600519,300750,002594
 # futu-api 10.8 仅支持 IPv4；Docker 连接宿主机 OpenD 时请勿使用容器内的 127.0.0.1，详见 docs/full-guide.md。
 # FUTU_OPEND_HOST=127.0.0.1
 # FUTU_OPEND_PORT=11111
+# FUTU_HK_REALTIME_SOURCE_PRIORITY=futu,longbridge,akshare,yfinance
 # FUTU_SECURITY_FIRM=NONE          # 可选；默认由 OpenD 自动识别，也可显式指定券商
 # FUTU_ACC_ID=                    # 可选；正整数，指定后只读取该真实账户
 

apps/dsa-web/src/locales/settingsHelp.ts

@@ -351,6 +351,30 @@ const settingsHelpZhCN: SettingsHelpMap = {
     impact: ['影响现价、技术指标、盘中分析和部分报告字段。'],
     notes: ['单一数据源失败应降级到后续数据源，不应拖垮主流程。'],
   },
+  'settings.data_source.FUTU_OPEND_HOST': {
+    title: 'Futu OpenD 地址',
+    summary: '配置 Futu OpenD 服务地址。留空时不启用 Futu 数据源。',
+    usage: '填写 IPv4 地址或可解析到 IPv4 的主机名。',
+    valueNotes: ['OpenD 必须允许 DSA 容器访问。'],
+    impact: ['影响港股 Futu 实时行情、历史行情和基本面数据访问。'],
+    notes: ['这是服务地址，不是 Futu 账号密码。'],
+  },
+  'settings.data_source.FUTU_OPEND_PORT': {
+    title: 'Futu OpenD 端口',
+    summary: '配置 Futu OpenD TCP 端口。',
+    usage: '填写 1 到 65535 之间的端口，默认 11111。',
+    valueNotes: ['端口必须与 OpenD 实际监听端口一致。'],
+    impact: ['影响 DSA 与 Futu OpenD 的连接。'],
+    notes: ['修改后通常需要重启 DSA 服务以重新建立连接。'],
+  },
+  'settings.data_source.FUTU_HK_REALTIME_SOURCE_PRIORITY': {
+    title: 'Futu 港股实时数据源优先级',
+    summary: '配置港股实时行情的 Futu/Longbridge/AkShare/Yfinance 尝试顺序。',
+    usage: '使用英文逗号分隔，可选 futu、longbridge、akshare、yfinance。',
+    valueNotes: ['前面的数据源优先尝试；失败后自动回退。'],
+    impact: ['只影响港股实时行情，不改变 A 股实时数据源优先级。'],
+    notes: ['未配置 Futu OpenD 时会自动跳过 futu。'],
+  },
   'settings.data_source.search_api_keys': {
     title: '搜索服务 API Key',
     summary: '配置新闻与搜索增强所需的第三方搜索服务密钥。',
@@ -1523,6 +1547,30 @@ const settingsHelpEnUS: SettingsHelpMap = {
     impact: ['Affects request count and per-request pressure for TickFlow batch prefetch.'],
     notes: ['This setting only affects TickFlow batch paths.'],
   },
+  'settings.data_source.FUTU_OPEND_HOST': {
+    title: 'Futu OpenD Host',
+    summary: 'Configures the Futu OpenD service address. Leave empty to disable Futu.',
+    usage: 'Use an IPv4 address or a hostname resolving to IPv4.',
+    valueNotes: ['The OpenD service must be reachable from the DSA container.'],
+    impact: ['Affects Futu HK realtime, historical, and fundamental data access.'],
+    notes: ['This is a service address, not a Futu account credential.'],
+  },
+  'settings.data_source.FUTU_OPEND_PORT': {
+    title: 'Futu OpenD Port',
+    summary: 'Configures the Futu OpenD TCP port.',
+    usage: 'Use a port from 1 to 65535; the default is 11111.',
+    valueNotes: ['The port must match the OpenD listener.'],
+    impact: ['Affects the DSA connection to Futu OpenD.'],
+    notes: ['Restarting DSA is normally required after changing it.'],
+  },
+  'settings.data_source.FUTU_HK_REALTIME_SOURCE_PRIORITY': {
+    title: 'Futu HK Realtime Source Priority',
+    summary: 'Configures the Futu/Longbridge/AkShare/Yfinance order for HK realtime quotes.',
+    usage: 'Use comma-separated futu, longbridge, akshare, or yfinance values.',
+    valueNotes: ['Earlier providers are tried first; failures fall back automatically.'],
+    impact: ['Affects HK realtime quotes only, not A-share realtime priority.'],
+    notes: ['The futu entry is skipped when OpenD is not configured.'],
+  },
   'settings.data_source.stock_index_remote': {
     title: 'Remote Stock Index',
     summary: 'Fetches the latest stock autocomplete index from GitHub main and caches it locally.',