Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 2 additions & 3 deletions API.md
Original file line number Diff line number Diff line change
Expand Up @@ -957,14 +957,13 @@ Authorization: Bearer <token>
### 9.2 altnames(ALTNAME_DATA,別名)

- `target.pk`:`c_personid`、`c_alt_name_chn`、`c_alt_name_type_code`(3-key,**不含** `c_sequence`)
- **update** 白名單:`c_alt_name_chn`、`c_alt_name`、`c_alt_name_type_code`、`c_source`、`c_pages`、`c_notes`、`c_sequence`、`c_alt_name_pinyin`、`c_alt_name_pinyin2`、`c_alt_name_pinyin3`、`c_alt_name_role`
- **update** 白名單:`c_alt_name_chn`、`c_alt_name`、`c_alt_name_type_code`、`c_source`、`c_pages`、`c_notes`、`c_sequence`
- **create** 白名單:同上再加 `c_personid`
- 哨兵欄:`c_source`(另 `c_alt_name_type_code` 的 `-999` 會轉 `0`)
- 特殊行為(都會靜默改寫送出的值):
- `c_alt_name_chn` 與 `c_alt_name` 都會做括號正規化(全角轉半角、括號前後補空格)。
- `c_alt_name_chn` 另會做**異體字嚴格替換**(strict:人名/別名欄專用的較少規則集,允許保留較多異體字);因為它同時是主鍵欄,替換後的值才是落庫主鍵——**請以回應的 `result.pk` 為準**。
- 同一列的其他文本欄(例如 `c_notes`、`c_pages`)走**全量規則**(lenient),所以會有 `c_alt_name_chn` 保留某個異體字、而 `c_notes` 裡同一個字被替換的情形——這是刻意的。
- `c_alt_name_pinyin`/`2`/`3` 的 `v` 會轉成 `ü`(靜默,無 `notices`)。
- 若正規化後與同類型的既有別名撞主鍵,回 409(訊息會說明需先手動整理)。
- 寫入成功後會同步重建姓名全文檢索索引(`CBDB__NAME_FTS`),這是預期的副作用。

Expand Down Expand Up @@ -1032,7 +1031,7 @@ Authorization: Bearer <token>
### 9.10 texts(BIOG_TEXT_DATA,著述)

- `target.pk`:`c_personid`、`c_textid`、`c_role_id`
- **update** 白名單:`c_textid`、`c_role_id`、`c_source`、`c_pages`、`c_notes`、`c_supplement`、`c_text_year`
- **update** 白名單:`c_textid`、`c_role_id`、`c_source`、`c_pages`、`c_notes`
- **create** 白名單:同上再加 `c_personid`
- 哨兵欄:`c_textid`、`c_source`
- `c_textid` 對應 `TEXT_CODES`,可用 `GET /api/v2/texts?ids=...` 查詢(見〈其他開放端點〉)。
Expand Down
8 changes: 8 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,14 @@

## 2026-08

### 清掉 v2 mutation 白名單裡 6 個資料庫不存在的欄位,並加上機械化守衛

- **問題**:`allowedFields()` 是手寫清單,與資料表實際欄位之間沒有任何保證。列進去卻不存在的欄會被白名單**放行**,錯誤要到 `DB::table()->update()`/`->insert()` 才發生 ⇒ 使用者拿到 **500 而不是 422**;`API.md` 也照著白名單把這些欄位公布為對外契約,外部協作者照文件實作就會撞 500。
- **清掉的 6 欄**:`ALTNAME_DATA` 的 `c_alt_name_pinyin`/`_pinyin2`/`_pinyin3`/`c_alt_name_role`(#1284),以及守衛順手抓到的 `BIOG_TEXT_DATA.c_supplement`/`c_text_year`。六欄 migration 均未曾建立(baseline `import_cbdb_schema.php` 的 `ALTNAME_DATA` 就是 12 欄)、prod 也查不到。`c_text_year` 屬於 `TEXT_CODES`,走 text-codes 聚合端,本來就不該出現在 `BIOG_TEXT_DATA` 的白名單裡。
- **為什麼半年沒被發現**:每支相關測試都**自己在合成表裡把那幾欄建了出來**(6 支測試檔),於是整條 PinyinUmlaut Tier 1 的 ALTNAME 分支是對著一個現實中不存在的表形在測——測試永遠綠,prod 永遠打不到。同一類的「幻影 `c_supplement`」先前已在 ASSOC/KIN/POSSESSION 三個 handler 逐一手工清掉過,BIOG_TEXT_DATA 這筆漏網。
- **連帶效應**:`PinyinUmlaut::ALTNAME_PINYIN_V_FIELDS` 隨之移除(刻意不留空常數,免得下一個人以為只是暫時沒欄位而重新填回),兩個 altname handler 不再呼叫 `normalizeFields()`。**Tier 2 不受影響**——ALTNAME 唯一的別名羅馬字欄 `c_alt_name` 照舊走前端互動確認、後端不轉。`VariantReplaceScope::EXCLUDED_COLUMNS['ALTNAME_DATA']` 從 4 欄縮為 `['c_alt_name']`;那裡「`c_alt_name_role` 是 prod-only」的註解與 `docs/CHAR_VARIANT_MAP_TEXT_COLUMN_ROLLOUT_PLAN.md` 的同旨段落都寫反了,一併更正。
- **防復發**:新增 `tests/Feature/MutationAllowedFieldsSchemaDriftTest.php`,掛 `RefreshDatabase` 讓 schema 真的來自 `database/migrations`(同 `VariantReplaceRegistryDriftTest` 的理由),逐一比對全部人物子資源 handler 的 `allowedFields()` + `keyColumns()`。這支守衛正是抓出 `BIOG_TEXT_DATA` 那兩欄的來源。**邊界**:它抓「白名單列了 migration 沒有的欄」,抓不到反向的「prod 有、migration 沒有」。

### 錄入端加上 Unicode NFC 正規化:相容表意文字折疊為統一表意文字
- **修的是一個獨立於異體字的真缺陷**:CJK 相容表意文字(U+F900–U+FAFF 與補充區)與統一表意文字在資料庫層是**不同位元組**,唯一鍵擋不住、精確比對找不到、搜尋互不可見。生產庫實測:`ALTNAME_DATA.c_alt_name_chn` 107 列、`OFFICE_CODES.c_office_chn` 23 列、`BIOG_MAIN.c_name_chn` 17 列、`TEXT_CODES.c_title_chn` 16 列含相容碼位。最具體的一例——`c_personid=551931`「李晄」的「李」是 U+F9E1,而其他人的「李」是 U+674E,**用「李」搜尋找不到這個人**。
- **與異體字落地替換是兩件不同性質的事**,新增的 `App\Support\UnicodeNfc` 類註有完整對照:NFC 折疊的兩個碼位在 Unicode 定義上**就是同一個字**(canonical equivalence,相容碼位存在的目的只是與舊編碼往返轉換),不涉及任何編輯判斷;而 愼→慎 那類異體字 Unicode **刻意不折疊**(統一表意文字永不給 canonical decomposition),那是應用層的策管決定。實測 `char_variant_map` 的 7 筆種子變體字**全部 NFC 不變**——兩套機制作用域不重疊。
Expand Down
8 changes: 0 additions & 8 deletions app/Services/Mutations/AltnameCreateHandler.php
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@
use App\Services\AuditLogService;
use App\Services\BracketNormalizer;
use App\Services\NameSearchIndexService;
use App\Support\PinyinUmlaut;
use Illuminate\Http\JsonResponse;
use Illuminate\Support\Facades\Schema;

Expand Down Expand Up @@ -52,19 +51,12 @@ protected function allowedFields(): array {
'c_pages',
'c_notes',
'c_sequence',
'c_alt_name_pinyin',
'c_alt_name_pinyin2',
'c_alt_name_pinyin3',
'c_alt_name_role',
];
}

protected function preprocessCreateData(array $data): array {
$data = BracketNormalizer::normalizeAltname($data);

// 保存時拼音 v→ü 歸一化(Tier 1;僅 c_alt_name_pinyin/2/3。c_alt_name 走前端 Tier 2、不在此轉)
$data = PinyinUmlaut::normalizeFields($data, PinyinUmlaut::ALTNAME_PINYIN_V_FIELDS);

$data = $this->normalizeSentinelValues($data, ['c_alt_name_type_code', 'c_source']);

// #71:非 PK 碼欄 c_source 完全幂等(null/''/-999→0),對齊已修的 AltnameMutationHandler。
Expand Down
8 changes: 0 additions & 8 deletions app/Services/Mutations/AltnameMutationHandler.php
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,6 @@
use App\Services\AuditLogService;
use App\Services\BracketNormalizer;
use App\Services\NameSearchIndexService;
use App\Support\PinyinUmlaut;
use Illuminate\Http\JsonResponse;
use Illuminate\Support\Facades\DB;
use Illuminate\Support\Facades\Schema;
Expand Down Expand Up @@ -52,20 +51,13 @@ protected function allowedFields(): array {
'c_pages',
'c_notes',
'c_sequence',
'c_alt_name_pinyin',
'c_alt_name_pinyin2',
'c_alt_name_pinyin3',
'c_alt_name_role',
];
}

protected function preprocessUpdateData(array $data): array {
// 括號正規化
$data = BracketNormalizer::normalizeAltname($data);

// 保存時拼音 v→ü 歸一化(Tier 1;僅 c_alt_name_pinyin/2/3。c_alt_name 走前端 Tier 2、不在此轉)
$data = PinyinUmlaut::normalizeFields($data, PinyinUmlaut::ALTNAME_PINYIN_V_FIELDS);

// -999 → 0 轉換
$data = $this->normalizeSentinelValues($data, ['c_alt_name_type_code', 'c_source']);
// sentinel 完全幂等:c_source(legacy 哨兵 0=Unknown)的 null/'' 也→0(normalizeSentinelValues 只做 -999)。
Expand Down
2 changes: 0 additions & 2 deletions app/Services/Mutations/AssociationCreateHandler.php
Original file line number Diff line number Diff line change
Expand Up @@ -249,10 +249,8 @@ protected function allowedFields(): array {
'c_source',
'c_pages',
'c_notes',
// ⚠️ 移除幻影 c_supplement(ASSOC_DATA 無此欄)。
'c_sequence',
'c_assoc_count',
// Task 27:補回舊表單可錄入欄位(皆 ASSOC_DATA 真實欄)。
'c_topic_code',
'c_occasion_code',
'c_tertiary_personid',
Expand Down
2 changes: 0 additions & 2 deletions app/Services/Mutations/AssociationMutationHandler.php
Original file line number Diff line number Diff line change
Expand Up @@ -332,10 +332,8 @@ protected function allowedFields(): array {
'c_source',
'c_pages',
'c_notes',
// ⚠️ 移除幻影 c_supplement(ASSOC_DATA 無此欄)。
'c_sequence',
'c_assoc_count',
// Task 27:補回舊表單可錄入欄位(皆 ASSOC_DATA 真實欄)。
'c_topic_code',
'c_occasion_code',
'c_tertiary_personid',
Expand Down
2 changes: 0 additions & 2 deletions app/Services/Mutations/KinshipCreateHandler.php
Original file line number Diff line number Diff line change
Expand Up @@ -179,8 +179,6 @@ protected function allowedFields(): array {
'c_source',
'c_pages',
'c_notes',
// Task 27 補回:c_autogen_notes 為 KIN_DATA 真實欄(舊表單以 textarea 暴露為可錄入);
// 移除幻影 c_supplement(KIN_DATA 無此欄,SHOW COLUMNS 確認)。
'c_autogen_notes',
];
}
Expand Down
1 change: 0 additions & 1 deletion app/Services/Mutations/KinshipMutationHandler.php
Original file line number Diff line number Diff line change
Expand Up @@ -265,7 +265,6 @@ protected function allowedFields(): array {
'c_source',
'c_pages',
'c_notes',
// Task 27 補回:c_autogen_notes 為 KIN_DATA 真實欄;移除幻影 c_supplement(KIN_DATA 無此欄)。
'c_autogen_notes',
];
}
Expand Down
2 changes: 0 additions & 2 deletions app/Services/Mutations/PossessionMutationHandler.php
Original file line number Diff line number Diff line change
Expand Up @@ -208,8 +208,6 @@ protected function validatePersonIdInRow(int $personId, object $original): ?Json
}

protected function allowedFields(): array {
// 僅列 POSSESSION_DATA 真實欄位(對照 schema migration)。
// 早期版本含 c_supplement / c_measure_value / c_firstyear / c_lastyear 等不存在欄位(假欄),已移除。
return [
'c_sequence',
'c_possession_act_code',
Expand Down
2 changes: 0 additions & 2 deletions app/Services/Mutations/TextCreateHandler.php
Original file line number Diff line number Diff line change
Expand Up @@ -41,8 +41,6 @@ protected function allowedFields(): array {
'c_source',
'c_pages',
'c_notes',
'c_supplement',
'c_text_year',
];
}

Expand Down
2 changes: 0 additions & 2 deletions app/Services/Mutations/TextMutationHandler.php
Original file line number Diff line number Diff line change
Expand Up @@ -40,8 +40,6 @@ protected function allowedFields(): array {
'c_source',
'c_pages',
'c_notes',
'c_supplement',
'c_text_year',
];
}

Expand Down
13 changes: 3 additions & 10 deletions app/Support/PinyinUmlaut.php
Original file line number Diff line number Diff line change
Expand Up @@ -28,18 +28,11 @@ class PinyinUmlaut {
*
* 僅列**定義上即漢語拼音**的欄:`c_surname`/`c_mingzi`(其他羅馬化另存於 `c_*_rm`/`c_*_proper`)、
* 以及由二者組出的 `c_name`。**刻意不含** `c_*_rm`(Wade-Giles)/`c_*_proper`(母語拉丁名,
* 可能含真 `v`)。設計見 docs/PINYIN_SAVE_NORMALIZE_DESIGN.md。
* 可能含真 `v`)。ALTNAME_DATA 沒有 Tier 1 欄位——其唯一的別名羅馬字欄 `c_alt_name` 可能含
* 西文別名(如 Denver),走前端 Tier 2 互動確認。設計見 docs/PINYIN_SAVE_NORMALIZE_DESIGN.md。
*/
public const BIOG_MAIN_PINYIN_V_FIELDS = ['c_surname', 'c_mingzi', 'c_name'];

/**
* 保存時「靜默」歸一化的 ALTNAME_DATA 拼音欄(Tier 1)。
*
* 僅列明確標為「拼音」的欄。**刻意不含** `c_alt_name`——後者為泛用別名羅馬字、可能含西文別名
* (如 Denver),改由前端 Tier 2 互動確認(見設計文件 §3/§4.2)。
*/
public const ALTNAME_PINYIN_V_FIELDS = ['c_alt_name_pinyin', 'c_alt_name_pinyin2', 'c_alt_name_pinyin3'];

/** 將字串中作為 ü 代寫的 v 正規化為 ü。null/空字串原樣返回。 */
public static function normalize(?string $value): string {
if ($value === null || $value === '') {
Expand All @@ -56,7 +49,7 @@ public static function normalize(?string $value): string {
/**
* 對 $data 中列於 $fields 的字串欄套用 normalize();非字串/缺欄/null 原樣略過。
*
* 保存前歸一化用:搭配 self::BIOG_MAIN_PINYIN_V_FIELDS/ALTNAME_PINYIN_V_FIELDS。冪等。
* 保存前歸一化用:搭配 self::BIOG_MAIN_PINYIN_V_FIELDS。冪等。
*
* @param array<string,mixed> $data
* @param list<string> $fields
Expand Down
2 changes: 1 addition & 1 deletion app/Support/UnicodeNfc.php
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@
*
* **已知取捨**:少數相容碼位在來源編碼裡帶有「讀音」資訊(U+F9E1 李 來自 KS X 1001 的
* 「이」讀音、與 U+674E 的「리」相對),NFC 會抹掉那個區別。對 CBDB 不構成損失——讀音存在
* 獨立的拼音欄(`c_name`/`c_surname`/`c_alt_name_pinyin`),不靠碼位承載;而庫中這些字
* 獨立的拼音欄(`c_name`/`c_surname`/`c_alt_name`),不靠碼位承載;而庫中這些字
* 出現在漢人姓名與官名裡,來源是輸入法/舊編碼轉換的意外,不是刻意的讀音標記。
* 這也是 W3C Character Model 對網路上文本的建議儲存形式。
*
Expand Down
12 changes: 2 additions & 10 deletions app/Support/VariantReplaceScope.php
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ final class VariantReplaceScope {
// 替換這欄會直接破壞該設計(「峯」的讀音條目會變成「峰」)。
'pinyin' => ['c_chn'],

// ── 拉丁人名欄(共 13 欄,D4):排除而非 strict。
// ── 拉丁人名欄(共 10 欄,D4):排除而非 strict。
// 理由 1:strict 仍會套那 6 筆規則,真有人填漢字時 愼→慎/靑→青 照樣被改;
// 只有排除能真正不碰。
// 理由 2:排除順帶消掉組合欄失步——BiogMainRepository::updateById() 的
Expand All @@ -91,15 +91,7 @@ final class VariantReplaceScope {
// 跨表代碼鍵 → INDEXYEAR_TYPE_CODES.c_index_year_type_code(varchar PK)。
'c_index_year_type_code',
],
'ALTNAME_DATA' => [
'c_alt_name', 'c_alt_name_pinyin', 'c_alt_name_pinyin2', 'c_alt_name_pinyin3',
// c_alt_name_role:prod-only varchar(50)(不在 migrations/DATABASE_SCHEMA.md,
// 只出現在 allowedFields 白名單與測試合成表),全庫**零**應用邏輯讀它。
// 語義無法從程式判定,所以刻意選保守側排除:漏一次替換是可回復的
// (日後歸類清楚再放進範圍),而誤改一個代碼/角色鍵是不可回復的。
// 若日後確認它是散文性質的說明文字,再移出這份清單。
'c_alt_name_role',
],
'ALTNAME_DATA' => ['c_alt_name'],

// ── 跨表 join/樹狀關聯的「代碼鍵」(D3)。
// 判準是「這個值是用來跟別表對上的代碼」,**不是**「是不是 varchar PK 成員」——
Expand Down
2 changes: 1 addition & 1 deletion docs/CHAR_VARIANT_MAP_CONSOLIDATION_PLAN.en.md
Original file line number Diff line number Diff line change
Expand Up @@ -129,7 +129,7 @@ The following describes the **future** direction for rewiring `TITLE_VARIANT_MAP
- **Update/proposal path**: `BiogMainMutationHandler::prepareProposalPayload()` (`app/Services/Mutations/BiogMainMutationHandler.php:196-200`) and `BiogMainRepository::updateById()` (`app/Repositories/BiogMainRepository.php:246`) each independently combine `c_surname_chn`+`c_mingzi_chn` into `c_name_chn` — a clear hook point, with the query condition `c_strict_excluded = 0` (strict mode; currently corresponds to 6 rows: 愼, 槀, 靑, 頴, 淸, 厰; 峯 is excluded because `c_strict_excluded=1`). **Open item**: implementation must explicitly decide the replacement order — either replace `c_surname_chn`/`c_mingzi_chn` in their own fields first and then compose `c_name_chn` (preserving the current invariant `c_name_chn === c_surname_chn.c_mingzi_chn`), or replace only the composed `c_name_chn` (which would leave it inconsistent with the component fields). This plan leans toward the former (replace the component fields first, then compose), but implementation must pin this down explicitly in code and tests before starting — the two paths must not each assume a different approach.
- **Create path**: `BiogMainCreateHandler.php` itself has no name-composition logic; it only lists `c_name_chn`/`c_surname_chn`/`c_mingzi_chn` in its field allowlist and delegates to `store(Request $request)` in `app/Repositories/BiogMainRepository.php:353-365`. `store()` starts with `$data = $request->all()`, then runs `timestamp()`, `auto_pinyin($data)`, `BracketNormalizer::normalizeBiogMain()`, `PinyinUmlaut::normalizeFields()` (lines 355-361) in sequence before `BiogMain::create($data)` — **none of these existing steps re-derive `c_name_chn`**. In other words, when a new person is created, `c_name_chn` is the raw string sent by the frontend throughout, with no logic that recomposes it from `c_surname_chn`+`c_mingzi_chn`. This means the create path **has no existing "data replacement" hook to modify** (unlike the update path, which already has name-composition logic where a condition can simply be added in place) — applying variant-character replacement here requires **adding** a new piece of normalization code inside `store()`. This is a larger scope of work than the update path, and it's entirely new behavior (the current create path does no variant-character replacement on person names at all), so future task planning needs to estimate create and update separately and should not assume they're symmetric.

3. **ALTNAME_DATA (person's alternate names) write path**: unlike BIOG_MAIN, `AltnameCreateHandler`/`AltnameMutationHandler` each already have clear preprocessing hook points (`AltnameCreateHandler::preprocessCreateData()` lines 61-71, `AltnameMutationHandler::preprocessUpdateData()` around lines 61-66), which already apply `BracketNormalizer`/`PinyinUmlaut::normalizeFields` normalization to fields like `c_alt_name_pinyin` — a more ready-made hook point than the BIOG_MAIN create path. The query condition is likewise `c_strict_excluded = 0` (strict mode), applied to the `c_alt_name_chn` field (`c_alt_name_chn` is part of `ALTNAME_DATA`'s composite primary key: `c_personid + c_alt_name_chn + c_alt_name_type_code`; rewriting this field constitutes a "composite primary key value change" rather than a plain field update, and implementation needs to stay consistent with existing rename/primary-key-change handling — e.g. the primary-key-change handling logic in `AltnameMutationHandler` — rather than being simplified into a plain field overwrite).
3. **ALTNAME_DATA (person's alternate names) write path**: unlike BIOG_MAIN, `AltnameCreateHandler`/`AltnameMutationHandler` each already have clear preprocessing hook points (`AltnameCreateHandler::preprocessCreateData()` lines 61-71, `AltnameMutationHandler::preprocessUpdateData()` around lines 61-66), which already apply `BracketNormalizer` normalization — a more ready-made hook point than the BIOG_MAIN create path. The query condition is likewise `c_strict_excluded = 0` (strict mode), applied to the `c_alt_name_chn` field (`c_alt_name_chn` is part of `ALTNAME_DATA`'s composite primary key: `c_personid + c_alt_name_chn + c_alt_name_type_code`; rewriting this field constitutes a "composite primary key value change" rather than a plain field update, and implementation needs to stay consistent with existing rename/primary-key-change handling — e.g. the primary-key-change handling logic in `AltnameMutationHandler` — rather than being simplified into a plain field overwrite).

4. Add `app/Services/CharVariantMapService` (working name; naming can be finalized during implementation) as a unified query interface.

Expand Down
Loading