postprocess: 允許在詞庫後製過程中，手動調整常見問題詞的分數 - #868
Conversation
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces a postprocessing pipeline for the compiled phrase database, including a new postprocess script, Mandarin phonetic component converters, a Viterbi-based grid walk algorithm, and associated unit tests. The Makefile has been updated to integrate this postprocessing step. The review feedback highlights several critical bugs in postprocess.py, specifically string formatting errors in promote_over_peers and promote_over_single_syllables (missing or mismatched arguments for %d), and potential TypeError exceptions in find_top_unigram_in_lm and its callers due to improper handling of missing readings.
Important
The consumer version of Gemini Code Assist on GitHub is being sunset. Starting June 18, 2026, new organization installations will be blocked, and all code review activity will officially cease on July 17, 2026.
For more details on the timeline and next steps, please review the Help Documentation.
|
我加了 #869 ，可以把特別需要調整的問題詞挑出來。 |
This comment was marked as resolved.
This comment was marked as resolved.
lukhnos
changed the title
- Implement Bopomofo syllable representation and grid walk in Python - Implement a postprocessor - Use Postprocess.txt as the script to the postprocessing step - Ensure that boosted phrases do not cause regressions - Add unit tests - Introduce `make format` and `make typecheck` for new Python code - Integrate the tests into the CI workflow
4 participants
先把這個 PR 發出來，供大家討論。
這個 PR 引進一個名叫
Postprocess.txt的 script。這個 script 有兩個功用：目前 script 提供兩種修正：
promote-over-single-syllables提供修改「好險」總是被「好」跟「顯」兩個高分字蓋過的問題。promote-over-peers提供修改「醫科」總是被同音的雙字詞「一顆」蓋過的問題。其他指令描述請參見
Postprocess.txt。為了要正確實作 assert，於是用 Python 實作了小麥的組詞邏輯。這樣做的好處是詞庫的 build system 不需要編譯其他 C++ 程式碼，讓這個流程維持原有的單純。
Python 程式還有許多需要強化的地方，例如寫成 type-checked 的風格，另外 script 在錯誤訊息等方面的 UI/UX 也還有許多有待改善之處。