手動提高幾個常見詞的分數,用以了解手動加分過程會遇到的問題 - #876
Merged
Merged
Conversation
|
Note Gemini is unable to generate a review for this pull request due to the file types involved not being currently supported. |
Contributor
Author
|
另外也有這種打語音不會遇到問題,打讀音才會遇到問題的詞: |
Contributor
|
要不要先看看那些常被抱怨的,像是「面是」那些? |
zonble
approved these changes
Jul 12, 2026
Also fix `make clean` which now removes data-raw.txt
Contributor
Author
我多增加了 #858 中列舉的「字彙」、「試試」跟「面試」。並沒有全部收錄的原因是其中有一些其實容易有問題(例如「不以/不已」,或是「鑰匙」這種或許可以靠另一讀音排除輸入障礙的詞彙等等)。另外只增加「字彙」分數也有其問題,我都列舉在修訂過的檔案中。 |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
這個 PR 手動增加一些詞的分數,有的是從改進過的
make analysis(參見 #875)選出的,有的是 #858 列出的常見問題詞。製作這 PR 過程中會發現,光只是「偏好長詞」或是「將分數調得比各高分同音字總分高」這樣的做法,仍有不足。好比說,「試用」其實是個常見詞,但語料中「是用」跟「適用」分數更高,而或許「試用/適用」也更該從上下文來選詞,這些都不是目前的選詞邏輯能處理的。
同時這個 PR 也一定程度上示範了要增加更多加分腳本,所需要考量的諸多因素。這是為何要加入
before跟assert的原因:這樣才能了解問題根源,跟是否有 regression。我的另一個心得是 #858 提供了一種作法,但或許也應該考慮將「偏好長詞」放在 user override model 這個層級,這樣的好處是使用者只要打斷一次選字,override model 就應該記得不要再選這個長詞了。當然 override model 本身也還有許多可供改進之處,得另開討論。