Skip to content

feat(accent): local UniDic + POS-driven patches (closes #48, #50) #51

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
torrid-fish wants to merge 18 commits into from
Closed

feat(accent): local UniDic + POS-driven patches (closes #48, #50) #51

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
c84bc65
refactor(accent): replace greedy aligner with Needleman-Wunsch DP + r…
torrid-fish May 21, 2026
93d74e1
feat(accent): regex reading-override layer + URL/non-JP preprocessing
torrid-fish May 21, 2026
c9b321d
feat(accent): add /MarkAccent/stream/ NDJSON endpoint + dev helpers
torrid-fish May 21, 2026
97eb0c9
feat(accent): swap Yahoo MA for local fugashi + UniDic with full poli…
torrid-fish May 21, 2026
24f8761
docs(accent): extend package README for new package shape (refs #50)
torrid-fish May 21, 2026
2afc434
chore(spike): import spike artefacts — docs + probe scripts (refs #50)
torrid-fish May 21, 2026
180ceec
fix(accent): merge acronyms + decimals so morae stay on the right tok…
torrid-fish May 21, 2026
3ad452d
fix(accent): make english_compound k=0 free; unblock DP from stealing…
torrid-fish May 21, 2026
00f9d7b
chore: drop Yahoo-era config/, .env, and spike scripts/docs (refs #50)
torrid-fish May 21, 2026
d69a4fe
feat(accent): symbol readings, okurigana subwords, script switch + En…
torrid-fish May 21, 2026
57e175a
refactor(accent): drop merge_readable_symbol_compounds; SYMBOL_READIN…
torrid-fish May 21, 2026
551b710
fix(accent): preserve unit-compound furigana from english toggle wipe…
torrid-fish May 21, 2026
b91ce0d
feat(accent): user-maintained patch table for OJAD/UniDic misreads (r…
torrid-fish May 21, 2026
989b47f
feat(accent): per-mora accent override in USER_PATCHES + seed 3 fixes…
torrid-fish May 21, 2026
406ceba
refactor(accent): require explicit per-mora int tuples in USER_PATCHE…
torrid-fish May 21, 2026
662d4ee
fix(accent): free-consume for override-synthesized tokens; refresh ac…
torrid-fish May 21, 2026
7ce6b37
fix(accent): clear per-mora furigana on katakana tokens when toggle i…
torrid-fish May 21, 2026
95c7ae1
docs(accent): rewrite api/accent/README.md in English (refs #50)
torrid-fish May 21, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 0 additions & 2 deletions .env.example
View file
Open in desktop

This file was deleted.

5 changes: 4 additions & 1 deletion .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -216,4 +216,7 @@ __marimo__/
.streamlit/secrets.toml

# Secrest files
secret*
secret*

data/
output/
5 changes: 2 additions & 3 deletions Dockerfile
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@ RUN uv sync --frozen --no-dev --no-install-project
COPY . .

# Compile only our app code (skip .venv) and drop the .py sources
RUN python -m compileall -b -q main.py api config \
&& find main.py api config -name "*.py" -delete
RUN python -m compileall -b -q main.py api \
&& find main.py api -name "*.py" -delete

FROM python:3.11-slim

Expand All @@ -25,7 +25,6 @@ WORKDIR /app
COPY --from=builder /app/.venv /app/.venv
COPY --from=builder /app/main.pyc /app/main.pyc
COPY --from=builder /app/api /app/api
COPY --from=builder /app/config /app/config

# Set PATH to use venv binaries
ENV PATH="/app/.venv/bin:$PATH"
Expand Down
Loading
Loading