Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
0e668f0
refactor(accent): replace greedy aligner with Needleman-Wunsch DP
torrid-fish May 21, 2026
8931d3e
feat(accent): regex reading-override layer + URL/non-JP preprocessing
torrid-fish May 21, 2026
bab83e8
feat(accent): add /MarkAccent/stream/ NDJSON endpoint + dev helpers
torrid-fish May 21, 2026
330c6bd
feat(accent): swap Yahoo MA for local fugashi + UniDic (refs #50)
torrid-fish May 21, 2026
0fa9214
docs(accent): extend package README for new package shape (refs #50)
torrid-fish May 21, 2026
8cf7a37
chore(spike): import spike artefacts — docs + probe scripts (refs #50)
torrid-fish May 21, 2026
83a476a
fix(accent): merge acronyms + decimals to keep morae aligned (refs #50)
torrid-fish May 21, 2026
97952cd
fix(accent): english_compound k=0 free, stop DP stealing morae (refs …
torrid-fish May 21, 2026
76d01b6
chore: drop Yahoo-era config/, .env, and spike scripts/docs (refs #50)
torrid-fish May 21, 2026
99c6623
feat(accent): symbol readings, okurigana subwords, scripts (refs #50)
torrid-fish May 21, 2026
64f54be
refactor(accent): drop merge_readable_symbol_compounds (refs #50)
torrid-fish May 21, 2026
e9f4ffe
fix(accent): preserve unit-compound furigana from toggle wipe (refs #50)
torrid-fish May 21, 2026
d9432b3
feat(accent): user patch table for OJAD/UniDic misreads (refs #50)
torrid-fish May 21, 2026
429f275
feat(accent): per-mora accent override in USER_PATCHES (refs #50)
torrid-fish May 21, 2026
436194e
refactor(accent): explicit per-mora int tuples in USER_PATCHES (refs …
torrid-fish May 21, 2026
bff2d49
fix(accent): free-consume override-synthesized tokens (refs #50)
torrid-fish May 21, 2026
aba22b9
fix(accent): clear katakana per-mora furigana when toggle off (refs #50)
torrid-fish May 21, 2026
a2c4188
docs(accent): rewrite api/accent/README.md in English (refs #50)
torrid-fish May 21, 2026
36dfbda
fix(docker): download UniDic dict at build to fix MarkAccent 500
torrid-fish May 27, 2026
a6ab1e3
chore: update ignore file and remove unused test scripts
torrid-fish May 29, 2026
e746364
chore: remove unused test
torrid-fish May 29, 2026
7bf63ec
docs: update root readme
torrid-fish May 29, 2026
c8f7811
feat: add version-selectable dict cwj and csj script
torrid-fish May 29, 2026
5b88135
fix(docker): install curl in builder stage for download_unidic.sh
torrid-fish Jun 4, 2026
6260235
fix(accent): propagate chunk warning through /MarkAccent/ aggregation
torrid-fish Jun 4, 2026
a68052c
fix(accent): make tagger singleton thread-safe with lru_cache
torrid-fish Jun 4, 2026
5b07a93
docs(accent): drop stray comment from USER_PATCHES schema table
torrid-fish Jun 4, 2026
eb72d9d
perf(accent): defer tokeniser-count debug log formatting
torrid-fish Jun 4, 2026
c509f70
fix(accent): cancel in-flight chunk tasks on client disconnect
torrid-fish Jun 4, 2026
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 0 additions & 2 deletions .env.example

This file was deleted.

9 changes: 8 additions & 1 deletion .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -216,4 +216,11 @@ __marimo__/
.streamlit/secrets.toml

# Secrest files
secret*
secret*

data/
output/

CLAUDE.md
AGENTS.md
.claude
24 changes: 19 additions & 5 deletions Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -2,20 +2,35 @@ FROM python:3.11-slim AS builder

WORKDIR /app

# Install uv for fast dependency management (pinned for reproducible builds)
RUN pip install --no-cache-dir "uv==0.9.0"
# Install uv for fast dependency management (pinned for reproducible builds),
# curl for downloading the NINJAL UniDic archive (scripts/download_unidic.sh
# line ~74), and unzip for extracting it (Deflate64-compressed).
RUN pip install --no-cache-dir "uv==0.9.0" \
&& apt-get update \
&& apt-get install -y --no-install-recommends curl unzip \
&& rm -rf /var/lib/apt/lists/*

# Copy dependency files first for better layer caching
COPY pyproject.toml uv.lock ./

# Install dependencies using uv
RUN uv sync --frozen --no-dev --no-install-project

# Download and install the UniDic dictionary into the venv so the image is
# self-contained — the `unidic` pip package ships the loader but not the
# dicdir, and fugashi.Tagger() fails at runtime without it.
# Uses scripts/download_unidic.sh so local dev and Docker share the same
# download logic. Kept as its own layer (before COPY . .) so app-code
# edits don't bust the download cache; only re-runs when the script or
# uv.lock changes.
COPY scripts/download_unidic.sh scripts/
RUN VIRTUAL_ENV=/app/.venv bash scripts/download_unidic.sh

COPY . .

# Compile only our app code (skip .venv) and drop the .py sources
RUN python -m compileall -b -q main.py api config \
&& find main.py api config -name "*.py" -delete
RUN python -m compileall -b -q main.py api \
&& find main.py api -name "*.py" -delete

FROM python:3.11-slim

Expand All @@ -30,7 +45,6 @@ WORKDIR /app
COPY --from=builder /app/.venv /app/.venv
COPY --from=builder /app/main.pyc /app/main.pyc
COPY --from=builder /app/api /app/api
COPY --from=builder /app/config /app/config

# Set PATH to use venv binaries
ENV PATH="/app/.venv/bin:$PATH"
Expand Down
Loading
Loading