Commit 9c62a33
refactor: Vertical domain decomposition with FastAPI migration (#67)
* refactor: Phase 0 — FastAPI foundation with uvicorn, AppContext, EventBus, security
Pre-Phase 0: Fix queue polling loops in helpers.py — replace
time.sleep(5) polling with blocking queue.get(timeout=N) to
eliminate 0-5s latency per queued item.
Phase 0: Create the FastAPI app skeleton and core infrastructure:
- AppContext dataclass replacing 130+ module-level globals
- EventBus with thread-safe publish_sync() via call_soon_threadsafe
- JWT auth (PyJWT, HS256 pinned, generation counter for revocation)
- API key auth (factory pattern for 3 scopes)
- OPDS HTTP Basic auth with bcrypt migration support
- Global CSRF middleware (not per-route)
- Security headers middleware (CSP, HSTS, X-Frame-Options, etc.)
- Setup gate middleware for first-run setup
- DomainError exception hierarchy with app-level handlers
- Common utilities extracted from helpers.py (strings, dates, numbers, filesystem)
- FastAPI lifespan with CherryPy WSGI bridge via a2wsgi
- Uvicorn startup in Comicarr.py replacing CherryPy HTTP server
- Health check at /api/health
- Test context factory + 31 unit tests (all passing)
Dependencies added: fastapi, uvicorn, sse-starlette, PyJWT, a2wsgi,
python-multipart, httpx (dev)
CherryPy CSRF and setup gate tools disabled under WSGI bridge to
avoid double-checking with FastAPI middleware.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: Phase 1 — System domain (auth, SSE, config, admin)
Migrate auth, SSE, and system admin endpoints to FastAPI:
Backend:
- system/router.py: JWT login/logout, session check, setup, SSE stream,
config CRUD, shutdown/restart, version, logs, jobs endpoints
- system/service.py: Login verification with rate limiting and bcrypt
migration, config management, initial setup, scheduler info
- Login is sync def (bcrypt blocks event loop on ARM NAS hardware)
- SSE via sse-starlette with per-subscriber async queues
- JWT token revocation via generation counter
Frontend:
- api.ts: Switch AUTH_BASE to /api/auth, add apiRequest() for RESTful
endpoints, convert login/setup to JSON body (from form-encoded)
- useServerEvents.ts: Switch SSE endpoint to /api/events/stream,
update session check URL
14 new tests (301 total, all passing)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: Phase 2 — Metadata domain (ComicVine, Metron, MangaDex)
Migrate metadata endpoints to FastAPI:
- metadata/router.py: Search (comics + manga), comic/issue info,
artwork serving, series image lazy-load, metatag (single/bulk/group)
- metadata/service.py: Provider routing (CV/Metron/MangaDex), in_library
flag enrichment, pagination passthrough, metatag delegation
- metadata/queries.py: Cover image cache queries, comics needing images
15 new tests (316 total, all passing)
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: Phase 3 — Story Arcs domain (arcs, reading list, upcoming)
Migrate story arc CRUD, reading list, and upcoming endpoints from
CherryPy cmd= dispatch to FastAPI RESTful routes. Follows the
vertical domain pattern established in Phases 0-2.
Backend: storyarcs/queries.py (SQLAlchemy Core), service.py
(module-level functions), router.py (11 endpoints).
Frontend: useStoryArcs.ts and useQueue.ts switched from apiCall()
to apiRequest() for all migrated endpoints.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: Phase 4 — Series domain (comics, issues, imports)
Migrate series CRUD, issue management, wanted list, and import
endpoints from CherryPy cmd= dispatch to FastAPI RESTful routes.
Includes reusable paginated_query() helper for consistent pagination.
Backend: series/queries.py (SQLAlchemy Core with column projections),
service.py (12 functions), router.py (16 endpoints).
Frontend: useSeries.ts (8 hooks), useImport.ts (5 hooks), and
useQueue.ts (wanted + bulk queue/unqueue) switched to apiRequest().
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* feat: Phase 5 — Search domain (providers, RSS, manga)
Migrate comic/manga search, add-to-library, force search, and RSS
monitoring endpoints from CherryPy cmd= dispatch to FastAPI routes.
Wraps existing search.py and rsscheck.py engines.
Backend: search/queries.py, service.py (7 functions), router.py
(7 endpoints under /api/search).
Frontend: useSearch.ts (4 hooks) and useQueue.ts (forceSearch +
removed unused apiCall import) switched to apiRequest().
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* Save ideation doc
* feat: Phase 6 — Downloads domain (history, post-processing, DDL queue)
Migrate download history, post-processing, and DDL queue management
from CherryPy to FastAPI. Preserves PP_QUEUE integration and
ComicRN/APC compatibility for external script callbacks.
Backend: downloads/queries.py, service.py, router.py (7 endpoints
under /api/downloads).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* ci: Add workflow to trigger docs site deploy on push to main
Sends a POST to the Cloudflare Pages deploy hook so the comicarr-docs
static site rebuilds whenever this repo's main branch is updated.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* refactor: Phases 2-8 — Complete FastAPI migration, remove CherryPy
Finish the vertical domain decomposition plan:
- Extract all 101 functions from helpers.py into domain modules
(common/, core/security, series/, search/, downloads/, metadata/,
storyarcs/, system/) with helpers.py retained as re-export shim
- Create OPDS router with 12 Atom XML feed endpoints and HTTP Basic auth
- Add DDL download endpoints (POST /ddl, GET /file/{id})
- Merge /rest API endpoints into series domain router
- Delete webserve.py (12k lines), webstart.py, auth.py, opds.py, api.py
- Remove CherryPy, cheroot, portend, Mako, a2wsgi from dependencies
- Mount frontend SPA via CachedStaticFiles with content-hash caching
- Migrate all frontend hooks from apiCall() to apiRequest()
- Remove legacy API key query parameter auth (now JWT cookie-based)
- Update MSW test handlers for RESTful URL patterns
- Move LoginRateLimiter to core/security.py
- Fix all broken imports from deleted modules
All tests pass: 316 backend, 26 frontend. Linting clean. TypeScript clean.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: Resolve CI lint and test failures
- Fix import sorting in helpers.py, maintenance_webstart.py, webviewer.py
- Set comicarr.LOG_LEVEL in test_system_domain.py to prevent
TypeError when logger.info checks LOG_LEVEL > 0 in CI
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* style: Apply ruff format to all new/modified files
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* docs: Add BYOK AI features ideation document
Ideation for 8 BYOK AI features: collection intelligence dashboard,
natural language library search, story arc reading order generator,
weekly pull list curation, filename parsing fallback, search query
expansion, ComicInfo.xml metadata enrichment, and metadata conflict
reconciliation.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* chore: Add docs/ideation/ to .gitignore
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* style: Remove AI-generated comment slop from FastAPI migration files
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: Address all code review findings (P1-P3)
Security (P1):
- Add config write allowlist to prevent privilege escalation via PUT /api/config
- Change CSP from report-only to enforcing mode
- Remove /api from CSRF exempt prefixes (was exempting all endpoints)
- Add Depends(require_session) to SSE endpoint
- Fix path traversal: fail closed when no dirs configured, use commonpath
- Add path validation to OPDS file serving endpoints
- Fix datetime.utcnow() deprecation → datetime.now(timezone.utc)
- Fix remove_apikey broken loop logic
Performance (P2):
- Store ThreadPoolExecutor reference and shut down in lifespan teardown
- Replace OPDS N+1 queries with batch loads (recent, storyarc, readlist)
- Add 30-second TTL cache to havetotals()
- Fix OPDS XML escaping (escape in _entry_xml/_feed_xml, quoteattr for attrs)
- Deduplicate search logic (metadata delegates to search service)
- Extract shared paginated_query to core/database.py
Cleanup (P3):
- Remove dead code: multikeysort, checked, radio, urlretrieve (~99 LOC)
- Fix all 56 bare except: clauses → except Exception:
- Remove unused ctx parameter from 21 service functions + callers
- Extract _build_arc_summary to deduplicate storyarcs service
- Fix DDL_QUEUED type mismatch (list → set, append → add)
- Remove debug print(flipflop) from cleanhtml
- Simplify clean_url to url.strip()
All tests pass: 306 backend, 26 frontend. Lint clean. TypeScript clean.
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>1 parent b7ed820 commit 9c62a33
80 files changed
Lines changed: 14140 additions & 24649 deletions
File tree
- .github/workflows
- comicarr
- app
- common
- core
- downloads
- metadata
- opds
- search
- series
- storyarcs
- system
- docs/ideation
- frontend
- src
- components/data-table/cells
- contexts
- hooks
- lib
- types
- tests
- mocks
- unit/lib
- tests
- unit
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
| 1 | + | |
| 2 | + | |
| 3 | + | |
| 4 | + | |
| 5 | + | |
| 6 | + | |
| 7 | + | |
| 8 | + | |
| 9 | + | |
| 10 | + | |
| 11 | + | |
| 12 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
58 | 58 | | |
59 | 59 | | |
60 | 60 | | |
| 61 | + | |
61 | 62 | | |
62 | 63 | | |
63 | 64 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
134 | 134 | | |
135 | 135 | | |
136 | 136 | | |
137 | | - | |
138 | 137 | | |
139 | 138 | | |
140 | | - | |
141 | 139 | | |
142 | 140 | | |
143 | 141 | | |
| |||
476 | 474 | | |
477 | 475 | | |
478 | 476 | | |
479 | | - | |
480 | | - | |
| 477 | + | |
| 478 | + | |
| 479 | + | |
| 480 | + | |
| 481 | + | |
| 482 | + | |
481 | 483 | | |
482 | 484 | | |
483 | 485 | | |
| |||
541 | 543 | | |
542 | 544 | | |
543 | 545 | | |
544 | | - | |
545 | | - | |
| 546 | + | |
| 547 | + | |
| 548 | + | |
| 549 | + | |
| 550 | + | |
| 551 | + | |
546 | 552 | | |
547 | 553 | | |
548 | 554 | | |
| |||
552 | 558 | | |
553 | 559 | | |
554 | 560 | | |
555 | | - | |
556 | | - | |
557 | | - | |
558 | | - | |
559 | | - | |
560 | | - | |
561 | | - | |
562 | | - | |
563 | | - | |
564 | | - | |
565 | | - | |
566 | | - | |
567 | | - | |
568 | | - | |
569 | | - | |
570 | | - | |
571 | | - | |
572 | | - | |
573 | | - | |
574 | | - | |
575 | | - | |
576 | | - | |
577 | | - | |
578 | | - | |
579 | | - | |
580 | | - | |
581 | | - | |
582 | | - | |
583 | | - | |
584 | | - | |
585 | | - | |
586 | 561 | | |
587 | 562 | | |
588 | 563 | | |
| |||
591 | 566 | | |
592 | 567 | | |
593 | 568 | | |
594 | | - | |
595 | | - | |
596 | | - | |
597 | 569 | | |
598 | 570 | | |
599 | 571 | | |
| |||
610 | 582 | | |
611 | 583 | | |
612 | 584 | | |
613 | | - | |
614 | | - | |
615 | | - | |
616 | | - | |
617 | | - | |
618 | | - | |
619 | | - | |
620 | | - | |
621 | | - | |
622 | | - | |
623 | | - | |
624 | | - | |
625 | | - | |
626 | | - | |
627 | | - | |
628 | | - | |
629 | | - | |
630 | | - | |
| 585 | + | |
631 | 586 | | |
632 | | - | |
| 587 | + | |
| 588 | + | |
| 589 | + | |
| 590 | + | |
| 591 | + | |
| 592 | + | |
| 593 | + | |
| 594 | + | |
| 595 | + | |
| 596 | + | |
| 597 | + | |
| 598 | + | |
| 599 | + | |
| 600 | + | |
| 601 | + | |
| 602 | + | |
| 603 | + | |
| 604 | + | |
| 605 | + | |
| 606 | + | |
633 | 607 | | |
634 | 608 | | |
635 | 609 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
235 | 235 | | |
236 | 236 | | |
237 | 237 | | |
238 | | - | |
| 238 | + | |
239 | 239 | | |
240 | 240 | | |
241 | 241 | | |
| |||
0 commit comments