Commit 269cae6
committed
feat(providers): wire missing media handlers + fix video routing + add CartesiaTTS
Closes the gaps reported in NEUROLINK_SDK_GAPS.md from a downstream
consumer (Director). After v9.65.0, several shipped capabilities could
not be invoked through nl.generate(...) because:
- package.json exports map exposed only 7 subpaths; internal modules
were blocked by Node ESM resolution.
- Music + Avatar handlers were registered only on ProviderRegistry's
lazy path, so consumers importing MusicProcessor / AvatarProcessor
directly saw an empty registry.
- FishAudioTTS shipped in dist/voice/providers/ but was never
registered with TTSProcessor.
- Top-level package re-exports surfaced only GoogleTTSHandler; every
other handler class was unreachable from import * as NL.
- handleVideoGeneration in baseProvider ignored output.video.provider
and always called generateVideoWithVertex, so Kling/Runway/Replicate
routing was dead.
- The Cartesia adapter in dist/ was a streaming WebSocket class, not a
TTSHandler, so tts.provider:"cartesia" failed.
Changes
-------
* package.json: add 11 subpaths (./voice, ./music, ./avatar, ./image-gen,
./hitl, ./evaluation, ./workflow, ./rag, ./files, ./processors,
./processors/*, ./adapters/*).
* Module-level auto-registration in voice/music/avatar barrels; every
shipped handler whose API key is in process.env registers itself at
import time. Idempotent with ProviderRegistry's existing block.
* New src/lib/voice/providers/CartesiaTTS.ts: synchronous TTSHandler
over Cartesia's /tts/bytes endpoint. CartesiaStream remains for the
realtime voice server.
* providerRegistry.ts: add fish-audio and cartesia registration blocks
alongside the other TTS handlers for architectural consistency.
* src/lib/index.ts: re-export every shipped TTS/STT/Realtime/Music/
Avatar/Video handler + ImageGenService + HITLManager + STTProcessor +
RealtimeProcessor + registerDefault* helpers.
* src/lib/types/{tts,music,avatar,multimodal}.ts: add TTSProviderName,
MusicProviderName, AvatarProviderName, VideoProviderName unions with
(string & {}) escape hatch.
* src/lib/core/baseProvider.ts: handleVideoGeneration now dispatches
through VideoProcessor.generate(provider, ...) honoring output.video
.provider; propagates real provider in logs + result.provider;
rejects unknown providers with VIDEO_ERROR_CODES.PROVIDER_NOT_SUPPORTED
instead of silently falling back to Vertex.
Tests
-----
* test/continuous-test-suite-tts.ts: registration assertions for
fish-audio + cartesia plus live e2e tests (real Cartesia API call
verified — 30 KB MP3 output).
* test/continuous-test-suite-media-gen.ts: video registration check,
unknown-provider rejection (regression guard for the silent-vertex-
fallback bug), per-provider routing tests for Kling/Runway/Replicate.
Each PASSes either via a successful generation or via the handler's
own typed error (which proves the dispatcher reached the right
handler).
Docs
----
* New docs/getting-started/providers/cartesia.md provider guide.
* docs/features/tts.md: env-setup, Supported Providers table, CLI
--tts-provider enum, and per-provider examples updated with Fish
Audio + Cartesia.
* docs/features/video-generation.md: four-provider compatibility table,
per-provider routing examples, and a Unknown Provider Behavior
subsection documenting the new PROVIDER_NOT_SUPPORTED guard.
* docs/getting-started/provider-setup.md: new Fish Audio + Cartesia TTS
configuration sections after Azure Speech.
* docs/getting-started/providers/index.md: Cartesia card under TTS plus
provider feature-matrix row.
* README.md: TTS provider count 4 -> 6; tts.provider union widened.
* CLAUDE.md: project-overview line widened to include Fish Audio,
Cartesia, and the media-gen handler set.
Verification
------------
- pnpm exec tsc --noEmit --strict: pass
- pnpm run lint: pass (0 errors, 15 pre-existing warnings)
- pnpm run build: pass; publint "All good!"
- pnpm run test:tts: 18/18 PASS (Fish Audio + Cartesia live e2e)
- pnpm run test:music: 12/12 PASS
- pnpm run test:avatar: 10/10 PASS
- pnpm run test:media: 23/23 PASS (Kling/Runway/Replicate routing)
- pnpm run test:bugfixes: 48/48 PASS
- pnpm run test:credentials: 15/15 PASS
- pnpm run test:unit: 7 sub-suites all PASS
- pnpm run test:ci: exit 0 (main + client 13/13 + hitl 4/4)1 parent 74d73cc commit 269cae6
52 files changed
Lines changed: 2120 additions & 103 deletions
File tree
- docs-site
- scripts
- static/docs/assets/images
- docs
- assets/images
- features
- getting-started
- providers
- visual-content/screenshots/cli-screenshots
- src/lib
- avatar
- core
- factories
- music
- types
- utils
- voice
- providers
- test
Some content is hidden
Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
176 | 176 | | |
177 | 177 | | |
178 | 178 | | |
| 179 | + | |
| 180 | + | |
| 181 | + | |
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
16 | 16 | | |
17 | 17 | | |
18 | 18 | | |
19 | | - | |
| 19 | + | |
20 | 20 | | |
21 | 21 | | |
22 | 22 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
43 | 43 | | |
44 | 44 | | |
45 | 45 | | |
46 | | - | |
| 46 | + | |
47 | 47 | | |
48 | 48 | | |
49 | 49 | | |
| |||
71 | 71 | | |
72 | 72 | | |
73 | 73 | | |
74 | | - | |
| 74 | + | |
75 | 75 | | |
76 | 76 | | |
77 | 77 | | |
| |||
955 | 955 | | |
956 | 956 | | |
957 | 957 | | |
958 | | - | |
959 | | - | |
960 | | - | |
961 | | - | |
962 | | - | |
963 | | - | |
964 | | - | |
965 | | - | |
966 | | - | |
967 | | - | |
| 958 | + | |
| 959 | + | |
| 960 | + | |
| 961 | + | |
| 962 | + | |
| 963 | + | |
| 964 | + | |
| 965 | + | |
| 966 | + | |
| 967 | + | |
968 | 968 | | |
969 | 969 | | |
970 | 970 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
1364 | 1364 | | |
1365 | 1365 | | |
1366 | 1366 | | |
| 1367 | + | |
1367 | 1368 | | |
1368 | 1369 | | |
1369 | 1370 | | |
| |||
Loading
Loading
Loading
Loading
Loading
0 commit comments