- Wire template into llama-server — Add
--chat-template-file ./template.jinja(and optionally--host/--port) tomod_llama/start.sh. - Run Heretic on Devstral weights — Use
server/hereticto produce a decensored GGUF. Update the model path instart.shif the output path differs.
- Implement
mod_whisper— Write integration code (shell or C++) that runswhisper.cpp'sstreambinary (ormainfor push-to-talk), captures mic input, and outputs transcribed text. This is the STT entry point. - Fix
mod_piper/start.sh— Replace the hardcoded"Good morning."test with the proper invocation: pass beanstalkd host/port as args tohamelinand pipe output toffplay. - Populate
mod_whisper/whisper.cppsubmodule — Rungit submodule update --init --recursiveto pull whisper.cpp source, then write abuild.shfor it.
- Write the orchestrator process — A daemon (Ruby, Python, or shell) that:
- Reads transcribed text from the
mod_whisperintegration - POSTs to
llama-server's/v1/chat/completions(OpenAI-compatible, maintains conversation history) - Pushes the LLM response text into beanstalkd's
ttstube forhamelinto consume
- Reads transcribed text from the
- Define beanstalkd host/port config — Standardize how host/port are passed across
piper-serve, the orchestrator, and any future modules (env vars or a shared config file).
- Create
ecosystem.config.js(PM2) — Define managed processes for the client:beanstalkd,hamelin(TTS), whisper listener, orchestrator. Optionally a separate server-side ecosystem file forllama-server. - Test full pipeline end-to-end — Mic → whisper → orchestrator → llama-server → beanstalkd → hamelin → speaker.