fix(podcasts): restore the language instruction and stop the prompts handing the model a copyable skeleton

CHANGELOG.md

@@ -12,10 +12,13 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 - **Screen-by-screen reskin in the "Quiet Green" language** (the second redesign PR): the app shell gets the recomposed tri-hue pebble wordmark (fern/gold/teal — no red left), a fern spine on the active nav item and destination-hued nav icons; the notebook workspace gets display-type titles, panel headers with identity ticks (sources sage / notes gold / chat teal), one-line source-card metadata with the overflow menu at top-right, and de-washed chat bubbles (the AI speaks in teal accents, never washed backgrounds); the notebooks home separates compact recently-viewed rows from active-notebook cards; the sources table gets the library treatment (content-type pebbles, quiet embedded pills, hover that raises the surface); dialogs and the source viewer are flattened (no card-in-card boxes, uniform breathing room, no ⋮/close collision); and Models/Settings/Podcasts/Transformations/Ask-Search get the badge diet, quiet audio-player containers, mono for data and teal AI accents. Still purely visual — no behavior, columns, navigation or i18n changes
 
 ### Changed
+- Podcast generation failures now carry a hint that matches the failure: a placeholder/renamed speaker name, a `voice_id` its TTS model doesn't support, and a response that was truncated rather than swallowed by `<think>` tags each get their own explanation. The GPT-5 extended-thinking note used to be the only hint, so the two most common real failures got either nothing or advice about the wrong provider (#1238)
 - Community contribution intake now separates exploration from execution: feature requests, product/design/architecture ideas and contribution proposals start in GitHub Discussions, while Issues are reserved for reproducible bugs and maintainer-approved work items. The Issue chooser routes contributors accordingly, a structured Ideas Discussion form starts from user goals and outcomes, and the contributor/maintainer docs plus PR template now describe the Discussion → Issue → PR graduation path (#1204).
 - Release image gate gained a `probe` scenario (`make release-test` runs it as part of `all`): container-level checks that a Python test suite can't cover because they depend on the shipped image's process supervision — `OPEN_NOTEBOOK_WORKER_MAX_TASKS` reaching the in-image worker (the supervisord `sh -c` expansion), and the worker surviving startup with `HTTP_PROXY` set while a user's `NO_PROXY` value is preserved (the internal SurrealDB websocket not being tunneled). Both were manual probes during the v1.14.0 release; they now run automatically. Release-process docs gained the post-tag re-cut sequence and a note on never leaving the version bump uncommitted (v1.14.0 retro)
 
 ### Fixed
+- **Podcast episode profiles honor their `language` again.** The app's `prompts/podcast/{outline,transcript}.jinja` shadow podcast-creator's bundled templates (the library resolves `Path.cwd()/prompts/podcast/` before its own resources) and had lost the `{{ language }}` block, so a profile set to `he-IL` produced an English outline and English segment titles — the field looked supported and did nothing. A regression test now fails when a variable the bundled template uses is missing from the app's copy (#1238)
+- **A copied prompt placeholder no longer aborts a podcast mid-generation.** Both podcast templates showed the model a fill-in JSON skeleton (`"speaker": "[Actual Speaker Name]"`, a bare `...`) inside a ```json fence while also instructing it not to use fences; models returned the skeleton verbatim, which failed podcast-creator's speaker-name validation and discarded the outline and every segment already generated. The examples now carry the episode's real speaker names and written-out sample values, so a verbatim copy is valid output; placeholders and truncation are banned explicitly; and for a run with a `language` set no sample is shown at all, since the only sample that can be hard-coded is English (#1238)
 - **Ask answers are no longer silently truncated.** The three Ask stages (search strategy, per-search answers, final synthesis) were capped at 2000 output tokens, well below the 8192 that chat and transformations use. Token-dense languages such as Japanese hit the cap mid-sentence, and reasoning models spent the whole budget thinking and returned blank search terms, so Ask answered "no documents found" from a corpus that had the answer. All three stages now share an 8192 budget; a strategy with no usable search terms fails with an explicit error instead of running empty searches, and thinking-only partial answers are dropped before synthesis (#1221)
 - **Remote Crawl4AI servers that require a bearer token work again.** Crawl4AI Docker ≥ 0.9.0 rejects unauthenticated external connections by default, so pointing `CRAWL4AI_API_URL` at a current instance failed URL processing. content-core is bumped to 2.0.7, which sends `CRAWL4AI_API_TOKEN` as `Authorization: Bearer`; the variable is documented in the environment reference, `.env.example` and `docker-compose.yml`. No behavior change when the token is unset (#1269, lfnovo/content-core#80)
 - **Source Chat streaming no longer drops or corrupts tokens.** The SSE reader decoded each network chunk in isolation, so a `data:` line split across two chunks was silently discarded and a multibyte character straddling a chunk boundary rendered as `�`. The reader now keeps a carry-over buffer and streaming decoder, matching the pattern Ask already used (#1289)

commands/podcast_commands.py

@@ -41,6 +41,58 @@ def build_episode_output_dir(podcasts_folder: str = PODCASTS_FOLDER) -> tuple[st
     return episode_dir_name, output_dir
 
 
+def explain_generation_failure(error_msg: str) -> Optional[str]:
+    """Map a podcast-generation failure to an actionable hint, or None.
+
+    Ordered most specific first. The GPT-5 extended-thinking hint used to be
+    the only one, so the two most common real failures got either nothing
+    (`Invalid speaker name`) or advice about the wrong provider - a truncated
+    Gemini response was told to switch to gpt-4o (#1238).
+    """
+    if "Invalid speaker name" in error_msg:
+        return (
+            "The transcript model returned a speaker name that is not in the "
+            "speaker profile - usually a placeholder copied from the prompt "
+            'such as "..." rather than an invented person. Speaker names must '
+            "match the profile exactly. Retrying the episode often succeeds, "
+            "since each attempt is a fresh sample."
+        )
+
+    if "Voice name" in error_msg and "not supported" in error_msg:
+        return (
+            "The speaker profile's voice_id is not valid for its TTS model. "
+            "Check the voices in Settings -> Speaker Profiles against the ones "
+            "your voice model provides (the profiles seeded on install use "
+            "OpenAI voice names)."
+        )
+
+    if "Requested entity was not found" in error_msg:
+        return (
+            "Google returns this for any resource it cannot find, without "
+            "naming which one. Two candidates, likeliest first: a speaker "
+            "profile voice_id that its TTS model doesn't provide (the profiles "
+            "seeded on install use OpenAI voice names, which Gemini voice "
+            "models reject with exactly this message), or a model id in the "
+            "episode profile that doesn't exist for its provider. If the "
+            "transcript finished and the failure came during audio, it is the "
+            "voice."
+        )
+
+    if "Invalid json output" in error_msg or "Expecting value" in error_msg:
+        return (
+            "The model's response could not be parsed as JSON. Two common "
+            "causes: (1) the response was truncated - podcast-creator caps a "
+            "transcript segment at 5000 output tokens unless the episode "
+            "profile sets max_tokens, which is tight for long segments or "
+            "token-expensive languages, so raise max_tokens or use fewer and "
+            "shorter segments; (2) a model using extended thinking (e.g. "
+            "GPT-5) put all of its output inside <think> tags, leaving nothing "
+            "to parse - try gpt-4o, gpt-4o-mini or gpt-4-turbo instead."
+        )
+
+    return None
+
+
 class PodcastGenerationInput(CommandInput):
     episode_profile: str
     # Speaker profile record ID or name (the API boundary resolves the
@@ -357,19 +409,27 @@ async def generate_podcast_command(
             processing_time=processing_time,
         )
 
-    except ValueError:
-        raise
+    except ValueError as e:
+        # ValueError is the command layer's "permanent failure, do not retry"
+        # signal (retry config uses stop_on=[ValueError]), so the type has to
+        # survive - but LangChain's OutputParserException and
+        # json.JSONDecodeError are ValueError subclasses too. Every parser
+        # failure therefore left through here, past the hint mapper below:
+        # the placeholder speaker name and the truncated-JSON cases reached
+        # the user with no guidance at all (#1238).
+        hint = explain_generation_failure(str(e))
+        if not hint:
+            raise
+        logger.error(f"Podcast generation failed: {e}")
+        raise ValueError(f"{e}\n\nNOTE: {hint}") from e
 
     except Exception as e:
         logger.error(f"Podcast generation failed: {e}")
         logger.exception(e)
 
         error_msg = str(e)
-        if "Invalid json output" in error_msg or "Expecting value" in error_msg:
-            error_msg += (
-                "\n\nNOTE: This error commonly occurs with GPT-5 models that use extended thinking. "
-                "The model may be putting all output inside <think> tags, leaving nothing to parse. "
-                "Try using gpt-4o, gpt-4o-mini, or gpt-4-turbo instead in your episode profile."
-            )
+        hint = explain_generation_failure(error_msg)
+        if hint:
+            error_msg += f"\n\nNOTE: {hint}"
 
         raise RuntimeError(error_msg) from e

docs/7-DEVELOPMENT/podcasts.md

@@ -18,6 +18,16 @@ The legacy string fields (`tts_provider`, `outline_provider`, …) that predated
 
 `PodcastEpisode` stores `episode_profile` and `speaker_profile` as **dicts (snapshots)**, not references. Editing a profile never retroactively changes past episodes — that's intentional. Corollary: deleting a profile does not cascade to episodes.
 
+## Prompt templates shadow podcast-creator's
+
+`prompts/podcast/{outline,transcript}.jinja` are **not** just this app's copies of the library's prompts — they replace them. podcast-creator resolves templates as inline config → `prompts_dir` config → `Path.cwd()/prompts/podcast/<name>.jinja` → its own package resources, and this app configures only profiles, so the working directory wins and the bundled prompts are never read.
+
+Consequences to keep in mind when touching these files:
+
+- Library prompt improvements are invisible here. The `{{ language }}` block was lost exactly this way, which made `EpisodeProfile.language` silently do nothing (#1238). `tests/test_podcast_prompt_templates.py` fails when a variable the bundled template uses is missing from the app's copy.
+- The variables available are whatever `podcast_creator.nodes` passes: the transcript template gets `speaker_names`, the outline template does **not**.
+- Never show the model a fill-in skeleton it can return verbatim. Placeholders (`...`, `[like this]`, `<like this>`) are banned explicitly — a copied `"speaker": "..."` used to abort the whole episode on podcast-creator's speaker-name validation, discarding the segments already generated. The two examples differ: the **transcript** one carries the episode's real speaker names (serialized with `tojson`, so a name containing a quote can't break the example) plus fully written sample dialogue, while the **outline** one has fixed sample segment values — `speaker_names` isn't passed to that template and it has no dialogue. Both are labelled as two-entry excerpts and restate the required `turns` / `num_segments` count, so a copy is valid output without reading as a complete answer. **A sample can only be hard-coded English, so none is rendered at all when the episode profile sets a `language`** — the structure is stated in prose there, and the schema still reaches the model through `format_instructions`.
+
 ## Job lifecycle and the retry policy
 
 Generation runs as a `generate_podcast_command` job on the surreal-commands worker:

prompts/podcast/outline.jinja

@@ -25,7 +25,11 @@ The podcast will feature the following speakers:
   Personality: {{ speaker.personality }}
 {% endfor %}
 </speakers>
+{% if language %}
 
+IMPORTANT LANGUAGE INSTRUCTION: You MUST generate ALL content in {{ language }}. This includes segment names, descriptions, and all text in your response. Do not use English unless the content itself contains English terms. The entire output must be written in {{ language }}.
+
+{% endif %}
 Please create an outline based on this briefing. Your outline should consist of {{ num_segments }} main segments for the podcast episode, along with a description of each segment. Follow these guidelines:
 
 1. Read the briefing carefully and identify the main topics and themes.
@@ -37,30 +41,19 @@ Please create an outline based on this briefing. Your outline should consist of
 7. This is a whole podcast so no need to reintroduce speakers or topics on each segment. Segments are just markers for us to know to change the topics, nothing else.
 8. Include an introduction segment at the beginning and a conclusion or wrap-up segment at the end.
 
-Format your outline using the following structure:
+Return exactly one JSON object with a single root key "segments" whose value is a list of exactly {{ num_segments }} entries. Each entry has exactly three keys: "name" and "description" (strings) and "size".
+{% if not language %}
+The line below is a two-entry EXCERPT showing that structure with sample values - keep the structure, write your own segments from the briefing, and return all {{ num_segments }} of them rather than the two shown:
+
+{"segments": [{"name": "Setting the scene", "description": "Introduce the subject and the questions this episode sets out to answer.", "size": "short"}, {"name": "Working through the detail", "description": "Take the main points from the briefing in turn, with the specifics that matter most.", "size": "medium"}]}
+{% endif %}
 
-```json
-{
-    "segments": [
-        {
-            "name": "[Segment Name]",
-            "description": "[Description of the segment content]",
-            "size": "short"
-        },
-        {
-            "name": "[Segment Name]",
-            "description": "[Description of the segment content]",
-            "size": "medium"
-        },
-        {
-            "name": "[Segment Name]",
-            "description": "[Description of the segment content]",
-            "size": "long"
-        },
-    ...
-    ]
-}
-```
+   - "size" must be exactly one of "short", "medium" or "long".
+{%- if not language %}
+   - The sample values above show the structure only - write segments drawn from the briefing instead of reusing them.
+{%- endif %}
+   - Never emit placeholder or elided content: no "..." or "…", no "[like this]", no "<like this>", no "TODO", no empty strings, no trailing commas.
+   - Write out all {{ num_segments }} segments in full; never shorten or truncate the list.
 
 Formatting instructions:
 {{ format_instructions}}
@@ -76,8 +69,6 @@ IMPORTANT OUTPUT FORMAT:
 - If you use extended thinking with <think> tags, put ALL your reasoning inside <think></think> tags
 - Put the final JSON output OUTSIDE and AFTER any <think> tags
 - Do NOT wrap the JSON in ```json code blocks - return the raw JSON object only
-- Example correct format:
-  <think>Let me analyze the briefing...</think>
-  {"segments": [...]}
+- Correct format: any reasoning inside <think></think> tags, then the JSON object described above, and nothing else
 
 Please provide your outline now, following the format and guidelines provided above.

prompts/podcast/transcript.jinja

@@ -27,7 +27,11 @@ The podcast features the following speakers:
   Personality: {{ speaker.personality }}
 {% endfor %}
 </speakers>
+{% if language %}
 
+IMPORTANT LANGUAGE INSTRUCTION: You MUST generate ALL dialogue and content in {{ language }}. Every speaker's dialogue must be written entirely in {{ language }}. Do not use English unless quoting specific English terms. The entire transcript must be in {{ language }}.
+
+{% endif %}
 Next, examine the outline produced by our director:
 <outline>
 {{ outline }}
@@ -74,17 +78,24 @@ Follow these format requirements strictly:
 {% endif %}
 
 
-```json
-{
-    "transcript": [
-        {
-            "speaker": "[Actual Speaker Name]",
-            "dialogue": "[Speaker's dialogue based on their personality and expertise]"
-        },
-    ...
-    ]
-}
-```
+Return exactly one JSON object with a single root key "transcript" whose value is a list of at least {{ turns }} entries. Each entry has exactly two keys, "speaker" and "dialogue", both strings.
+{% if not language %}
+The line below is a {% if speakers|length == 1 %}one-entry{% else %}two-entry{% endif %} EXCERPT showing that structure with sample dialogue - keep the structure, write your own words for this segment, and return at least {{ turns }} entries rather than the {% if speakers|length == 1 %}one{% else %}two{% endif %} shown:
+
+{% if speakers|length == 1 %}
+{"transcript": [{"speaker": {{ speaker_names[0]|tojson }}, "dialogue": "Let's pick up where we left off, because this is where the material really starts to come together."}]}
+{% else %}
+{"transcript": [{"speaker": {{ speaker_names[0]|tojson }}, "dialogue": "Let's pick up where we left off, because this is where the material really starts to come together."}, {"speaker": {{ (speaker_names[1] if speaker_names|length > 1 else speaker_names[0])|tojson }}, "dialogue": "Agreed, and the detail I keep coming back to is the one that changes how you read everything before it."}]}
+{% endif %}
+{% endif %}
+
+   - Every "speaker" value must be copied character-for-character from this list: {{ speaker_names|join(', ') }}
+   - Every "dialogue" value must be the finished words the speaker says out loud; it is sent straight to a text-to-speech engine.
+{%- if not language %}
+   - The sample dialogue above shows the structure only - write dialogue for this segment instead of reusing it.
+{%- endif %}
+   - Never emit placeholder or elided content: no "..." or "…", no "[like this]", no "<like this>", no "TODO", no empty strings, no trailing commas.
+   - Never shorten or truncate the list. Write out every entry in full.
 
 Formatting instructions:
 {{ format_instructions}}
@@ -119,9 +130,7 @@ IMPORTANT OUTPUT FORMAT:
 - If you use extended thinking with <think> tags, put ALL your reasoning inside <think></think> tags
 - Put the final JSON output OUTSIDE and AFTER any <think> tags
 - Do NOT wrap the JSON in ```json code blocks - return the raw JSON object only
-- Example correct format:
-  <think>Let me plan the dialogue...</think>
-  {"transcript": [...]}
+- Correct format: any reasoning inside <think></think> tags, then the JSON object described above, and nothing else
 
 When you're ready, provide the transcript.
 {% if speakers|length == 1 %}