Skip to content
Open
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 8 additions & 3 deletions src/content/docs/docs/dotprompt.mdx
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ Greet a guest{{#if name}} named {{name}}{{/if}}{{#if style}} in the style of {{s

The portion in the triple-dashes is YAML front matter, similar to the front
matter format used by GitHub Markdown and Jekyll; the rest of the file is the
prompt, which can optionally use <a href="https://handlebarsjs.com/guide/" target="_blank" >Handlebars</a> templates. The following sections will go into more detail about each of
prompt, which can optionally use [Handlebars](https://handlebarsjs.com/guide/) templates. The following sections will go into more detail about each of
the parts that make a `.prompt` file and how to use them.

## Before you begin
Expand Down Expand Up @@ -1149,13 +1149,19 @@ response = await my_prompt({"input_args": "go here"}, opts={"tools": ["my_tool"]
The portion of a `.prompt` file that follows the front matter (if present) is
the prompt itself, which will be passed to the model. While this prompt could be
a simple text string, very often you will want to incorporate user input into
<<<<<<< Updated upstream
the prompt. To do so, you can specify your prompt using the

<a href="https://handlebarsjs.com/guide/" target="_blank">
Handlebars
</a>
templating language. Prompt templates can include placeholders that refer to the
values defined by your prompt's input schema.
=======
the prompt. To do so, you can specify your prompt using the [Handlebars](https://handlebarsjs.com/guide/)
templating language. Prompt templates can include placeholders that refer to the values defined by your prompt's input
schema.
>>>>>>> Stashed changes
Comment thread
ssbushi marked this conversation as resolved.
Outdated

You already saw this in action in the section on input and output schemas:

Expand Down Expand Up @@ -1232,8 +1238,7 @@ Invent a menu item for a {{#if theme}}{{theme}} themed{{/if}} restaurant.
In this example, the prompt renders as "Invent a menu item for a restaurant"
when the `theme` property is unspecified.

See the <a href="https://handlebarsjs.com/guide/builtin-helpers.html" target="_blank">Handlebars
documentation</a> for information on all of the built-in logical helpers.
See the [Handlebars documentation](https://handlebarsjs.com/guide/builtin-helpers.html) for information on all of the built-in logical helpers.

In addition to properties defined by your input schema, your templates can also
refer to values automatically defined by Genkit. The next few sections describe
Expand Down
10 changes: 7 additions & 3 deletions src/generate-llms-direct.ts
Original file line number Diff line number Diff line change
Expand Up @@ -291,6 +291,10 @@ function generateMainLlmsTxt(): string {
return content;
}

function escapeHtml(text: string): string {
return text.replace(/</g, '&lt;').replace(/>/g, '&gt;');
}
Comment on lines +294 to +296

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The escapeHtml function performs a global replacement of < and >. This will escape these characters even when they are part of valid code snippets (e.g., if (x < y) becomes if (x &lt; y)), which is generally undesirable for llms.txt files intended for LLM consumption. LLMs typically process Markdown and raw code better than HTML entities.

If the goal is to ensure the output doesn't contain raw HTML tags, it is better to convert those tags to Markdown in the source files (as seen in your changes to dotprompt.mdx) or use a more targeted approach to strip HTML tags during the content processing phase in src/utils/content-processor.ts.

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a stopgap for now.


export async function generateLlmsDirectly(): Promise<void> {
console.log('Generating llms.txt files directly from source files...');

Expand All @@ -307,13 +311,13 @@ export async function generateLlmsDirectly(): Promise<void> {

// Generate main llms.txt
const mainContent = generateMainLlmsTxt();
await writeFile(path.join(outputDir, 'llms.txt'), mainContent);
await writeFile(path.join(outputDir, 'llms.txt'), escapeHtml(mainContent));
console.log('Generated main llms.txt');

// Generate complete unfiltered documentation (llms-full.txt)
console.log('Generating complete unfiltered documentation...');
const fullContent = generateFullDocumentation(docs);
await writeFile(path.join(outputDir, 'llms-full.txt'), fullContent);
await writeFile(path.join(outputDir, 'llms-full.txt'), escapeHtml(fullContent));
console.log('Generated llms-full.txt');

// Generate language-specific complete documentation
Expand All @@ -322,7 +326,7 @@ export async function generateLlmsDirectly(): Promise<void> {
for (const lang of languages) {
console.log(`Generating complete documentation for ${lang}...`);
const content = generateLanguageSpecificContent(docs, lang);
await writeFile(path.join(outputDir, `llms-${lang}.txt`), content);
await writeFile(path.join(outputDir, `llms-${lang}.txt`), escapeHtml(content));
console.log(`Generated llms-${lang}.txt`);
}

Expand Down