Feature request
Problem
We use annotation queues to build golden evaluation datasets — annotators review inputs, correct the outputs, and those corrected outputs feed into our eval dataset.
Our inputs contain HTML content that is difficult to review as raw markup in the current UI. The existing Custom Output Rendering feature would let us render this HTML beautifully, but it replaces the output panel, which is exactly where our annotators need to make edits.
This creates a tradeoff that makes the feature unusable for our workflow: we can either have readable inputs OR editable outputs, but not both.
Proposed Solution
Add support for Custom Input Rendering — a separate configuration (at the annotation queue, dataset, or tracing project level) that renders the input pane using a custom HTML page via postMessage, while leaving the output pane in its default editable state.
This could work similarly to the existing custom output rendering:
- Configure a URL for a hosted HTML page
- LangSmith sends the input data via postMessage
- The custom page renders it however the team needs
- The output side remains fully editable as normal
Motivation
Use Case
We're building evaluation datasets where:
- The input is an HTML document that needs to be viewed as rendered HTML (not raw markup)
- Annotators review the rendered input and then correct/edit the corresponding output
- The corrected output goes into the eval dataset
This likely applies to anyone working with rich input formats (HTML, markdown, structured documents, etc.) who also needs annotators to edit outputs.
Current Workaround
We can strip all the html structuring out and just display the text but that is still hard to read for reviewers
Feature request
Problem
We use annotation queues to build golden evaluation datasets — annotators review inputs, correct the outputs, and those corrected outputs feed into our eval dataset.
Our inputs contain HTML content that is difficult to review as raw markup in the current UI. The existing Custom Output Rendering feature would let us render this HTML beautifully, but it replaces the output panel, which is exactly where our annotators need to make edits.
This creates a tradeoff that makes the feature unusable for our workflow: we can either have readable inputs OR editable outputs, but not both.
Proposed Solution
Add support for Custom Input Rendering — a separate configuration (at the annotation queue, dataset, or tracing project level) that renders the input pane using a custom HTML page via postMessage, while leaving the output pane in its default editable state.
This could work similarly to the existing custom output rendering:
Motivation
Use Case
We're building evaluation datasets where:
This likely applies to anyone working with rich input formats (HTML, markdown, structured documents, etc.) who also needs annotators to edit outputs.
Current Workaround
We can strip all the html structuring out and just display the text but that is still hard to read for reviewers