Skip to content

add time to first token in OnnxDiscrepancyCheck#2535

Open
xadupre wants to merge 6 commits into
mainfrom
xadupre/tts
Open

add time to first token in OnnxDiscrepancyCheck#2535
xadupre wants to merge 6 commits into
mainfrom
xadupre/tts

Conversation

@xadupre

@xadupre xadupre commented Jun 22, 2026

Copy link
Copy Markdown
Member

Describe your changes

  • add metrics time to first token, time to first 5 tokens in OnnxDiscrepancyCheck

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Note

Copilot couldn't run its full agentic review because no GitHub Actions runner was available. Make sure your repository has a runner available to run Copilot's review, or add a copilot-setup-steps.yml file specifying one with the runs-on attribute. See the docs for more details.

Adds time-to-first-token and time-to-first-N-tokens latency metrics to OnnxDiscrepancyCheck’s generation comparison output, and updates tests accordingly.

Changes:

  • Introduces a new time_to_first_n_tokens config option and surfaces latency metrics for both Transformers and ORT GenAI generation.
  • Changes compare_generation to return a results dictionary (instead of an int) and updates result aggregation/logging.
  • Updates ONNX discrepancy check tests to validate the new return shape and metric keys.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated 4 comments.

File Description
test/passes/onnx/test_discrepancy_check.py Adjusts assertions for the new dict return type and validates presence of new latency metric fields.
olive/passes/onnx/discrepancy_check.py Adds latency measurement/reporting for TTFT/TTFN and changes generation comparison to return structured results.

Comment thread olive/passes/onnx/discrepancy_check.py Outdated
Comment thread olive/passes/onnx/discrepancy_check.py
Comment thread olive/passes/onnx/discrepancy_check.py Outdated
Comment thread test/passes/onnx/test_discrepancy_check.py
@xadupre xadupre marked this pull request as ready for review June 22, 2026 14:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants