Skip to content

Commit 3603d3c

Browse files
author
ashokkumarkannan1
committed
Bringup XTTS-v2 model + track prims::view_of functionalization bug
Adds the XTTS-v2 (coqui/XTTS-v2) runner test_config entry and a CPU logic-equivalence test, and bumps tt_forge_models to the XTTS-v2 loader. The model compiles and runs end-to-end on TT (PCC 0.82 vs CPU) once the HiFiGAN decoder's no-op squeezes are rewritten as reshape. Without that workaround it fails FE compilation on prims::view_of, a tt functionalization limitation tracked in #5375. This branch intentionally omits the decoder workaround so the failure stays visible; config is marked KNOWN_FAILURE_XFAIL (FAILED_FE_COMPILATION) linking #5375. The CPU equivalence test (tests/torch/models/xtts_v2) confirms the custom forward matches native Xtts.inference at PCC 1.0, isolating the issue to the compiler.
1 parent f4c45f5 commit 3603d3c

4 files changed

Lines changed: 121 additions & 1 deletion

File tree

tests/runner/test_config/torch/test_config_inference_single_device.yaml

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -793,6 +793,12 @@ test_config:
793793
reason: "Aborted - https://github.com/tenstorrent/tt-mlir/issues/6786"
794794
bringup_status: FAILED_RUNTIME
795795

796+
xtts_v2/pytorch-XTTS_v2-single_device-inference:
797+
assert_pcc: false
798+
status: KNOWN_FAILURE_XFAIL
799+
reason: "FE compilation fails: HiFiGAN decoder's no-op squeeze lowers to prims::view_of, which the tt functionalization pass cannot handle - https://github.com/tenstorrent/tt-xla/issues/5375"
800+
bringup_status: FAILED_FE_COMPILATION
801+
796802
albert/token_classification/pytorch-Xlarge_v2-single_device-inference:
797803
status: EXPECTED_PASSING
798804

tests/torch/models/xtts_v2/__init__.py

Whitespace-only changes.
Lines changed: 114 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,114 @@
1+
# SPDX-FileCopyrightText: (c) 2026 Tenstorrent AI ULC
2+
#
3+
# SPDX-License-Identifier: Apache-2.0
4+
5+
"""XTTS-v2 — logic-equivalence test (CPU only).
6+
7+
The tt-xla bringup of XTTS-v2 does not run the model's native ``Xtts.inference``
8+
(its autoregressive ``gpt.generate`` sampling loop is not traced). Instead the
9+
loader precomputes ``gpt_codes`` and a custom monkey-patched ``forward`` runs the
10+
deterministic tail of inference (a single GPT forward to produce latents followed
11+
by the HiFiGAN decoder).
12+
13+
This test proves that custom decomposition is mathematically exact: it runs the
14+
original ``Xtts.inference`` (greedy / ``do_sample=False`` so it is deterministic),
15+
captures the exact ``gpt_codes`` produced by the internal generate step, feeds
16+
those same codes into the custom ``forward``, and asserts the two output
17+
waveforms have PCC ~= 1.0.
18+
19+
A PCC of ~1.0 here means any PCC gap observed on TT hardware is attributable to
20+
device numerical precision, not to a flaw in the bringup decomposition.
21+
22+
This runs entirely on CPU and requires the optional ``coqui-tts`` dependency.
23+
"""
24+
25+
import importlib.util
26+
27+
import pytest
28+
import torch
29+
30+
# Optional heavy dependencies; skip cleanly where they are absent. Check for the
31+
# TTS package without importing it: its import chain needs the isin_mps_friendly
32+
# shim that ModelLoader.load_model installs (importing TTS here would fail).
33+
pytest.importorskip("torchaudio")
34+
if importlib.util.find_spec("TTS") is None:
35+
pytest.skip("coqui-tts (TTS) not installed", allow_module_level=True)
36+
37+
from third_party.tt_forge_models.xtts_v2.pytorch import ModelLoader, ModelVariant
38+
39+
40+
def _pcc(a: torch.Tensor, b: torch.Tensor) -> float:
41+
"""Pearson correlation coefficient between two tensors (flattened)."""
42+
a = a.flatten().to(torch.float64)
43+
b = b.flatten().to(torch.float64)
44+
return torch.corrcoef(torch.stack([a, b]))[0, 1].item()
45+
46+
47+
@pytest.mark.model_test
48+
def test_custom_forward_matches_native_inference():
49+
torch.manual_seed(0)
50+
51+
loader = ModelLoader(variant=ModelVariant.XTTS_V2)
52+
model = loader.load_model() # float32, CPU; applies the monkey patches
53+
model.eval()
54+
55+
text = loader.DEFAULT_TEXT
56+
language = loader.DEFAULT_LANGUAGE
57+
speaker = model.speaker_manager.speakers[loader.DEFAULT_SPEAKER]
58+
gpt_cond_latent = speaker["gpt_cond_latent"]
59+
speaker_embedding = speaker["speaker_embedding"]
60+
61+
# --- 1. Native inference (deterministic), capturing the exact gpt_codes ---
62+
captured = {}
63+
original_generate = model.gpt.generate
64+
65+
def capturing_generate(*args, **kwargs):
66+
codes = original_generate(*args, **kwargs)
67+
captured["gpt_codes"] = codes
68+
return codes
69+
70+
model.gpt.generate = capturing_generate
71+
try:
72+
result = model.inference(
73+
text,
74+
language,
75+
gpt_cond_latent,
76+
speaker_embedding,
77+
do_sample=False,
78+
num_beams=1,
79+
temperature=0.75,
80+
top_k=50,
81+
top_p=0.85,
82+
length_penalty=1.0,
83+
repetition_penalty=10.0,
84+
)
85+
finally:
86+
model.gpt.generate = original_generate
87+
88+
wav_native = torch.as_tensor(result["wav"])
89+
gpt_codes = captured["gpt_codes"]
90+
assert gpt_codes is not None, "Native inference did not invoke gpt.generate"
91+
92+
# --- 2. Custom forward, fed the SAME gpt_codes produced above ---
93+
text_tokens = torch.IntTensor(
94+
model.tokenizer.encode(text.strip().lower(), lang=language)
95+
).unsqueeze(0)
96+
text_len = torch.tensor([text_tokens.shape[-1]])
97+
expected_output_len = torch.tensor(
98+
[gpt_codes.shape[-1] * model.gpt.code_stride_len]
99+
)
100+
101+
with torch.no_grad():
102+
wav_custom = model(
103+
text_tokens=text_tokens,
104+
text_len=text_len,
105+
gpt_codes=gpt_codes,
106+
expected_output_len=expected_output_len,
107+
gpt_cond_latent=gpt_cond_latent,
108+
speaker_embedding=speaker_embedding,
109+
)
110+
111+
# --- 3. The custom forward must reproduce native inference exactly ---
112+
pcc = _pcc(wav_native, wav_custom)
113+
print(f"native vs custom forward PCC: {pcc:.8f}")
114+
assert pcc >= 0.9999, f"Custom forward diverges from native inference: PCC={pcc}"

0 commit comments

Comments
 (0)