Matcha-TTS — LiteRT (on-device, FFT-free, GPU)

On-device English text-to-speech for Android via LiteRT CompiledModel. This is the FFT-free TTS lane: Matcha-TTS pairs a conditional flow-matching (CFM) acoustic model with a HiFi-GAN time-domain vocoder, so there is no FFT/iSTFT anywhere in the synthesis path. 22.05 kHz, LJSpeech voice.

Try it in your browser: john-rocky.github.io/litertjs-demos/matcha-tts — the same four .tflite files below running on LiteRT.js (text encoder + vocoder on WebGPU, decoder on WASM). Nothing to install; inference runs on your machine.

Converted from the official matcha_ljspeech + hifigan_T2_v1 checkpoints with litert-torch, re-authored to be ML-Drift-GPU-clean (per-graph tflite-vs-torch corr 1.000000; end-to-end waveform corr ≥0.99). fp16 weights.

Files

File	Size	In → Out	Delegate (Pixel 8a)
matcha_textenc_fp16.tflite	15 MB	emb[1,256,192] + mask[1,1,256] → mu[1,80,256], logw[1,1,256]	GPU
matcha_decoder_fp16.tflite	23 MB	x,mu[1,80,512] + t_sin[1,160] + mask[1,1,512] → v[1,80,512]	CPU¹
matcha_vocoder_fp16.tflite	29 MB	mel[1,80,512] → wav[1,1,131072]	GPU
dp_g2p_matcha_fp16.tflite	26 MB	text[1,96] (char ids) → logits[1,96,64] (IPA)	CPU
emb.bin	0.1 MB	phoneme embedding table (178×192 f32, host lookup)	host
g2p_dict.txt.gz	1.8 MB	275k-entry espeak-IPA dictionary (primary G2P)	host
config.json, g2p_meta.json	—	symbols, shapes, mel stats, G2P tokenizer tables	host

¹ The CFM decoder runs on the CompiledModel CPU delegate. It converts GPU-clean and is correct on CPU, but the Mali ML Drift GPU delegate mis-fuses the decoder's transformer blocks at large activation magnitude (the same block is correct as a standalone GPU graph, corr 0.984, but collapses to corr 0.006 fused — a graph-fusion bug, not a bad op). text encoder + vocoder run on the GPU; the GPU vocoder dominates wall time so the pipeline stays realtime (RTF ~0.8).

Pipeline (host orchestration)

text --G2P(CPU dict+neural)--> phoneme ids
     --host: embed + intersperse + pad-->     text_encoder(GPU) -> mu, logw
     --host: durations + length-regulator-->  mu_y[1,80,T]
     --host: Euler ODE loop (N steps)-->        decoder(CPU) x N -> v
     --host: denormalize-->                     vocoder(GPU)     -> waveform

Fixed shapes (256 phonemes, 512 mel frames ≈ 5.9 s); a runtime float mask makes padded positions a no-op so one compiled graph handles any length.

How to use

Android (Kotlin, LiteRT CompiledModel)

fun load(name: String, acc: Accelerator) =                    // models staged in filesDir
    CompiledModel.create(File(filesDir, name).absolutePath, CompiledModel.Options(acc), null)

val textenc = load("matcha_textenc_fp16.tflite", Accelerator.GPU)
val decoder = load("matcha_decoder_fp16.tflite", Accelerator.CPU)  // Mali mis-fuses this graph on GPU
val vocoder = load("matcha_vocoder_fp16.tflite", Accelerator.GPU)

val teIn = textenc.createInputBuffers(); val teOut = textenc.createOutputBuffers()
teIn[0].writeFloat(emb)     // [1,256,192] host phoneme-embedding lookup (emb.bin), blanks interspersed
teIn[1].writeFloat(tmask)   // [1,1,256]   1 = real phoneme position
textenc.run(teIn, teOut)    // -> mu[1,80,256], logw[1,1,256]
// host: durations ceil(exp(logw))·0.95 -> length-regulate mu -> mu_y[1,80,512]; 10 Euler steps of
// decoder(x, mu_y, t_sin[1,160], ymask[1,1,512]); mel = x·2.116101 − 5.536622 -> vocoder -> wav.
// Full pipeline: the text_to_speech (Matcha-TTS) sample in google-ai-edge/litert-samples.

Python (desktop verification)

import gzip, json, math, numpy as np, soundfile as sf
from ai_edge_litert.interpreter import Interpreter

MAXT, MAXM, LS = 256, 512, 0.95
cfg = json.load(open("config.json"))                     # symbols, mel stats, hop, sample rate
SYM = {s: i for i, s in enumerate(cfg["symbols"])}
DICT = dict(l.rstrip("\n").split("\t", 1) for l in
            gzip.open("g2p_dict.txt.gz", "rt", encoding="utf-8") if "\t" in l)
emb = np.fromfile("emb.bin", "<f4").reshape(178, 192)    # phoneme embedding table

def run(path, *ins):
    it = Interpreter(model_path=path); it.allocate_tensors()
    for d, x in zip(it.get_input_details(), ins): it.set_tensor(d["index"], x.astype(np.float32))
    it.invoke(); return [it.get_tensor(o["index"]) for o in it.get_output_details()]

# text -> espeak-IPA -> symbol ids (dictionary G2P; the neural OOV fallback is skipped here)
ipa = " ".join(DICT[w] for w in "the quick brown fox jumps over the lazy dog".split()) + "."
pids = [SYM[c] for c in ipa if c in SYM]

ids = np.zeros(MAXT, np.int64); ids[1:2 * len(pids):2] = pids   # intersperse blanks (id 0)
tmask = (np.arange(MAXT) < 2 * len(pids) + 1).astype(np.float32)[None, None]
mu, logw = sorted(run("matcha_textenc_fp16.tflite", emb[ids][None], tmask),
                  key=lambda a: -a.shape[1])                    # mu[1,80,256], logw[1,1,256]

w = np.ceil(np.exp(logw[0, 0]) * tmask[0, 0]) * LS              # durations -> length regulator
cum = np.cumsum(w); ylen = int(min(max(cum[-1], 1), MAXM))
mu_y = np.zeros((1, 80, MAXM), np.float32)
mu_y[0, :, :ylen] = mu[0][:, np.searchsorted(cum, np.arange(ylen), "right").clip(max=MAXT - 1)]
ymask = (np.arange(MAXM) < ylen).astype(np.float32)[None, None]

def t_sin(t, half=80):                                          # sinusoidal ODE-time embedding
    e = 1000.0 * t * np.exp(np.arange(half) * -math.log(10000) / (half - 1))
    return np.concatenate([np.sin(e), np.cos(e)]).astype(np.float32)[None]

x = np.zeros((1, 80, MAXM), np.float32)                         # Euler ODE, 10 steps
x[0, :, :ylen] = np.random.randn(80, ylen); N = 10
for k in range(N):
    x += run("matcha_decoder_fp16.tflite", x, mu_y, t_sin(k / N), ymask)[0] / N

mel = np.zeros_like(x); mel[0, :, :ylen] = x[0, :, :ylen] * cfg["mel_std"] + cfg["mel_mean"]
wav = run("matcha_vocoder_fp16.tflite", mel)[0].reshape(-1)[:ylen * cfg["hop"]]
sf.write("out.wav", np.clip(wav, -1, 1), cfg["sample_rate"])

G2P (espeak-free)

Matcha-LJSpeech is trained on espeak en-us IPA, but espeak is GPL. The clean replacement is a 275k-entry espeak-IPA dictionary (from OpenPhonemizer, Clear BSD) as primary + DeepPhonemizer (MIT) on LiteRT CPU for out-of-dictionary words. Output IPA maps 1:1 onto the keithito 178-symbol set.

Sample

See the LiteRT compiled_model_api/text_to_speech sample (Matcha-TTS) in google-ai-edge/litert-samples for the full Android app and the conversion scripts.

For the web, john-rocky/litertjs-demos has the full browser pipeline on LiteRT.js (G2P, length regulation, the Euler ODE loop, and playback), deployed at the Try-it link above.

Performance

Measured on an Apple M4 Max, CPU/XNNPACK at 8 threads, ai-edge-litert 2.1.6 — median of 15 warm runs per graph, with zero-filled inputs of each graph's declared static shape. Run-to-run spread stayed within 5%.

Graph	Calls per utterance	Warm median	First call
dp_g2p_matcha_fp16.tflite	1	7.2 ms	15.2 ms
matcha_textenc_fp16.tflite	1	14.4 ms	18.3 ms
matcha_decoder_fp16.tflite	10 (Euler ODE steps)	19.0 ms	30.5 ms
matcha_vocoder_fp16.tflite	1	691.0 ms	747.9 ms

Summing the graphs as the pipeline calls them — G2P, text encoder, ten decoder steps, vocoder — gives 903 ms of graph time for one 512-frame chunk, which is 5.94 s of audio at 22.05 kHz: RTF 0.15, about 6.6× faster than real time. Host orchestration (dictionary G2P lookup, duration/length regulation, mel denormalization) is not included; it is array bookkeeping, not inference. The vocoder is 77% of the total, so it is the graph to optimize.

For comparison, the card's Android figure is RTF ~0.8 on a Pixel 8a with the text encoder and vocoder on the GPU delegate and the decoder on CPU — a different device and a different backend split.

Android (Pixel 8a)

Android figures use the standard TFLite benchmark_model on a Pixel 8a (Tensor G3, Android 16) — 5 warm-up runs then 20 timed runs, CPU at 4 threads.

Graph	GPU (OpenCL)	CPU (XNNPACK, 4 threads)
matcha_textenc_fp16.tflite	did not run	36 ms
matcha_decoder_fp16.tflite	248 ms	189 ms
dp_g2p_matcha_fp16.tflite	did not run	25 ms
matcha_vocoder_fp16.tflite	1177 ms	5881 ms

2 of these graphs do not load on the OpenCL delegate at all, so the CPU column is the only Android number for them. The matcha_vocoder_fp16.tflite graph is the exception — it is the one place the GPU pays off here (1177 ms against 5881 ms).

Snapdragon NPU (Hexagon)

• matcha_textenc_fp16.tflite — the NPU is 1.49x faster than the GPU (7.15 ms against 10.69 ms) and loads 5.89x faster (152 ms against 899 ms).

• matcha_decoder_fp16.tflite — the NPU is 1.46x faster than the GPU (9.42 ms against 13.76 ms) and loads 10.04x faster (137 ms against 1373 ms).

• dp_g2p_matcha_fp16.tflite — the GPU delegate declines this graph on the S26: LiteRtException: Failed to compile model. The NPU runs it at 2.06 ms.

• matcha_vocoder_fp16.tflite — did not compile for the Hexagon NPU. The ahead-of-time compile for SM8850 failed, so it never reached the device and has no S26 number on either backend.

file	backend	inference (median / min)	load
matcha_textenc_fp16.tflite	NPU (Hexagon v81)	7.15 ms / 7.12 ms	152 ms
matcha_textenc_fp16.tflite	GPU (Adreno)	10.69 ms / 10.48 ms	899 ms
matcha_decoder_fp16.tflite	NPU (Hexagon v81)	9.42 ms / 9.35 ms	137 ms
matcha_decoder_fp16.tflite	GPU (Adreno)	13.76 ms / 13.24 ms	1373 ms
dp_g2p_matcha_fp16.tflite	NPU (Hexagon v81)	2.06 ms / 2.01 ms	124 ms

Measured on a Samsung Galaxy S26 (Snapdragon 8 Elite Gen 5 / SM8850, Hexagon v81, Android 16), LiteRT CompiledModel 2.2.0, one accelerator per process, 5 warm-up runs then N=50 timed runs, median reported. Every run held thermal status NONE throughout. Headroom 0.70-0.74, where 1.0 is the throttling threshold.

The NPU rows here ran artifacts compiled ahead of time for SM8850 with QAIRT 2.47.0; the GPU rows ran the published files as they are. LiteRT can also compile for the NPU on the device at first load, which is what lets you ship the published file unchanged — that path and the ten runtime libraries it needs are in the NPU recipe, and we did not measure it here. GPU wiring is in the GPU recipe.

License

Model: MIT (Matcha-TTS / HiFi-GAN). G2P dict: Clear BSD (OpenPhonemizer) + MIT (DeepPhonemizer).