Translation BenchmarkLLM Leaderboard
A systematic, LLM-as-judge evaluation of machine-translation quality across Asian &
ASEAN language pairs. Every model runs the same fixed corpus and is graded by a 3-judge
MQM panel — the whole leaderboard is published from committed
benchmark/results/*.json.
Methodology Updated March 26, 2026
3 Models
1 Language pairs
80 Sentences
8 Domains
MQM weights
40% accuracy
30% fluency
20% terminology
10% style
5-judge panel
claude-haiku-4.5 qwen3.5-flash-02-23 gemini-3.1-flash-lite-preview claude-sonnet-4.6 gemini-3-flash-preview
| # | Model | Composite | Accuracy | Fluency | Terminology | Style |
|---|---|---|---|---|---|---|
| 1 | MiMo-V2-Pro (Mar 26) Xiaomi | 92.1 | 92.8 | 91.8 | 91.4 | 91.6 |
| 2 | Gemma 3 12b (Mar 26) Google | 90.5 | 90.5 | 90.3 | 90.8 | 90.5 |
| 3 | Gemma 3 4b (Mar 25) Google | 88.5 | 89.5 | 88.4 | 86.2 | 89.6 |
EN → TH
| # | Model | Composite | Accuracy | Fluency | Terminology | Style |
|---|---|---|---|---|---|---|
| 1 | MiMo-V2-Pro (Mar 26) Xiaomi | 92.1 | 92.8 | 91.8 | 91.4 | 91.6 |
| 2 | Gemma 3 12b (Mar 26) Google | 90.5 | 90.5 | 90.3 | 90.8 | 90.5 |
| 3 | Gemma 3 4b (Mar 25) Google | 88.5 | 89.5 | 88.4 | 86.2 | 89.6 |
| Model | business | casual | ecommerce | idiomatic | legal | marketing | medical | technical |
|---|---|---|---|---|---|---|---|---|
| MiMo-V2-Pro (Mar 26) Xiaomi | 95.4 | 93.0 | 95.8 | 83.0 | 95.3 | 91.3 | 93.2 | 89.7 |
| Gemma 3 12b (Mar 26) Google | 90.7 | 89.3 | 90.1 | 92.8 | 87.8 | 91.2 | 88.3 | 93.8 |
| Gemma 3 4b (Mar 25) Google | 92.9 | 85.6 | 93.5 | 88.2 | 85.5 | 88.2 | 83.9 | 90.4 |
| # | Model | TTFT (ms) | Tokens/sec | Cost / 1K words | Quality |
|---|---|---|---|---|---|
| 1 | MiMo-V2-Pro (Mar 26) Xiaomi | 8423 | 55.3 | $0.0170 | 92.1 |
| 2 | Gemma 3 12b (Mar 26) Google | 681 | 40.9 | $0.0003 | 90.5 |
| 3 | Gemma 3 4b (Mar 25) Google | 702 | 39.4 | $0.0003 | 88.5 |