Skip to main content

Open models & benchmarks

Open simplifiers, measured in every language.

Two fine-tunes of Qwen3 cover 29 easy-language registers: 0.6B for live captioning and 1.7B for quality. Both use 4-bit weights, fit edge devices, and are available now. Every language clears the ship gate against stock Qwen3 at the same size.

Ship gate
29 / 29

Every language passes

Mean SARI · 0.6B
+14.9

52.7 vs 37.7 stock · 27 locales

Compliance · 0.6B
94%

vs 29% for stock Qwen3

Small beats big
22 / 27

0.6B specialist ≥ stock 1.7B on SARI

Two sizes

A live model and a quality model, on the same model line

One multilingual fine-tune at each size covers all 29 locales. The sizes trade latency for quality: the small model finishes a caption segment inside the live budget, and the larger model gives the reference quality when a few extra milliseconds do not matter.

Qwen3 0.6B · live

live

4-bit · ~331 MB · lowest latency

The model for real-time captions. It makes on-device simplification practical on a phone or a fanless laptop.

Qwen3-0.6B-EasyLanguage-4bit

Qwen3 1.7B · quality

quality

4-bit · ~934 MB · highest quality

The higher-fidelity option: stronger SARI and higher register compliance when the segment budget allows it. This is the reference model behind the figures below.

Qwen3-1.7B-EasyLanguage-4bit

Edge profile

Both artifacts are 4-bit MLX builds, about 331 MB and 934 MB, and run on the device. We tested the macOS app on an M1 (16 GB), M2 (96 GB), M3 (128 GB), and an A18 Pro (8 GB). The app sends no inference request, transcript, or audio sample off the device.

Head-to-head

The fine-tuned models hold the register

We compare each fine-tuned model with stock Qwen3 at the same size, the same 4-bit quantization, and the same held-out test set of about 200 sentences per language. That comparison isolates the effect of the fine-tune.

The baseline, and what the table leaves out

The baseline is stock Qwen3 at the same size, quantization, and prompt budget. The table leaves out proprietary API models because no public per-language easy-language benchmark exists for them. We publish measured numbers. The evaluation harness works with any model, so a reproducible proprietary baseline can join the table later.

Horizontal bar chart of SARI scores for fine-tuned EasyLanguage and stock Qwen3-1.7B across 27 languages that use whitespace-delimited scripts. The fine-tuned model scores higher in every language.
SARI scores for Qwen3-1.7B across 27 whitespace-delimited languages. We leave out Japanese and Simplified Chinese because whitespace tokenization makes SARI meaningless for those scripts; those locales use chrF, compliance, and language ID.
Horizontal bar chart of register-compliance percentages for fine-tuned and stock Qwen3-1.7B across all 29 languages. The fine-tuned model follows the easy-language rules in 84 to 100 percent of outputs.
Register compliance for Qwen3-1.7B across all 29 languages. The fine-tuned model follows the easy-language rules in 96% to 100% of outputs for most languages, while stock Qwen3 ranges from 60% to 96%. Hindi, the lowest at 84%, passes the gate and beats stock by 24 points.

0.6B · live

Mean SARI 52.7 vs 37.7 stock · mean compliance 94% vs 29% · language ID 100% · parrot rate 0% vs 10%.

1.7B · quality

Mean SARI 54.6 vs 49.6 stock · mean compliance 97% vs 74% · the 1.7B fine-tune beats the 0.6B fine-tune on SARI in all 27 whitespace-delimited languages.

Every language

The full held-out result, by locale

We report fine-tuned (ft) and stock Qwen3 results with greedy decoding on about 200 held-out sentences per language. SARI appears for both sizes; chrF, register compliance, and language ID appear for the 1.7B model. Higher is better in every column.

What SARI measures

SARI compares a model’s simplified sentence with a reference easy-language version. It gives credit for words the model kept, added, or deleted in the right places, and it penalizes changes that break the match. Scores run from 0 to 100, and higher is better. The number is not a percentage, and the absolute score depends on the test set, so read the gap between models on the same rows. The 0.6B fine-tune averages 52.7 against 37.7 for stock Qwen3, a 15-point gap. The 1.7B fine-tune averages 54.6 against 49.6, a 5-point gap. Those gaps mean the fine-tuned output lands closer to the reference easy-language rewrites. The compliance column shows the other half of the result: the 0.6B fine-tune follows the register rules in 94% of outputs, against 29% for stock.

Per-language evaluation of the Live Linguist 0.6B and 1.7B easy-language models against stock Qwen3 at the same size, on held-out test sets of about 200 sentences per language.
Language0.6B SARIft / stock1.7B SARIft / stock1.7B chrFft / stock1.7B complianceft / stock1.7B LIDft
VietnameseETR / ISO plain language60.2 / 41.362.5 / 57.360.9 / 54.687% / 51%100%
EnglishEasy / Plain English61.0 / 32.562.4 / 52.057.1 / 42.7100% / 68%100%
FrenchFALC58.0 / 33.362.0 / 54.155.6 / 45.9100% / 61%100%
SpanishLectura Fácil59.5 / 36.661.7 / 56.957.8 / 51.499% / 61%100%
Portuguese (PT)Leitura Fácil58.2 / 39.061.2 / 54.361.0 / 51.897% / 82%100%
Portuguese (BR)Leitura Fácil55.3 / 37.557.8 / 56.954.0 / 49.789% / 32%100%
ItalianEasy to Read54.0 / 49.256.2 / 49.551.9 / 40.689% / 48%100%
DanishLetlæst53.7 / 35.655.6 / 49.252.8 / 42.199% / 81%100%
DutchMakkelijk Lezen51.7 / 39.055.1 / 51.851.4 / 46.494% / 45%99%
SwedishLättläst53.6 / 40.255.0 / 49.557.9 / 48.296% / 89%98%
HindiEasy-to-Read51.9 / 35.254.5 / 49.247.6 / 40.285% / 60%98%
RussianEasy-to-Read51.1 / 41.154.5 / 51.051.5 / 43.797% / 73%100%
Norwegian (Bokmål)Lettlest51.8 / 30.054.3 / 41.456.2 / 38.199% / 96%100%
RomanianEasy-to-Read51.7 / 40.853.7 / 48.152.9 / 41.5100% / 93%100%
CroatianEasy-to-Read51.3 / 33.953.5 / 45.454.5 / 42.6100% / 86%100%
UkrainianEasy-to-Read52.1 / 39.653.1 / 51.854.8 / 51.4100% / 88%100%
BulgarianEasy-to-Read50.8 / 34.252.7 / 51.755.0 / 52.598% / 73%99%
ArabicEasy-to-Read (RTL)50.0 / 50.152.1 / 51.050.5 / 46.399% / 68%100%
PolishEasy-to-Read50.6 / 36.951.5 / 44.549.8 / 36.6100% / 83%100%
SlovakEasy-to-Read51.2 / 40.651.5 / 46.947.7 / 40.396% / 82%100%
GermanLeichte Sprache47.6 / 33.051.1 / 41.946.6 / 31.1100% / 60%100%
HungarianEasy-to-Read49.7 / 38.650.7 / 49.852.3 / 50.0100% / 82%100%
FinnishSelkokieli49.9 / 35.250.6 / 47.356.0 / 48.8100% / 91%100%
TurkishKolay Anlaşılır50.5 / 31.750.6 / 46.450.6 / 44.399% / 85%100%
CzechSnadné čtení49.4 / 39.550.4 / 48.649.2 / 45.9100% / 81%100%
KoreanEasy-to-Read49.2 / 43.850.1 / 46.542.9 / 33.798% / 88%100%
EstonianEasy-to-Read47.6 / 30.548.4 / 45.650.7 / 45.599% / 73%100%
Chinese (zh-CN)ISO plain languagen/an/a53.9 / 41.699% / 94%100%
JapaneseYasashii Nihongon/an/a45.5 / 38.899% / 93%100%

SARI

Simplification quality (Xu et al., 2016): a single score for words kept, added, and deleted relative to references.

chrF

Character n-gram F-score (Popović, 2015). It helps where whitespace tokenization fails, as it does for Japanese and Chinese.

Compliance

The share of outputs that pass the target register’s deterministic validators: sentence caps, simple grammar, anchoring, numbers, anti-parroting.

LID / parrot

LID is the share of outputs that stay in the target language. Parrot is the share that copies the input or a prompt example instead of rewriting.

Notes. Arabic ships on the 1.7B quality model, where it clears stock on SARI, compliance, and LID. At 0.6B its SARI sits level with stock. Japanese and Simplified Chinese ship on chrF, compliance, and LID because whitespace tokenization makes SARI meaningless for those scripts. Compliance and LID are validator rates, not human judgments.

On-device

Small enough for the edge, fast enough for live captions

The 0.6B model simplifies a caption segment inside the streaming budget on a phone or laptop. The 1.7B model gives the quality option for recorded material, offline work, and slower conversations. Both 4-bit MLX artifacts fit in local memory and need no network call.

0.6B artifact
~331 MB

4-bit MLX, live model

1.7B artifact
~934 MB

4-bit MLX, quality model

Device classes tested
4

M1, M2, M3, A18 Pro

Network calls
0

Inference stays local

A note on speed numbers

We publish model size and device coverage. We do not publish tokens-per-second figures we have not verified. Latency depends on the speech segment, the speech-recognition front end, and the device. The live-caption claim rests on the architecture: the small model is the one built to keep up, and the pipeline does not wait on a server round trip.

The dataset

Building the training data

The run-3 dataset holds about 74,000 supervised pairs across 29 locales (73,972 train), with about 200 held-out test sentences per language. German pairs come from German4All. We generated the remaining spoken-style pairs with a teacher model and checked their quality.

Every pair passed the same deterministic validators the app uses at runtime: sentence-length caps, simple-grammar checks, source anchoring, number fidelity, anti-parroting, and a wrong-language guard. The training signal reflects the register rules, so the model learns more than “shorter text.”

live-linguist-easylanguage-sft on Hugging Face

Limitations

  • Varying maturity. Data volume, standard confidence, and compliance differ across the 29 locales. The four founding languages are the most mature.
  • Small models. These are 0.6B and 1.7B parameters. They make mistakes, and the smaller one makes more.
  • Model-generated data. Much of the training data is model-generated. It can carry the teacher model’s biases and errors even after validator checks.
  • Standards are an interpretation. For many languages we map to the pan-European Easy-to-Read framework. That mapping is our reading of the standard; no national body certifies it.
  • Not for high-stakes use. Do not rely on these outputs for medical, legal, financial, or safety-critical communication. They support human review; they do not replace it.

Use, citation & ownership

Free for education, with citation samples and clear ownership

Dylan Goldblatt created Live Linguist’s free software, open-source datasets, and open-source models for educational purposes. They support language learners, educators, accessibility teams, and researchers. They do not replace a certified accessibility service, a medical device, or human easy-language review.

If you use the project in teaching, research, or a published build, attribute Dylan Goldblatt. The samples below cover the usual style guides; update the version and year when you cite a later release. For the models and dataset, cite the Hugging Face repository you used.

APA 7

Goldblatt, D. (2026). Live Linguist: On-device easy-language caption simplification (Version 3) [Computer software]. https://ndgold.com/live-linguist/

MLA 9

Goldblatt, Dylan. Live Linguist: On-Device Easy-Language Caption Simplification. Version 3, ndgold.com, 2026, ndgold.com/live-linguist/.

Chicago 17

Goldblatt, Dylan. Live Linguist: On-Device Easy-Language Caption Simplification. Version 3. Computer software. 2026. https://ndgold.com/live-linguist/.

IEEE

D. Goldblatt, "Live Linguist: On-device easy-language caption simplification," Version 3, 2026. [Online]. Available: https://ndgold.com/live-linguist/

BibTeX: project

@software{goldblatt2026livelinguist,
  author  = {Goldblatt, Dylan},
  title   = {Live Linguist: On-Device Easy-Language Caption Simplification},
  year    = {2026},
  version = {3},
  url     = {https://ndgold.com/live-linguist/},
  note    = {Open models and datasets for 29 easy-language locales}
}

BibTeX: quality model

@misc{goldblatt2026easylanguage,
  author    = {Goldblatt, Dylan},
  title     = {Qwen3-1.7B-EasyLanguage-4bit},
  year      = {2026},
  publisher = {Hugging Face},
  url       = {https://huggingface.co/ndgold/Qwen3-1.7B-EasyLanguage-4bit}
}

Ownership

Aquilops LLC holds the sole intellectual property in the closed-source applications. They cost nothing, and their source stays closed. © 2026 Aquilops LLC. All rights reserved.

The open models and datasets carry their own licenses: Apache-2.0 for the models, CC BY-SA 4.0 for the dataset. The Qwen3 base model and German4All source data keep their upstream licenses. Aquilops LLC owns the closed-source application, its name, and its icon.