We train our polish model in the open

MacWispr's dictation polish — the model that turns raw speech-to-text into clean, structured writing — is a 0.8B model post-trained by us, on a Mac Studio. The full pipeline, eval suite, reward code, and results (including the failures) are open source.

The recipe

1. SFT — supervised fine-tuning of Qwen3.5-0.8B on dictation-polish pairs (2,400 iterations, MLX on Apple Silicon).
2. DPO — preference training against flattened outputs. Honest result: it made the model worse (see below), so our pipeline now gates it out automatically.
3. GRPO — reinforcement learning where the reward is our open polish_verifier — the same rule-based scorer used in evals, so the model trains on exactly what we measure.
4. 4-bit quantization — shrinks the best checkpoint to ~450 MB so it ships inside the app and runs offline.

Honest numbers

Our out-of-distribution suite: 40 unseen dictations covering lists, mixed formats, emails, and “don't touch this” prose, scored by the shared verifier. Same prompts, same scorer, against Claude Sonnet via API:

Two cases behind a frontier cloud model, at 23× lower latency, with zero audio or text leaving your Mac. We're currently running a GRPO pass targeting the remaining gaps (multi-list and numbered lists) — results land in the repo when it finishes.

What we got wrong (and kept public)

Our DPO stage regressed the model from 55% to 48% because the preference pairs only taught “don't flatten lists” — and collapsed checklist and mixed-format skills as collateral. Our first GRPO attempt crashed on a dataset-format bug and the pipeline silently shipped the worse checkpoint. Both are documented in the run log, with the fixes: a verifier-driven reward and a gate that only promotes a checkpoint if it actually beats the previous best.