Update: a 32 MB model for ASR paragraph breaks

A smaller, better successor to my Gemma 3 270M paragraph-break model is out: MiniLM-L12-ASR-Paragraph-Breaks.

It does the same job, adding paragraph breaks to flat speech-to-text output, but in a different way. The Gemma model *generated* the text again with breaks in it, so it needed a check in case it changed a word. This one is a token classifier: it decides, word by word, whether a new paragraph starts there. It can't change the text, so the fallback check goes.

Headline figures

On the same 26 held-out voice memos (290 hand-reviewed breaks) as the Gemma model:

• Break F1 0.85 (precision 0.86, recall 0.84), against 0.78 for Gemma and 0.65 for SaT in paragraph mode.

• 32 MB int8 ONNX, against 437 MB for the Gemma build.

• 0.65 s for a 340-word memo on one laptop CPU thread, where Gemma took 23 to 48 s.

• 33.2M parameters (MiniLM-L12-H384), MIT licence.

The int8 build differs from full precision on only 33 of 8,824 test words. Phone latency isn't measured yet, but at this size on-device use is realistic, and getting it onto phones was the reason for the rewrite.

Training data is the same ASR-Paragraph-Breaks dataset. For the whole story of why paragraph breaks are worth a model of their own, see the original post.