Dictation that finally understands Hebrish, on your phone and offline

If you live in Israel and speak English, you know the sentence. "I have to go to Bituach Leumi tomorrow, and then pick up some bamba from the makolet." It's English, mostly. But every few words there's a Hebrew one, and no speech-to-text model I've used has ever got them right. Stock models hear whichever English words sound closest, and type those instead (with total confidence, naturally).

I dictate a lot (most of what I write starts out spoken), so this isn't a party trick for me. It's a daily irritation. I had a go at it with Whisper a while back. This time I wanted something smaller, faster and running entirely on my phone. So I fine-tuned NVIDIA's Parakeet. The first proof-of-concept version is now on Hugging Face, and this post is me sharing it.

Update, 15 October 2026: release 2 of the fine-tune is out (still built on Parakeet v2). Same links, new weights. It was trained on 120 minutes of code-switched speech (up from 77), and on a larger test set it more than halves stock Parakeet's error rate on code-switched speech (9.9% to 4.8%) and gets half of the Hebrew words it never saw in training right (stock v2: 29%), with no loss on plain English.

Direct downloads: Q8_0 GGUF (730 MB, recommended), Q4_K_M GGUF (475 MB), or the .nemo (2.5 GB). If you imported v1 into Offline Voice Input or Handy, delete it and import the new file; the app lists it as "Parakeet TDT 0.6B v2 English-Hebrew Code-Switching (v2)". The download links further down serve v2 too. The rest of this post describes v1.

🤗Hugging Face Model danielrosehill/Parakeet-TDT-0.6B-v2-English-Hebrew-Codeswitching A fine-tune of nvidia/parakeet-tdt-0.6b-v2 for English speech with Hebrew words dropped in, the way English speakers in Israel talk: "pick up bamba from the makolet", "I have an appointment at Misrad Hapnim tomorrow".… Automatic speech recognitionCC-BY-4.0Updated Oct 2026 View model on Hugging Face →

What it is

Parakeet-TDT-0.6B-v2-English-Hebrew-Codeswitching is a fine-tune of nvidia/parakeet-tdt-0.6b-v2, a 600-million-parameter English model that is, in my experience, the most accurate thing you can comfortably run on a phone. I picked v2 over the newer multilingual v3 deliberately: v3 adds 25 languages but gives up a bit of English accuracy, and my dictation is overwhelmingly English.

v2's tokeniser has no Hebrew characters at all, so the model can't output Hebrew script. What it can learn is a single romanised spelling for each Hebrew word: Bituach Leumi, Misrad Hapnim, tzaharon, Motzei Shabbat. Those canonical spellings come from the lexicon behind my Hebrew-Latin token classifier (which I wrote about here), so the transcripts are consistent rather than a different spelling every time.

Built for Android

The whole point is on-device inference on Android. The training audio was recorded on the same phone that will run the model: same microphone, same rooms, same way of holding it. I built a little phone web app in the style of Mozilla's Common Voice for that (read a sentence, record, review, next).

The target app is Offline Voice Input, an open-source Android voice keyboard that also registers as the system speech-to-text provider. It runs models through transcribe.cpp, which uses the GGUF format (the same ggml family as llama.cpp and whisper.cpp), and it lets you import your own GGUF file. So that's what I've published, in two quantisations:

• Q8_0 (8-bit, about 730 MB): download the Q8_0 GGUF. The safer choice if your phone has the memory.

• Q4_K_M (4-bit k-quant, about 475 MB): download the Q4_K_M GGUF. Smaller and lighter, at some cost in accuracy that I haven't measured yet.

Both were converted with transcribe.cpp v0.1.3, and the same files should also load in Handy on the desktop. Import one in Offline Voice Input and select it as the model.

The unquantised model is there too, as the original NeMo checkpoint: download the .nemo file (about 2.5 GB). That's the one to use on a GPU, or to convert and quantise yourself (to sherpa-onnx ONNX, for example). Loading it in Python is two lines:

import nemo.collections.asr as nemo_asr
model = nemo_asr.models.ASRModel.restore_from("Parakeet-TDT-0.6B-v2-English-Hebrew-Codeswitching.nemo")
print(model.transcribe(["audio_16k_mono.wav"])[0].text)

The data

The training data is public. The code-switching sentences are in English-Hebrew-Codeswitching-Sentences, and you can browse them in a little Space without downloading anything. They're a subset of a bigger dataset, Read-Aloud-Voice-Sentences, which also holds my plain English takes.

That English matters more than it sounds. Fine-tuning a model only on code-switched speech is a well-documented way to wreck its English (one study saw Whisper-small go from 3.4% to 13.3% error on LibriSpeech). So v1 was trained on roughly 70% code-switching and 30% English: 77 minutes of code-switched speech, 22 minutes of my own plain English, and 11 minutes of LibriSpeech so it doesn't forget what other people sound like. The run took 19 minutes on a single GPU.

Does it work?

For a first attempt on about an hour and a half of audio, better than I expected. The test set is held out by term: some Hebrew words appear only in test, never in training, which is the honest way to check whether the model has learned anything general.

Here's a real example, from the phone, that sums up the whole project better than any chart. I said: "I want to go to the makolet today to buy my groceries."

Stock Parakeet v2: "I want to go to the Micollette today to buy my groceries."

The fine-tune: "I want to go to the makolet today to buy my groceries."

The stock model did what stock models always do with a Hebrew word: it reached for something English-shaped (Micollette, which sounds like a French pastry, or possibly a French surname) and capitalised it for good measure. The fine-tune left the English alone and got the Hebrew word right, in the agreed spelling. That's code-switching handled correctly, and it's exactly the behaviour I was after.

• Word error rate on held-out code-switched speech: 9.5% down to 4.5%.

• Hebrew words the model never saw in training, transcribed correctly: 30% up to 55%.

• Hebrew words it did see in training, in new sentences: 45% up to 73%.

• Long recordings of about three minutes: 7.2% down to 4.1%.

• English didn't get worse: my plain English went from 2.9% to 2.5%, and LibriSpeech moved from 1.2% to 1.4% (noise, as far as I can tell).

The jump on unseen words is the bit I find most encouraging. It suggests the model is learning how my Hebrew maps onto the romanisation, not just memorising a word list. If you want to hear the difference rather than read about it, there's a side-by-side demo of stock v2 against the fine-tune.

How it compares with other models

Comparing a fine-tune with its own starting point only tells you so much, so I also built a small open benchmark that puts seven speech-to-text models through the same 84 Hebrish sentences. Release 2 of this model got 66% of the Hebrew words right. The best off-the-shelf model, Whisper Large V3, got 46%, and stock Parakeet v2 got 33%.

How many Hebrew words each model gets right: my fine-tune release 2 (built on Parakeet v2) 66%, release 1 62%, Whisper-Hebrish 54%, Whisper Large V3 46%, Whisper Large V3 Turbo 40%, stock Parakeet v2 33%, stock Parakeet v3 31%

The leaderboard has the word-by-word results and every transcript, and the benchmark write-up covers what it measures and what it can't tell you.

The caveats

This is a proof of concept, and I'd rather be clear about that than oversell it:

• It's a single-speaker model. It's trained on my voice, an English speaker originally from Ireland, living in Jerusalem. It may help other voices a little; I haven't tested that.

• The GGUF quantisations haven't been evaluated yet. The numbers above come from the full-precision model in PyTorch. How much Q8_0 and especially Q4_K_M lose is the next thing to measure, on the phone.

• Most Hebrew words were recorded only once or twice, so 55% on unseen words is still a coin flip with a slight lean.

• It uses one spelling convention, so it will prefer makolet to makolett whether or not you agree with me.

What's next

More recording, mainly. The baseline showed that stock Parakeet already handles about 165 of my 615 Hebrew terms (Shabbat, Pesach, Bamba and other words that have more or less become English). The other 390 it never gets right, so that's where recording is now focused. After that comes a proper comparison of English ratios, a full fine-tune at whichever mix wins, and an evaluation of the quantised files on the phone itself.

The model is CC-BY-4.0, like the base model, so do what you like with it. Progress is tracked in the project repository on GitHub. If you speak this particular mix of English and Hebrew and want to try it, I'd be curious to hear how it does with a voice that isn't mine.