Benchmarks and evals — measuring models against real workloads.https://blog.danielrosehill.com/topics/technology/ai/evaluations/en-ushttps://blog.danielrosehill.com/posts/tech-graphics-benchmark-image-models-exact-text-first-look-i1f860/https://blog.danielrosehill.com/posts/tech-graphics-benchmark-image-models-exact-text-first-look-i1f860/Limited evaluation: 95 images from five cheap image models, 12 annotated so far. In this run, 10 of 11 had every label spelled exactly; the problems were composition, stretching and style. Not a verdict on any model.Tue, 03 Nov 2026 12:00:00 GMTTechnologyAIEvaluationshttps://blog.danielrosehill.com/posts/boxty-v1-english-dictation-without-hebrew-mhsv9w/https://blog.danielrosehill.com/posts/boxty-v1-english-dictation-without-hebrew-mhsv9w/Boxty is the no-Hebrew sibling of my Jachnun code-switching model: Parakeet fine-tuned on my voice for plain English and tech dictation. Tech WER drops from 2.51% (stock) to 1.35%, and general English holds.Tue, 20 Oct 2026 18:00:00 GMTTechnologyAIEvaluationshttps://blog.danielrosehill.com/posts/llm-conspiracy-endorsement-eval-first-look-she8dc/https://blog.danielrosehill.com/posts/llm-conspiracy-endorsement-eval-first-look-she8dc/A small, hand-graded first look at my conspiracy-endorsement eval: 7 prompts, 7 models, one sample each. In this run two uncensored models helped a believer and an invented theory tripped up two more. Not a verdict on any model.Tue, 20 Oct 2026 12:00:00 GMTTechnologyAIEvaluationshttps://blog.danielrosehill.com/posts/update-whisper-hebrish-small-acft-futo/https://blog.danielrosehill.com/posts/update-whisper-hebrish-small-acft-futo/A phone-sized Whisper Hebrish for FUTO Keyboard on Android. It halves the stock English model's error rate on English-with-Hebrew dictation (11.8% to 5.8% WER) and imports as a single 264 MB file.Sat, 17 Oct 2026 16:00:00 GMTTechnologyAIEvaluationshttps://blog.danielrosehill.com/posts/update-whisper-hebrish-v2-released/https://blog.danielrosehill.com/posts/update-whisper-hebrish-v2-released/Version 2 of Whisper Hebrish, my Whisper fine-tune for English with Hebrew words mixed in. Code-switching WER drops from 14.7% (stock turbo) to 6.7%, and there are whisper.cpp builds.Sat, 17 Oct 2026 14:30:00 GMTTechnologyAIEvaluationshttps://blog.danielrosehill.com/posts/update-inventory-spec-extractor/https://blog.danielrosehill.com/posts/update-inventory-spec-extractor/A small token-classification model that pulls voltages, connectors, materials and 22 other attributes out of one line of a product spec. Runs on CPU in about 20 ms per line. F1 0.85 against a regex baseline's 0.58.Thu, 15 Oct 2026 18:00:00 GMTTechnologyAIEvaluationshttps://blog.danielrosehill.com/posts/english-hebrew-code-switching-asr-benchmark/https://blog.danielrosehill.com/posts/english-hebrew-code-switching-asr-benchmark/English speakers in Israel talk Hebrish, and dictation software mangles the Hebrew words. I built an open benchmark to measure it: my fine-tune gets two-thirds of them right, off-the-shelf models a third to under half.Thu, 15 Oct 2026 15:00:00 GMTTechnologyAIEvaluationshttps://blog.danielrosehill.com/posts/audio-multimodal-llm-bitrate-eval/https://blog.danielrosehill.com/posts/audio-multimodal-llm-bitrate-eval/An eval across 12 audio-capable models on OpenRouter, 5 MP3 bitrates, 4 dictation samples. The default upload quality in most dictation apps is 2–4× overprovisioned — and the more interesting finding had nothing to do with bitrate.Fri, 17 Apr 2026 12:00:00 GMTTechnologyAIEvaluationshttps://blog.danielrosehill.com/posts/bias-censorship-eval-tests-llms/https://blog.danielrosehill.com/posts/bias-censorship-eval-tests-llms/Test prompts and notes exploring cultural bias, censorship, and worldview differences across Western and Eastern large language models.Wed, 25 Mar 2026 21:00:00 GMTTechnologyAIEvaluations