Tag

benchmarks

3 posts · all tags

Can image models spell the labels? A first look at my Tech Graphics Benchmark

23 Cheshvan 5787

Can image models spell the labels? A first look at my Tech Graphics Benchmark

Limited evaluation: 95 images from five cheap image models, 12 annotated so far. In this run, 10 of 11 had every label spelled exactly; the problems were composition, stretching and style. Not a verdict on any model.

evals benchmarks image-generation +2
Jachnun and Sabich: the first stable Hebrish models, in depth

10 Cheshvan 5787

Jachnun and Sabich: the first stable Hebrish models, in depth

A deep dive into the first stable Hebrish models, Jachnun v1.0 and Sabich v1.0: the data, the training recipe, the results against stock Parakeet and Whisper, the formats, and the full white paper.

hebrish speech-recognition code-switching +3
Can speech-to-text spell ‘makolet’? I tested seven models on Hebrish

4 Cheshvan 5787

Can speech-to-text spell ‘makolet’? I tested seven models on Hebrish

English speakers in Israel talk Hebrish, and dictation software mangles the Hebrew words. I built an open benchmark to measure it: my fine-tune gets two-thirds of them right, off-the-shelf models a third to under half.

benchmarks code-switching evals +5