I had a conversation with my own voice clone

I've wanted a voice clone that actually sounds like me for longer than I'd like to admit. The trouble is that I'm Irish, and almost every cloning attempt I've made has quietly turned me into an American. An Orpheus fine-tune did it. A zero-shot commercial clone (fed about an hour of my audio) did it too. The vowels drift west somewhere over the Atlantic and never come back.

So when the third attempt, Meorav, came out of training this week still sounding Irish, I wanted a proper test. Not a few sample sentences: a conversation. If it can hold up next to the real thing, line for line, how much harder is it to tell apart than I'd assumed?

The video shows who is talking: orange for me, blue for the clone. The audio below is the episode as it ran on the podcast.

What you're hearing

This is a ten-minute episode of My Weird Prompts, the AI podcast I run. Normally I send in a prompt and two AI hosts (Herman, a donkey, and Corn, a sloth) talk about it. This time there are no hosts. It's just me and the clone, talking about how much it sounds like me.

Every line on my side is me, recorded on my phone. Every line on its side is generated by Meorav v2.0, a text-to-speech model I fine-tuned on my own voice. Listening back is a strange experience. I can tell us apart, but it's subtle (subtle enough that I think it works better as a demo than I expected).

The model

Meorav is the text-to-speech line of Hebrish, my project on English–Hebrew speech. (The name is Hebrew for "mixed": it's what code-switching is, and it's also the Jerusalem mixed grill. Every Hebrish model line is named after Israeli food.) Version 2.0 is a full fine-tune of Alibaba's Qwen3-TTS-12Hz-1.7B-Base on 1,103 sentences I read aloud: about 88 minutes of my voice. Training was three epochs on a single L40S on Modal.

The weights are on Hugging Face at Hebrish/Meorav (access by request, free for non-commercial use), and the recordings it was trained on are in Hebrish-TTS-Voice-Recordings.

How the conversation was made

The clone can't hear me. That's the trick of the whole thing: the conversation was written in advance, and the two sides were recorded separately and stitched together.

1. Claude wrote the script: 103 lines, alternating between me and the clone.

2. The clone's 52 lines were rendered first, on an L4 GPU on Modal.

3. I read my 51 lines from a read sheet into my phone, in one take, without hearing the clone at all.

4. My take went through Whisper large-v3 with word timestamps. Aligning those words against the script gave the start and end of each of my lines.

5. Every clone render went back through the same speech-to-text model and was checked against the script. Fifty of the 52 came back right. One dropped its last sentence and one said "I never" instead of "I'd never", so those two were rendered again.

6. Each clip was level-matched, they were interleaved with short gaps, and one shared EQ and compressor went over both voices so neither sits in its own acoustic space.

The result sounds like a real exchange, even though neither side ever heard the other.

What gives it away

Two things, mostly. The first is the microphone: the clone learned from quiet-room recordings on a decent mic, and my side is a phone, so the clone often sounds like the better-recorded of the two. The second is the delivery. The training data is me reading sentences carefully, so the clone sounds like me on a good day. It has never heard me tired, or rummaging for my keys, or on the phone to the electricity company. The episode makes this point itself (the clone makes it, in fact).

Has anyone done this before?

Partly. Reid Hoffman interviewed a deepfake of himself in 2024, Peter Russell interviewed his own clone, and NPR's Planet Money put Robert Smith on a call with his synthetic voice. Every example I found used a commercial cloning service. I couldn't find one where the person had fine-tuned a voice model on their own recordings, then had a scripted conversation with it, with real takes cut in by speech recognition. I'm not going to claim it's the first. It's probably one of the first.

Compare every line yourself

I've put the whole thing in a Hugging Face Space: Me and My Voice Clone. Every one of the 103 lines is there as its own clip, with a blind mode that hides who's speaking, plus the full methodology, the alignment data and the scripts. There's also a write-up on ai.danielrosehill.com. The video version, with the two of us on screen taking turns, is at the top of this post.

The next version of Meorav needs messier data: me tired, me rambling, me with the filler words left in. Eighty-eight minutes of careful reading gets you a very good impression of a person on their best behaviour. It doesn't get you the person.