What 290 people with chronic illness told us about AI voices

To choose a voice for Juno, we tested 22 voices over six rounds with US adults living with long-term conditions. Here is every voice, how each one did, and what we learned.

Listen first

These are the three voices from our final check. Each one says the same three lines from a symptom check-in:

  1. “I've noted that you're feeling more tired today. When did that start?”
  2. “Was the headache mild, moderate, or severe?”
  3. “I've recorded the headache and tiredness for today. Would you like to add anything else?”

Play each voice, then pick the one you'd want. We'll show you which is which, and what 171 people chose.

Today's voice

58 of 171 picked it first · 5.6 out of 7

New voice S

73 of 171 picked it first · 5.7 out of 7

New voice V

40 of 171 picked it first · 5.5 out of 7

What we found

  1. No voice clearly beat today's.

    In the final check, the best new voice scored 5.7 out of 7 and today's voice 5.6. The gap was 0.1, with a likely range of −0.2 to +0.3. None of the three reached our bar of nine in ten people rating it 5 or higher: 84%, 83% and 81%.

  2. A single rating is mostly noise.

    Everyone in the final check heard their first voice again at the end. 76 of 171 gave it exactly the same score, and 139 were within one point. Even so, scores for the same voice varied by 1.2 points, almost as much as the 1.4 they varied between different voices. Only about a quarter of the spread looks like real taste.

  3. Today's voice sounded cold more than slow.

    People in our focus group had said our voice was slightly too slow. In the final check, only 26 of 171 people called it too slow, but 69 said it had too little warmth, against 46 for the best new voice. That gap is statistically clear (p < 0.01).

  4. Asking for warmth, or for speed, backfired.

    Voice G was made from the prompt “a warm, caring voice with a gentle smile in it, like a kind nurse checking in”. It came second to last in its round, and 14 of 24 people said it had too little warmth. One wrote: “She sounds so depressed.” Telling a stock voice to be warmer raised “too little warmth” from 6 to 11 of 24. Telling a voice to speak quicker made it faster, from 185 to 203 words a minute, and more people called it too slow: 15 of 24, against 10 for the same voice without the instruction.

  5. Robotic and slow were the top complaints.

    When people told us why they liked a voice least, the two most common reasons were that it sounded robotic or fake (56 of 271 reasons) or too slow (52).

  6. Tired doesn't mean slower.

    Of the 275 people who chose a voice both times, 232 (84%) picked the same voice for a day when they're exhausted as for an ordinary day. The 43 who switched moved to a quicker voice about as often as a slower one, 22 to 20.

  7. Words in a voice prompt don't always do what they say.

    Across 23 trial voices, the ones we described as low-pitched averaged 186 Hz, and the ones described as high-pitched averaged 184 Hz. In round 1, the voice we asked to be high came out low, and the one we asked to be low came out high. And in round 5, five voices made from exactly the same prompt scored anywhere from 4.4 to 5.8. One of them became our best new voice, and another was the least liked in its round.

Every voice we tested

All 22 voices, with the exact prompt behind each one. Eighteen were made with GPT-Live's prompt-to-voice, and two are OpenAI's stock voices, marin and gleam. Voices P and Q are earlier voices with one line added to their call instructions.

Results by round

Score is the average answer to “How suitable is this voice for an assistant you would use to record your symptoms?”, from 1 to 7. The other columns count people.

Round 1, 25 people

VoiceScoreRated 5+Picked firstLiked leastToo slowToo little warmth
Today's voice5.584%5107
E High, plain5.272%54712
D High, calm5.276%3558
C Low, bright5.268%24110
A Low, calm5.076%5358
B Low, plain4.960%14817
F High, bright4.652%431510

Round 2, 24 people

VoiceScoreRated 5+Picked firstLiked leastToo slowToo little warmth
Today's voice5.679%9116
H Friendly5.583%6046
J Conversational5.062%34613
E High, plain4.862%00713
D High, calm4.762%40312
G Warm4.446%27714
I Steady3.833%0121417

Round 3, 23 people

VoiceScoreRated 5+Picked firstLiked leastToo slowToo little warmth
H Friendly5.783%8062
Today's voice5.778%6405
N Gleam5.683%2124
J Conversational5.587%1275
K Friendly, long4.557%03411
M Grounded, long4.235%42210
L Bright, long3.843%110817

Round 4, 24 people

VoiceScoreRated 5+Picked firstLiked leastToo slowToo little warmth
S New draw6.092%7123
N Gleam5.883%5346
H Friendly5.888%12103
Today's voice5.892%4427
R New draw5.579%31312
Q Told warmer5.571%26711
P Told quicker5.275%121511

Round 5, 23 people

VoiceScoreRated 5+Picked firstLiked leastToo slowToo little warmth
S New draw5.891%5026
V New draw5.896%4165
Today's voice5.787%2045
N Gleam5.787%4136
H Friendly5.683%61106
W New draw5.274%13313
U New draw4.448%010216

Final check, 171 people

VoiceScoreRated 5+Picked firstLiked leastToo slowToo little warmth
S New draw5.784%73341946
Today's voice5.683%58532669
V New draw5.581%40845953

How the study worked

Who took part. US adults living with a long-term health condition, recruited on Prolific. Everyone was paid above Prolific's recommended hourly rate, between £11 and £17 an hour at the median time. People were told that short quotes from their written answers might be published. We counted answers from 290 people.

What they heard. Each voice said the same three lines from a symptom check-in, recorded through GPT-Live with the same call instructions (voices P and Q each had one line added). Voices played in a balanced order under neutral labels, so no voice was always first, and nobody knew which voice Juno uses today.

What they answered. For each voice, people rated how suitable it was for an assistant they'd use to record their symptoms, how easy it was to follow, and how respectful and how natural it sounded, each from 1 to 7. They said whether its speed was too slow or too fast, and whether it had too little or too much warmth. Then they picked a favourite, a least favourite, and a voice for a day when they're exhausted.

How it ran. Five rounds of about 24 people tested seven voices each. After each round we kept the best voices and rewrote the rest. Then a final check with 171 new people compared today's voice with the two best new ones. Everyone had to choose, and the first voice they heard played again at the end so we could measure how much a single rating moves.

How we compared voices. We compared each voice with today's using each person's own scores for both. In the final check, the likely ranges are wider because we made three comparisons at once. Head to head, the best new voice won 87 of 154 choices against today's voice (56%, likely 47% to 66%), which is not a clear win.

What this can't tell us. These were short clips, heard once, by people on an online research panel in the US. Every voice was feminine, with a General American accent. A real call is longer, and the voice is only part of it. That's why the next step is a test on real calls.

What we're doing next

We're keeping today's voice for now. Next, we'll test the best new voice against it on real calls with people calling Juno for the first time, and see which one brings more people back.

We also want Juno to adapt to how you're feeling. Most people want the same voice on a hard day, so we'll keep the voice and change how Juno speaks instead, starting with fewer words rather than a slower pace. This was our first study of voice, and we'll be running more.

Talk to Juno yourself.

Download

Juno: Chronic Illness Support is an app by SharedGenes, Inc. and is not affiliated with Juno Health (EHR software), Juno.com (email and internet service) or the film Juno.