> ## Documentation Index
> Fetch the complete documentation index at: https://docs.usepatchwork.co/llms.txt
> Use this file to discover all available pages before exploring further.

# Voice

> Let people speak to your agent and hear it answer. You render the mic button; we own the pipeline.

Voice is two things, and it is worth being precise about which two.

* **Dictation.** Capture an utterance, get the text back. You decide what to do with it.
* **Read-aloud.** Hand us text, get audio back. You decide what gets spoken and when.

**It is not a spoken loop.** Nothing sends on its own and nothing speaks on its own. `stop()` returns a transcript — it does not submit it, and the agent's reply stays silent unless you call `speak()`. Build on those two primitives rather than on continuous conversation.

The division of labour is the same as everywhere else in Patchwork: **your app renders the interface, we run the pipeline.** Provider keys are ours, the browser only ships and plays audio.

## Turn it on

```tsx theme={null}
const { send, voice } = useAgent({ voice: true });
```

That is the whole setup. `voice` is a single object rather than ten members on the hook, and it does nothing until you call something on it.

## A working mic button

```tsx theme={null}
import { useAgent } from "@usepatchwork/react";

export function MicButton() {
  const { send, voice } = useAgent({ voice: true });

  async function toggle() {
    if (!voice.isListening) {
      await voice.start();
      return;
    }

    const text = await voice.stop();
    if (text) await send(text);
  }

  return (
    <button onClick={toggle} disabled={voice.isTranscribing}>
      {voice.isListening ? "Stop" : "Speak"}
    </button>
  );
}
```

`start()` asks the browser for the microphone, so it must be called from a user gesture. `stop()` ends the take and resolves with the transcript — **empty if the take was silent, cancelled, or failed**, which is why the example checks before sending. Telling those three apart is [below](#how-errors-reach-you).

`cancel()` throws the take away. Use it for an escape key or a swipe-away, so a mistake costs nothing.

## Reading a reply aloud

```tsx theme={null}
const { messages, voice } = useAgent({ voice: true });
const last = messages.at(-1);

<button
  onClick={() => last && voice.speak(last.content, last.id)}
  disabled={voice.isSpeaking}
>
  Read aloud
</button>;
```

The second argument is an id you choose. It comes back as `voice.speakingId` while that clip is sounding, which is how you show a playing state on the right message rather than on all of them:

```tsx theme={null}
{messages.map((message) => (
  <Message
    key={message.id}
    message={message}
    isSpeaking={voice.speakingId === message.id}
  />
))}
```

`stopSpeaking()` cuts playback short.

<Note>
  Replies you have already spoken are cached for the session, so replaying one costs nothing and returns instantly.
</Note>

### Markdown is handled

Agents write markdown, and markdown read literally sounds like punctuation. We render it into something worth hearing before synthesis — you do not need to strip it yourself.

## Drawing the microphone

`waveformRef` attaches a canvas that we draw the live input into while recording:

```tsx theme={null}
<canvas ref={voice.waveformRef} width={160} height={32} />
```

Attach it whenever you like; it idles until `start()`.

## Picking a voice

Pass `voiceId` to use a specific synthesizer voice. Omit it and the platform default applies.

```tsx theme={null}
useAgent({ voice: true, voiceId: "your-voice-id" });
```

To build a picker, list what is available:

```tsx theme={null}
const client = usePatchworkClient();
const voices = await client.listVoices();
```

Each entry is `{ id, name, category?, preview_url? }`. `preview_url` is a sample you can play in the picker itself.

## The state you render against

| Field | Type | Means |
| - | - | - |
| `isListening` | `boolean` | the microphone is capturing |
| `isTranscribing` | `boolean` | the take ended, text is on its way |
| `isSpeaking` | `boolean` | audio is playing |
| `speakingId` | `string \| null` | which `speak()` call is sounding |

`isTranscribing` is the one people forget. There is a gap between releasing the button and the text arriving, and without a state for it the interface looks broken.

## Without the React SDK

Three endpoints, authenticated exactly like the text path — the same subject token, and `Patchwork-Connection` if you use it. See the [API reference](/api-reference/voice) for the full shapes.

| Endpoint | Does |
| - | - |
| `POST /v1/voice/transcribe` | multipart `audio` → `{ text, confidence }` |
| `POST /v1/voice/speak` | `{ text, voice_id?, format?, language? }` → raw audio bytes |
| `GET /v1/voice/voices` | the voices available to you |

`speak` returns **audio, not JSON** — it is the one endpoint in the platform that does.

## How errors reach you

**The voice controls never throw.** Every failure goes to the hook's `error` and to `onError`, the same as the rest of `useAgent`:

```tsx theme={null}
const { voice, error } = useAgent({
  voice: true,
  onError: (e) => {
    if (e.code === "voice_unavailable") setMicAvailable(false);
  },
});
```

This matters for one case in particular: **`stop()` returns `""` for a silent take, a cancelled take, and a failed one.** The empty string does not tell you which. If you need to tell a silent recording apart from a broken pipeline, watch `error` — do not infer it from the transcript.

`start()` only touches the browser, so what fails there is permission or hardware. The server is not reached until `stop()` transcribes, which is where `voice_unavailable` actually appears — not at the point you asked for the microphone.

## When voice is not configured

Every voice endpoint returns **`503 voice_unavailable`** when the workspace's API has no voice provider configured. This is a normal state, not a fault: a deployment without voice keys still runs agents perfectly well.

Treat it as "hide the mic button", not "show an error". The text path is unaffected.

## Errors

| Code | Where it comes from | Means |
| - | - | - |
| `voice_disabled` | the SDK, locally | you called `start()` without `{ voice: true }` |
| `voice_unavailable` | the server, on `stop()` or `speak()` | no provider configured — hide the affordance |
| `voice_provider_error` | the server | the provider failed; retrying is reasonable |
| `voice_invalid_format` | the server | `format` was not `markdown` or `plain` |
| `voice_empty_text` | the server | nothing left to speak after rendering |

`speak()` **swallows `voice_empty_text` deliberately.** A reply that is all code or all data renders to an empty script, and a read-aloud button that asked for it should quietly do nothing rather than raise. You will not see that one on `error`.

## Metering

Voice is metered as its own product, so transcription and synthesis appear in [Activity](/guides/requests) as their own entries rather than being folded into the runs they belong to.


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.