> For the complete documentation index, see [llms.txt](https://dataqueue.gitbook.io/voicehub-docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://dataqueue.gitbook.io/voicehub-docs/api-reference/tts-v1/tts-stream-socket-v1.md).

# TTS Stream Socket V1

**WebSocket URL:** `wss://<host>/api/v1/tts-stream`

***

### Connection Flow

```
connect → init → ready → text (stream) → stop → [audio drains] → closed → disconnect
```

***

### Session Initialization

#### `init` Message (Client → Server)

**Purpose:** Authenticates the client and configures the TTS session. Must be the first message sent after connecting. If not sent within 10 seconds, the connection is closed with an error.

**Request Payload**

```json
{
  "type": "init",
  "apiKey": "dqKey_...",
  "agentId": "64f1a2b3c4d5e6f7a8b9c0d1",
  "language": "en",
  "additionalLanguages": ["ar", "fr"]
}
```

| Field                 | Type       | Required | Description                                    |
| --------------------- | ---------- | -------- | ---------------------------------------------- |
| `type`                | `"init"`   | ✅        | Message type identifier                        |
| `apiKey`              | `string`   | ✅        | Workspace API key (`dqKey_...`)                |
| `agentId`             | `string`   | ✅        | MongoDB ObjectId of the agent                  |
| `language`            | `string`   | ✅        | Primary language code (e.g. `"en"`, `"ar"`)    |
| `additionalLanguages` | `string[]` | ❌        | Fallback languages for TTS provider resolution |

***

#### `ready` Message (Server → Client)

**Purpose:** Confirms the session is initialized and the server is ready to receive text.

```json
{
  "type": "ready"
}
```

***

### Text Streaming

#### `text` Message (Client → Server)

**Purpose:** Sends a chunk of text to be synthesized into audio.

**Request Payload**

```json
{
  "type": "text",
  "text": "Hello, how can I help you today?",
  "isEnd": false
}
```

| Field   | Type      | Required | Description                                                                                       |
| ------- | --------- | -------- | ------------------------------------------------------------------------------------------------- |
| `type`  | `"text"`  | ✅        | Message type identifier                                                                           |
| `text`  | `string`  | ✅        | Text chunk to synthesize                                                                          |
| `isEnd` | `boolean` | ❌        | If `true`, signals this is the final text chunk. Equivalent to sending a separate `stop` message. |

> **Note:** Once `isEnd: true` is sent (or a `stop` message is sent), further `text` messages are ignored.

***

### Audio Events

#### `audio` Message (Server → Client)

**Purpose:** Delivers a synthesized audio chunk. Sent continuously as text is processed.

```json
{
  "type": "audio",
  "data": "<base64-encoded PCM audio>",
  "text": "Hello, how can I help you today?",
  "isEnd": false,
  "chunkId": "chunk-abc123"
}
```

| Field     | Type                    | Description                                    |
| --------- | ----------------------- | ---------------------------------------------- |
| `type`    | `"audio"`               | Message type identifier                        |
| `data`    | `string`                | Base64-encoded PCM audio chunk                 |
| `text`    | `string`                | The text segment this audio corresponds to     |
| `isEnd`   | `boolean`               | `true` on the final audio chunk                |
| `chunkId` | `string` \| `undefined` | Optional chunk identifier (provider-dependent) |

**Audio Format**

| Property    | Value                                       |
| ----------- | ------------------------------------------- |
| Encoding    | PCM Linear16 (signed 16-bit, little-endian) |
| Sample rate | 16,000 Hz                                   |
| Channels    | Mono                                        |

**Decoding Example (JavaScript)**

```js
const audioMessage = JSON.parse(event.data);
const binaryStr = atob(audioMessage.data);
const bytes = new Uint8Array(binaryStr.length);
for (let i = 0; i < binaryStr.length; i++) {
  bytes[i] = binaryStr.charCodeAt(i);
}
// bytes.buffer is now a raw PCM Int16 buffer at 16,000 Hz
const pcmSamples = new Int16Array(bytes.buffer);
```

***

### Session Termination

#### `stop` Message (Client → Server)

**Purpose:** Signals that the client has finished sending text. The server will not close immediately — it waits for the TTS provider to finish synthesizing any remaining audio and emit the final chunks.

```json
{
  "type": "stop"
}
```

**Behavior after `stop` (or `isEnd: true`):**

* Further `text` messages are ignored
* Remaining `audio` events continue to arrive until synthesis is complete
* The connection closes automatically after all audio is flushed, or after a maximum of 60 seconds, whichever comes first
* A `closed` message is sent before disconnecting

***

#### `closed` Message (Server → Client)

**Purpose:** Confirms the session has ended and the connection is about to close.

```json
{
  "type": "closed"
}
```

***

### Error Events

#### `error` Message (Server → Client)

**Purpose:** Notifies the client of an error. The connection is closed after most errors, except for text synthesis errors which are non-fatal.

```json
{
  "type": "error",
  "message": "Invalid API key"
}
```

**Common error messages**

| Condition                   | Message                                   | Fatal |
| --------------------------- | ----------------------------------------- | ----- |
| No `init` within 10 seconds | `"Init timeout"`                          | ✅     |
| Invalid JSON payload        | `"Invalid JSON"`                          | ✅     |
| Non-`init` first message    | `"First message must be an init message"` | ✅     |
| Request validation failure  | `"Validation failed: <details>"`          | ✅     |
| Invalid API key             | `"Invalid API key"`                       | ✅     |
| TTS provider unavailable    | *(provider error message)*                | ✅     |
| Text synthesis failure      | `"Error synthesizing text"`               | ❌     |

***

### Complete Example (Node.js)

```js
const WebSocket = require('ws');

const ws = new WebSocket('wss://<host>/api/v1/tts-stream');
const audioChunks = [];

ws.on('open', () => {
  ws.send(JSON.stringify({
    type: 'init',
    apiKey: 'dqKey_...',
    agentId: '<agentId>',
    language: 'en',
  }));
});

ws.on('message', (data) => {
  const msg = JSON.parse(data);

  if (msg.type === 'ready') {
    // Send text in chunks
    ws.send(JSON.stringify({ type: 'text', text: 'Hello, how can I help you today?' }));
    ws.send(JSON.stringify({ type: 'text', text: ' Is there anything I can assist you with?', isEnd: true }));
    // isEnd: true is equivalent to sending a separate stop message
  }

  if (msg.type === 'audio') {
    const binaryStr = atob(msg.data);
    const bytes = new Uint8Array(binaryStr.length);
    for (let i = 0; i < binaryStr.length; i++) {
      bytes[i] = binaryStr.charCodeAt(i);
    }
    audioChunks.push(bytes);

    if (msg.isEnd) {
      console.log('All audio received, total chunks:', audioChunks.length);
    }
  }

  if (msg.type === 'closed') {
    console.log('Session closed');
    ws.close();
  }

  if (msg.type === 'error') {
    console.error('Error:', msg.message);
  }
});

ws.on('close', () => console.log('Disconnected'));
```