> For the complete documentation index, see [llms.txt](https://dataqueue.gitbook.io/voicehub-docs/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://dataqueue.gitbook.io/voicehub-docs/api-reference/stt-v1/stt-stream-socket-v1.md).

# STT Stream Socket V1

The STT Stream WebSocket provides real-time speech-to-text transcription by streaming PCM audio to the server and receiving transcript events as they are recognized.

**WebSocket URL:** `wss://<host>/api/v1/stt-stream`

***

### Connection Flow

```
connect → init → ready → audio (stream) → stop → [transcripts drain] → closed → disconnect
```

***

### Session Initialization

#### `init` Message (Client → Server)

**Purpose:** Authenticates the client and configures the transcription session. Must be the **first message** sent after connecting. If not sent within **10 seconds**, the connection is closed with an error.

**Request Payload**

```json
{
  "type": "init",
  "apiKey": "dqKey_...",
  "agentId": "64f1a2b3c4d5e6f7a8b9c0d1",
  "language": "en",
  "additionalLanguages": ["ar", "fr"],
  "diarizationEnabled": false
}
```

| Field                 | Type               | Required | Description                                          |
| --------------------- | ------------------ | -------- | ---------------------------------------------------- |
| `type`                | `"init"`           | ✅        | Message type identifier                              |
| `apiKey`              | `string`           | ✅        | Workspace API key                                    |
| `agentId`             | `string` (MongoId) | ✅        | The workspace (agent) ID the API key belongs to      |
| `language`            | `string`           | ✅        | Primary transcription language (e.g. `"en"`, `"ar"`) |
| `additionalLanguages` | `string[]`         | ❌        | Additional languages for multi-language detection    |
| `diarizationEnabled`  | `boolean`          | ❌        | Enable speaker diarization. Defaults to `false`      |

***

#### `ready` Message (Server → Client)

**Purpose:** Confirms the session is initialized and the server is ready to receive audio.

```json
{
  "type": "ready"
}
```

***

### Audio Streaming

#### `audio` Message (Client → Server)

**Purpose:** Sends a chunk of PCM audio to be transcribed.

**Audio Format Requirements:**

* Encoding: **PCM Linear16** (signed 16-bit, little-endian)
* Sample rate: **16,000 Hz**
* Channels: **Mono**

**Request Payload**

```json
{
  "type": "audio",
  "data": "<base64-encoded PCM chunk>"
}
```

| Field  | Type              | Required | Description                        |
| ------ | ----------------- | -------- | ---------------------------------- |
| `type` | `"audio"`         | ✅        | Message type identifier            |
| `data` | `string` (base64) | ✅        | Base64-encoded raw PCM audio chunk |

**Example (JavaScript)**

```js
const audioData = new Int16Array(rawPcmBuffer);
const base64Audio = btoa(String.fromCharCode(...new Uint8Array(audioData.buffer)));

socket.send(JSON.stringify({
  type: 'audio',
  data: base64Audio,
}));
```

***

### Transcript Events

#### `transcript` Message (Server → Client)

**Purpose:** Delivers a recognized speech segment. Sent continuously as audio is processed.

```json
{
  "type": "transcript",
  "transcript": {
    "type": "transcript",
    "isFinal": false,
    "results": [
      {
        "text": "hello how can I",
        "confidence": 0,
        "stability": 0.9,
        "speaker": "1"
      }
    ]
  }
}
```

| Field                  | Type      | Description                                                                        |
| ---------------------- | --------- | ---------------------------------------------------------------------------------- |
| `transcript.isFinal`   | `boolean` | `true` when the provider has finalized this utterance. `false` for interim results |
| `results[].text`       | `string`  | Transcribed text segment                                                           |
| `results[].confidence` | `number`  | Confidence score (0–1). Only meaningful when `isFinal` is `true`                   |
| `results[].stability`  | `number`  | Stability of the interim result (0–1). Only present when `isFinal` is `false`      |
| `results[].speaker`    | `string`  | Speaker label. Only present when `diarizationEnabled` is `true`                    |

> **Note:** Interim results (`isFinal: false`) are incremental and may be revised. Only `isFinal: true` results are confirmed and finalized by the provider.

***

### Session Termination

#### `stop` Message (Client → Server)

**Purpose:** Signals that the client has finished sending audio. The server will **not** close immediately — it waits for the STT provider to finish processing any buffered audio and emit remaining transcripts.

```json
{
  "type": "stop"
}
```

**Behavior after `stop`:**

* Further `audio` messages are ignored
* Transcript events continue to arrive until the provider goes silent
* The connection closes automatically after **8 seconds of no new transcripts**, or after a maximum of **60 seconds**, whichever comes first
* A `closed` message is sent before disconnecting

***

#### `closed` Message (Server → Client)

**Purpose:** Confirms the session has ended and the connection is about to close.

```json
{
  "type": "closed"
}
```

***

### Error Events

#### `error` Message (Server → Client)

**Purpose:** Notifies the client of an error. The connection may be closed after an error depending on the context.

```json
{
  "type": "error",
  "message": "Invalid API key"
}
```

**Common error messages:**

| Message                                   | Cause                                          |
| ----------------------------------------- | ---------------------------------------------- |
| `"Init timeout"`                          | No `init` message received within 10 seconds   |
| `"First message must be an init message"` | Non-`init` message sent before initialization  |
| `"Invalid API key"`                       | API key does not match the provided `agentId`  |
| `"Validation failed: ..."`                | Invalid fields in the `init` message           |
| `"Invalid audio message"`                 | `data` field is missing or not valid base64    |
| `"Error processing audio"`                | Internal error while processing an audio chunk |

***

### Complete Example (Node.js)

```js
const WebSocket = require('ws');
const fs = require('fs');

const ws = new WebSocket('wss://<host>/api/v1/stt-stream');

ws.on('open', () => {
  ws.send(JSON.stringify({
    type: 'init',
    apiKey: 'dqKey_...',
    agentId: '<agentId>',
    language: 'en',
  }));
});

ws.on('message', (data) => {
  const msg = JSON.parse(data);

  if (msg.type === 'ready') {
    const pcm = fs.readFileSync('/path/to/audio.pcm');
    for (let i = 0; i < pcm.length; i += 3200) {
      ws.send(JSON.stringify({
        type: 'audio',
        data: pcm.subarray(i, i + 3200).toString('base64'),
      }));
    }
    setTimeout(() => {
      ws.send(JSON.stringify({ type: 'stop' }));
    }, 2000);
  }

  if (msg.type === 'transcript') {
    console.log(msg.transcript.isFinal ? '[FINAL]' : '[interim]', msg.transcript.results[0]?.text);
  }

  if (msg.type === 'closed') {
    console.log('Session closed');
  }

  if (msg.type === 'error') {
    console.error('Error:', msg.message);
  }
});

ws.on('close', () => console.log('Disconnected'));
```