> ## Documentation Index
> Fetch the complete documentation index at: https://docs.upliftai.org/llms.txt
> Use this file to discover all available pages before exploring further.

# WebSocket TTS API

> Real-time text-to-speech streaming for conversational AI

Stream text-to-speech audio in real-time using WebSocket connections. Perfect for conversational AI applications that need low-latency audio synthesis.

## When to Use WebSocket TTS

<Info>
  **Best for**: Real-time conversational AI, voice agents, and applications needing continuous TTS streaming with multiple concurrent requests. Visit this [tutorial](/tutorials/websocket-conversational-ai) for an implementation.
</Info>

### Key Benefits

* **Low latency**: \~300ms to first audio chunk
* **Multiple requests**: Handle multiple synthesis requests on single connection
* **Real-time streaming**: Audio chunks stream as they're generated
* **Persistent connection**: Reuse connection for entire conversation

## Connection

### Endpoint

```
wss://api.upliftai.org/text-to-speech/multi-stream
```

### Authentication

Connect using your API key:

```javascript theme={null}
const socket = io('wss://api.upliftai.org/text-to-speech/multi-stream', {
  auth: {
    token: 'sk_api_your_key_here'
  },
  transports: ['websocket']
});
```

## Message Protocol

All messages use a unified format with a `type` field.

### Client → Server Messages

#### Synthesize Text

```json theme={null}
{
  "type": "synthesize",
  "requestId": "unique_request_id",
  "text": "سلام، آپ کیسے ہیں؟",
  "voiceId": "v_meklc281",
  "outputFormat": "MP3_22050_32"
}
```

**Parameters:**

* `requestId`: Unique ID for tracking this request
* `text`: Text to synthesize (max 10,000 characters)
* `voiceId`: Voice to use (e.g., "v\_meklc281" for Urdu female)
* `outputFormat`: Audio format (optional, defaults to PCM\_22050\_16)

#### Cancel Request

```json theme={null}
{
  "type": "cancel",
  "requestId": "unique_request_id"
}
```

### Server → Client Messages

All server messages come through the `message` event:

#### Connection Ready

```json theme={null}
{
  "type": "ready",
  "sessionId": "session_abc123"
}
```

#### Audio Start

```json theme={null}
{
  "type": "audio_start",
  "requestId": "unique_request_id",
  "timestamp": 1234567890
}
```

#### Audio Chunk

```json theme={null}
{
  "type": "audio",
  "requestId": "unique_request_id",
  "audio": "base64_encoded_audio_data",
  "sequence": 0
}
```

#### Audio End

```json theme={null}
{
  "type": "audio_end",
  "requestId": "unique_request_id",
  "timestamp": 1234567890
}
```

#### Error

```json theme={null}
{
  "type": "error",
  "requestId": "unique_request_id",
  "code": "synthesis_failed",
  "message": "Voice not found"
}
```

## Simple Example

```javascript theme={null}
import { io } from 'socket.io-client';

// Connect to WebSocket
const socket = io('wss://api.upliftai.org/text-to-speech/multi-stream', {
  auth: { token: 'sk_api_your_key' },
  transports: ['websocket']
});

// Handle messages
socket.on('message', (data) => {
  switch(data.type) {
    case 'ready':
      console.log('Connected!');
      // Start synthesis
      socket.emit('synthesize', {
        type: 'synthesize',
        requestId: 'req_001',
        text: 'سلام، یہ ایک ٹیسٹ ہے۔',
        voiceId: 'v_meklc281',
        outputFormat: 'MP3_22050_32'
      });
      break;
      
    case 'audio':
      // Decode and play audio chunk
      const audioData = Buffer.from(data.audio, 'base64');
      // Play audioData...
      break;
      
    case 'audio_end':
      console.log('Audio complete!');
      break;
      
    case 'error':
      console.error('Error:', data.message);
      break;
  }
});
```

## Output Formats

| Format          | Description               | Use Case                |
| --------------- | ------------------------- | ----------------------- |
| `PCM_22050_16`  | Raw PCM, 22.05kHz, 16-bit | Direct audio processing |
| `MP3_22050_32`  | MP3, 22.05kHz, 32kbps     | Small file size, web    |
| `MP3_22050_128` | MP3, 22.05kHz, 128kbps    | High quality streaming  |
| `WAV_22050_32`  | WAV, 22.05kHz, 32-bit     | Lossless audio          |
| `ULAW_8000_8`   | μ-law, 8kHz, 8-bit        | Telephony systems       |

## Available Voices

Use the same voice IDs as REST API:

* `v_meklc281` - Urdu female
* `v_8eelc901` - Info/Education
* `v_30s70t3a` - Nostalgic News
* `v_yypgzenx` - Dada Jee (storytelling)

## Error Codes

| Code                  | Description             | Action             |
| --------------------- | ----------------------- | ------------------ |
| `auth_failed`         | Invalid API key         | Check your API key |
| `synthesis_failed`    | TTS service error       | Retry with backoff |
| `duplicate_request`   | Request ID already used | Use unique IDs     |
| `rate_limit_exceeded` | Too many requests       | Slow down requests |
| `text_too_long`       | Text > 10,000 chars     | Split into chunks  |

## Rate Limits

* **Synthesis requests**: 60 per minute per connection
* **Cancel requests**: 100 per minute per connection
* **Max text length**: 10,000 characters per request
* **Daily limit**: Based on your plan

## Best Practices

<AccordionGroup>
  <Accordion title="Use Unique Request IDs">
    Generate unique IDs (like UUIDs) for each synthesis request to track audio chunks properly.
  </Accordion>

  <Accordion title="Maintain Single Connection">
    Keep one WebSocket connection open and reuse it for multiple synthesis requests.
  </Accordion>

  <Accordion title="Buffer Audio Chunks">
    Collect audio chunks before playback for smooth streaming experience.
  </Accordion>

  <Accordion title="Handle Reconnection">
    Implement exponential backoff for reconnection attempts on connection loss.
  </Accordion>
</AccordionGroup>

## Testing with wscat

Quick test using command line:

```bash theme={null}
# Install wscat
npm install -g wscat

# Connect
wscat -c wss://api.upliftai.org/text-to-speech/multi-stream \
  -H "Authorization: Bearer sk_api_your_key"

# Send synthesize message
{"type":"synthesize","requestId":"test-1","text":"Hello world","voiceId":"v_meklc281"}
```

## Next Steps

<CardGroup cols={2}>
  <Card title="WebSocket Tutorial" href="/tutorials/websocket-conversational-ai">
    Build a conversational AI app
  </Card>

  <Card title="Voice Samples" href="/orator_voices">
    Listen to available voices
  </Card>
</CardGroup>