Skip to main content
Beta: The SDK is in beta and actively being improved. Check npm for latest version.

​
Installation

npm install @upliftai/assistants-react @livekit/components-react livekit-client

​
Core Components

​
<UpliftAIRoom>

The main wrapper component that establishes connection to the assistant. 
import { UpliftAIRoom } from '@upliftai/assistants-react';

<UpliftAIRoom
  token={sessionToken}
  serverUrl={wsUrl}
  connect={true}
  audio={true}
  video={false}
  tools={toolsArray}
  onConnectionChange={(connected, agentIdentity) => {}}
  onToolsChange={(tools) => {}}
>
  <YourComponents />
</UpliftAIRoom>

​
Props

PropTypeRequiredDescription
tokenstringYesJWT session token from createSession
serverUrlstringYesWebSocket URL for LiveKit server
connectbooleanNoAuto-connect on mount (default: true)
audiobooleanNoEnable audio (default: true)
videobooleanNoEnable video (default: false)
toolsToolConfig[]NoArray of tool configurations
onConnectionChangefunctionNoConnection status callback
onToolsChangefunctionNoTools change callback

​
Hooks

​
useUpliftAIRoom()

Access room functionality and state. 
const {
  // Methods
  addTool,
  updateTool,
  removeTool,
  upsertTools,
  updateInstruction,
  
  // State
  isConnected,
  agentParticipant,
  room
} = useUpliftAIRoom();

​
Methods

addTool(config: ToolConfig): Promise<void>
Add a new tool to the assistant. 
await addTool({
  name: 'calculator',
  description: 'Perform calculations',
  parameters: { /* ... */ },
  handler: async (data) => { /* ... */ }
});
updateTool(config: ToolConfig): Promise<void>
Update an existing tool configuration. 
await updateTool({
  name: 'calculator',
  description: 'Updated description',
  // ... new configuration
});
removeTool(name: string): Promise<void>
Remove a tool by name. 
await removeTool('calculator');
upsertTools(configs: ToolConfig[]): Promise<void>
Replace all tools with new set. 
await upsertTools([tool1, tool2, tool3]);
updateInstruction(instruction: string): Promise<void>
Update assistant’s system instructions. 
await updateInstruction('You are now a pirate. Speak accordingly.');

​
useVoiceAssistant()

Get voice assistant state. 
const { state, participant } = useVoiceAssistant();

// state: 'listening' | 'thinking' | 'speaking'

​
useTracks()

Access audio/video tracks. 
import { useTracks } from '@upliftai/assistants-react';
import { Track } from 'livekit-client';

const tracks = useTracks([Track.Source.Microphone], {
  onlySubscribed: true,
});

const agentTrack = tracks.find(t => !t.participant.isLocal);

​
useConnectionState()

Monitor connection status. 
const { state, error } = useConnectionState();

// state: 'disconnected' | 'connecting' | 'connected' | 'reconnecting'

​
UI Components

​
Audio Components

​
<AudioTrack>

Renders audio output for a track. 
<AudioTrack trackRef={agentTrack} />

​
<BarVisualizer>

Animated audio visualization bars. 
<BarVisualizer
  state={agentState}
  trackRef={agentTrack}
  barCount={20}
  className="visualizer"
/>
Props:
  • state: Voice assistant state
  • trackRef: Audio track reference
  • barCount: Number of bars (default: 7)
  • className: CSS class name

​
Control Components

​
<TrackToggle>

Toggle microphone or camera. 
<TrackToggle source={Track.Source.Microphone}>
  Microphone
</TrackToggle>

​
<DisconnectButton>

End the session. 
<DisconnectButton>
  End Call
</DisconnectButton>

​
<ControlBar>

Full control bar with all controls. 
<ControlBar />

​
Type Definitions

​
ToolConfig

interface ToolConfig {
  name: string;
  description: string;
  parameters: {
    type: 'object';
    properties: Record<string, ParameterConfig>;
    required?: string[];
  };
  timeout: number;
  handler: (data: ToolInvocationData) => Promise<string>;
}

​
ParameterConfig

interface ParameterConfig {
  type: 'string' | 'number' | 'integer' | 'boolean' | 'array' | 'object';
  description?: string;
  enum?: any[];
  items?: ParameterConfig;
  properties?: Record<string, ParameterConfig>;
  required?: string[];
}

​
ToolInvocationData

interface ToolInvocationData {
  requestId: string;
  callerIdentity: string;
  payload: string; // JSON string to parse
  responseTimeout: number;
}

​
Complete Examples

​
Basic Voice Assistant

import React from 'react';
import {
  UpliftAIRoom,
  useUpliftAIRoom,
  useVoiceAssistant,
  BarVisualizer,
  TrackToggle,
  DisconnectButton,
  AudioTrack,
  useTracks
} from '@upliftai/assistants-react';
import { Track } from 'livekit-client';

function VoiceAssistant({ token, wsUrl }) {
  return (
    <UpliftAIRoom
      token={token}
      serverUrl={wsUrl}
      connect={true}
      audio={true}
    >
      <AssistantUI />
    </UpliftAIRoom>
  );
}

function AssistantUI() {
  const { isConnected, agentParticipant } = useUpliftAIRoom();
  const { state } = useVoiceAssistant();
  const tracks = useTracks([Track.Source.Microphone], {
    onlySubscribed: true
  });
  
  const agentTrack = tracks.find(t => !t.participant.isLocal);
  
  return (
    <div>
      <h2>Status: {isConnected ? 'Connected' : 'Disconnected'}</h2>
      
      {agentTrack && (
        <>
          <AudioTrack trackRef={agentTrack} />
          <BarVisualizer
            state={state}
            trackRef={agentTrack}
          />
        </>
      )}
      
      <p>
        {state === 'listening' && 'Listening...'}
        {state === 'thinking' && 'Thinking...'}
        {state === 'speaking' && 'Speaking...'}
      </p>
      
      <TrackToggle source={Track.Source.Microphone} />
      <DisconnectButton>End</DisconnectButton>
    </div>
  );
}

​
Assistant with Tools

function AssistantWithTools({ token, wsUrl }) {
  const tools = [
    {
      name: 'search',
      description: 'Search for information',
      parameters: {
        type: 'object',
        properties: {
          query: {
            type: 'string',
            description: 'Search query'
          }
        },
        required: ['query']
      },
      timeout: 10,
      handler: async (data) => {
        const { query } = JSON.parse(data.payload).arguments.raw_arguments;
        const results = await searchAPI(query);
        return JSON.stringify({
          result: results,
          presentationInstructions: `Found ${results.length} results`
        });
      }
    }
  ];
  
  return (
    <UpliftAIRoom
      token={token}
      serverUrl={wsUrl}
      tools={tools}
    >
      <AssistantUI />
    </UpliftAIRoom>
  );
}

​
Dynamic Tool Management

function DynamicToolManager() {
  const { addTool, removeTool, isConnected } = useUpliftAIRoom();
  const [activeTools, setActiveTools] = useState([]);
  
  const handleAddTool = async (toolConfig) => {
    try {
      await addTool(toolConfig);
      setActiveTools([...activeTools, toolConfig.name]);
    } catch (error) {
      console.error('Failed to add tool:', error);
    }
  };
  
  const handleRemoveTool = async (toolName) => {
    try {
      await removeTool(toolName);
      setActiveTools(activeTools.filter(name => name !== toolName));
    } catch (error) {
      console.error('Failed to remove tool:', error);
    }
  };
  
  return (
    <div>
      <h3>Active Tools: {activeTools.join(', ')}</h3>
      <button 
        onClick={() => handleAddTool(weatherTool)}
        disabled={!isConnected}
      >
        Add Weather Tool
      </button>
      <button 
        onClick={() => handleRemoveTool('weather')}
        disabled={!isConnected}
      >
        Remove Weather Tool
      </button>
    </div>
  );
}

​
Error Handling

function RobustAssistant({ token, wsUrl }) {
  const [error, setError] = useState(null);
  
  return (
    <UpliftAIRoom
      token={token}
      serverUrl={wsUrl}
      onConnectionChange={(connected, agentIdentity) => {
        if (!connected && agentIdentity) {
          setError('Connection lost');
        }
      }}
    >
      {error ? (
        <ErrorView error={error} onRetry={() => setError(null)} />
      ) : (
        <AssistantUI />
      )}
    </UpliftAIRoom>
  );
}

​
LiveKit Components

The SDK re-exports all LiveKit components for convenience: 
import {
  // Audio/Video
  AudioTrack,
  VideoTrack,
  
  // Controls
  ControlBar,
  TrackToggle,
  DisconnectButton,
  
  // Chat
  Chat,
  ChatEntry,
  
  // Layout
  GridLayout,
  FocusLayout,
  
  // Utilities
  RoomAudioRenderer,
  ConnectionStateToast,
  
  // ... and more
} from '@upliftai/assistants-react';

​
Performance Tips

  1. Memoize tool handlers to prevent recreating on each render
  2. Use React.memo for components that don’t need frequent updates
  3. Batch tool operations when adding/removing multiple tools
  4. Lazy load tool implementations for better initial load time
⌘I