LLMJan 2025 → Present

AI Avatar Assistant (Real-Time)

Real-time avatar-based assistant combining LiveKit, WebRTC, Groq inference, and TTS/STT, with a Ready Player Me avatar for natural face-to-face interactions.

PythonFastAPILiveKitWebRTCGroqTTSSTTReady Player Me

Streaming latency: <300ms
Concurrent sessions: 200+
Avatars supported: RPM SDK

Problem & Solution

The Problem

Text and even voice chatbots feel impersonal. Customer-facing experiences benefit from a face-to-face avatar that can talk, listen, and respond in real time.

The Solution

Built a real-time avatar assistant on top of LiveKit for media transport and WebRTC for low-latency streaming. The avatar (Ready Player Me) is driven by a Groq-served LLM with a TTS/STT pipeline for natural conversation.

System Architecture

End-to-end flow from intake to outcome

01
Avatar Renderer
Ready Player Me
Ready Player Me avatar driven by client-side media for natural face-to-face UX.
02
Media Transport
LiveKitWebRTC
LiveKit + WebRTC stream audio and video in real time.
03
STT
STT
Speech-to-text transcription streamed to the LLM.
04
LLM
Groq
Groq-served LLM generates the response in under a second.
05
TTS
TTS
Response is synthesized and streamed back to the avatar in real time.

Engineering Challenges & Wins

Optimization results and engineering tradeoffs

Optimization Results

Real-time media transport via LiveKit + WebRTC.
Groq-served LLM for sub-second response generation.
TTS / STT pipeline with a Ready Player Me avatar for face-to-face UX.

Engineering Challenges

Synchronizing avatar lip movement with TTS output under tight latency.
Scaling concurrent real-time sessions without exploding media-server cost.

More case studies

Explore other production systems I've engineered end-to-end.

All Projects

Related Projects

LLM

NL2SQL & Data Intelligence Chatbots

Chatbots that turn natural language into SQL, Pandas, and Neo4j Cypher queries against structured and relational data, with an Excel-AI-Chatbot on top of DuckDB.

Sources: DuckDB · Neo4j · Excel
Query accuracy: 92%
Avg response: 1.8s

PythonDuckDBPandasNeo4jFastAPI+2

View Case Study

Autonomous AI AgentsFeatured

OmniShield ICES: Asynchronous AI Email Security Middleware

Mail-server gateway (Postfix Milter) that intercepts every inbound message, runs a fine-tuned BERT core (M-BSCE) through an asynchronous inference pipeline (RabbitMQ + Celery), and adds multimodal threat detection (PaddleOCR for Quishing) with SOAP clawbacks to the upstream mail system.

Quishing detection: 98%+
p95 classification: <400ms
Async throughput: 5k msg/min

PythonPostfix MilterRabbitMQCeleryBERT+6