Enterprise Voice Assistant & RAG Ecosystem
Voice-first enterprise assistant powered by vLLM-served LLMs, Pinecone vector search, Neo4j knowledge graph, and Whisper STT — answering HR and operations queries in real time.
- Voice p95
- 1.4s end-to-end
- HR docs indexed
- 10k+
- Answer accuracy
- +42% vs keyword
Problem & Solution
The Problem
Enterprise employees waste time hunting through HR documents, policies, and tools for answers. Text chatbots don't fit on-the-go workflows, and most enterprise search is keyword-bound.
The Solution
Built a voice-first assistant pipeline. Speech is transcribed with Whisper, queries are routed through LangChain over a hybrid Pinecone + Neo4j retrieval layer, and answers are generated by an LLM served via vLLM for low-latency, self-hosted inference.
System Architecture
End-to-end flow from intake to outcome
- 01
Voice Capture
WebRTCMicrophone audio is streamed in via WebRTC and chunked for STT.
- 02
STT
WhisperWhisper transcribes the audio stream into a text query.
- 03
Hybrid Retrieval
PineconeNeo4jLangChainQuery → vector search in Pinecone + graph traversal in Neo4j, fused via LangChain.
- 04
LLM Generation
vLLMvLLM-hosted LLM produces a grounded, spoken-friendly answer.
- 05
TTS
TTSAnswer is synthesized back to speech for the voice channel.
Engineering Challenges & Wins
Optimization results and engineering tradeoffs
Optimization Results
- Whisper STT front-end for voice-first workflows.
- Hybrid retrieval: Pinecone for vector similarity + Neo4j for relational HR context.
- vLLM-served LLM for low-latency, self-hosted inference.
Engineering Challenges
- Keeping end-to-end voice latency under 2 seconds for natural conversation.
- Fusing vector and graph retrieval without doubling the answer-generation cost.
Related Projects
Mail-server gateway (Postfix Milter) that intercepts every inbound message, runs a fine-tuned BERT core (M-BSCE) through an asynchronous inference pipeline (RabbitMQ + Celery), and adds multimodal threat detection (PaddleOCR for Quishing) with SOAP clawbacks to the upstream mail system.
- Quishing detection
- 98%+
- p95 classification
- <400ms
- Async throughput
- 5k msg/min
Autonomous multi-channel AI platform that unifies chat, voice, and email behind microservices and an API gateway, with shared context across channels.
- Channels
- 3 unified
- Tenant onboard
- <1 day
- Cross-channel memory
- 100%