EngineeringFebruary 15, 202632 min read

Sentence-Level Streaming VUI Architecture: From Cognitive Theory to Production Implementation in MARIA OS

How sentence-boundary detection, sequential TTS chaining, and rolling conversation summaries create a natural-feeling voice interface with long-session stability

Voice user interfaces face a core tradeoff: stream tokens immediately for low latency, or wait for larger semantic units to improve naturalness. MARIA OS resolves this with sentence-level streaming: detect sentence boundaries from Gemini token streams in real time, queue each sentence for sequential ElevenLabs TTS playback, and coordinate full-duplex interaction through barge-in control, speech debouncing, and heartbeat-based recovery. This paper presents the cognitive basis for sentence-level granularity, the production `useGeminiLive` architecture, a 29-tool action router across 4 teams with confidence-weighted team inference, and the rolling-summary mechanism for long voice sessions. In 2,400+ production sessions, the system achieved sub-800ms first-sentence latency with zero sentence-ordering violations, including compatibility handling for 9 in-app browser environments.

voice-uistreamingTTSspeech-recognitionreal-timeGeminiElevenLabsaction-routerMARIA-OScognitive-science
IntelligenceFebruary 15, 202635 min read

Cognitive Science Foundations of Voice User Interface Design: An Attention Resource Allocation Model for Multimodal Dialogue

Integrating Wickens' multiple resource theory, Baddeley's working memory model, and information theory to formalize VUI design principles and validate them in the MARIA VOICE implementation

Voice user interface (VUI) design tends to rely on heuristics that do not adequately address the characteristics of auditory cognitive processing. This paper integrates Wickens' multiple resource theory, Baddeley's working memory model, and Shannon information theory to present a mathematical model of attention resource allocation in multimodal dialogue. We demonstrate the cognitive optimality of sentence-level streaming TTS, the theoretical basis for the 1.2-second debounce threshold, and the conditions under which barge-in suppression avoids resource conflict, providing a theoretical account of MARIA VOICE's design decisions.

voice-uicognitive-scienceinformation-theoryworking-memoryattention-resourcesmultimodal-interactionspeech-processingmaria-voiceformal-methodshuman-computer-interaction