Project Overview

An advanced voice-controlled ROS 2 autonomous navigation system that allows users to issue complex spatial commands (e.g., 'rotate left 90 degrees', 'proceed forward 2 meters') via natural language. The architecture utilizes Google's Gemini 2.5 Flash model to interpret unstructured speech transcripts and synthesize strict, structured ROS 2 Twist messages. The system features a production-grade FastAPI backend and a real-time React 18 dashboard utilizing MediaRecorder for constant audio capture and WebSocket telemetry. The robot operates on ROS 2 Humble using the Nav2 stack, Cartographer for SLAM mapping, and AMCL for precise probabilistic localization. Designed with high-availability in mind, the backend includes an automatic regex-based parser to maintain operational continuity during AI service interruptions.