Nazura Wirayuda Tama, Novanto Yudistira, Daniel Geoffrey Manurung, Sarina Sulaiman, Pang Yee Yong, Farkhana Muchtar, Nur Zuraifah Syazrah Othman, Nor Azman Ismail
Modern navigation systems often stop at visual perception, providing raw detections without translating them into actionable guidance for drivers. We present a practical, deployable navigation assistance framework that tightly couples real-time object detection with Large Language Models (LLMs) to produce fluent, context-aware driving instructions. Unlike conventional Advanced Driver Assistance Systems (ADAS), our approach introduces a structured intermediate representation that encodes detector outputs, such as traffic sign identities and locations, into a compact, machine-readable format before prompting the LLM. This design improves controllability, reduces irrelevant generation, and enables adaptation to dynamic roadway and traffic contexts. Evaluated on the 21-class Traffic Sign in Indonesia Dataset, our system achieves state-of-the-art detection performance and introduces the Feasibility Score, a multi-criteria human evaluation metric that captures relevance, coherence, completeness, fluency, and specificity of generated instructions. Experiments across multiple LLM configurations demonstrate that coupling structured perception with LLM-based reasoning produces guidance rated as clearer, more specific, and more context-relevant than perception-only or naive LLM baselines. These results position our framework as a concrete step toward next-generation, human-centered navigation assistance that bridges the gap between visual recognition and actionable driver communication. © 2025 Copyright held by the owner/author(s).
Faculty of Computer Science, Universitas Brawijaya, East Java, Malang, Indonesia; Faculty of Computing, Universiti Teknologi Malaysia, Johor, Johor Bahru, Malaysia
