NavSense

NavSense turns navigation into touch. A pair of haptic smart gloves delivers turn-by-turn directions through vibration, freeing eyes and ears for people who are blind, low-vision, Deaf, or hard-of-hearing, and for anyone who wants to move through the world without staring at a screen.

problem

Navigation today asks you to give up a sense. Audio directions occupy your hearing in the middle of a busy street, and screens pull your eyes off the path ahead. For people who depend on those senses to stay safe outdoors, that tradeoff isn't minor, it's a real risk. There was an opening for a different channel: one that's free and available even when sound is loud and vision is occupied.

solution

NavSense moves guidance into the hands. Two smart gloves pair to an iOS app and translate each maneuver into a distinct vibration pattern, a left-hand buzz for a left turn, separate cues for right, forward, pause, and recalculation. Every cue is short and unambiguous, so directions land instantly without overwhelming the wearer. The app pulls a route from the Google Maps Directions API, parses it into steps, and maps each one to a haptic pattern with consistent timing for advance notice and execution. A built-in voice agent, powered by ElevenLabs and Gemini, handles start points, destinations, travel modes, rerouting, and quick questions through natural conversation. On-device machine learning listens for ambient sirens and alerts the wearer to pause when danger is near. LEDs mirror each haptic state for cross-sensory clarity, and the gloves recharge over USB-C. Rather than replacing canes, guide dogs, or existing safety apps, NavSense layers on top of them, adding a quiet, hands-free stream of spatial information through touch.

NavSense was built at Hack the Valley X, where it placed 3rd out of 400+ participants.

I focused on the intelligence and safety layers of the system. I integrated a conversational voice agent into the app, combining ElevenLabs for natural speech with a Gemini model for intent and dialogue, prompt-tuning it and wiring it to the backend over webhooks so users could plan and adjust routes entirely by voice. I also built a custom TensorFlow audio-classification model to detect surrounding sirens in real time, the feature that lets the gloves warn a wearer of approaching danger, and put together the final presentation.

Beyond my own work, the project meant taking an idea from scratch to a working prototype in a single weekend: designing wearable hardware, writing embedded firmware, integrating real-time routing, and stitching hardware and software into one reliable system. We hit the usual roadblocks, two Macs between the whole team, a bootloader failure on an ESP32 mid-assembly, and midterms competing for everyone's time, but dividing responsibilities and adapting fast made crossing the finish line that much more rewarding. The result is a tool built around inclusivity and accessibility, and proof of how much a small team can ship under pressure.

Check it out: https://devpost.com/software/navsense-jp2e7z

year

2025

timeframe

16 days

tools

React Native · TypeScript · ESP32 · C/C++ · BLE · Core ML · TensorFlow Lite · Google Maps API · Gemini API · ElevenLabs · Fusion 360 / 3D Printing