Horizon UI

Horizon UI is a multimodal desktop control system that allows users to control computer actions using hand gestures and voice commands. It uses real-time hand tracking and speech recognition to create a touchless interaction layer for desktop environments.

Desktop interaction is still heavily dependent on physical input devices, which limits hands-free workflows and accessibility-focused interaction patterns.

Built the interaction pipeline, gesture mapping, voice command layer, desktop automation flow, and demo narrative for the final year project.

• Camera feed is processed through OpenCV and MediaPipe for hand landmarks.
• Gesture logic maps tracked landmarks into desktop commands.
• Speech recognition layer listens for voice commands and triggers mapped actions.
• PyAutoGUI executes desktop automation actions from validated inputs.

• Maintaining responsive control while processing real-time video input.
• Reducing false positives in gesture recognition during live demos.
• Creating a command model that is flexible without being unpredictable.

• 3rd Place at Industrial Open House 2026
• Multimodal gesture and voice control prototype

• Human-computer interaction projects need as much UX thinking as algorithmic accuracy.
• Live demos improve when technical triggers are explainable and recoverable.