Toward Ergonomic and Productive Human-Machine Interaction: A Conceptual Study of Hybrid Gesture and Voice Interfaces.

Abstract

Industrial automation increasingly relies on Human Machine Interfaces (HMIs), yet conventional unimodal controls such as buttons or gestures alone often lead to operator fatigue and reduced productivity. This thesis addresses this challenge by evaluating a Hybrid HMI that integrates gesture and voice control to enhance ergonomics, efficiency, and operator performance. Using a conceptual evaluation approach, mockups and workflow diagrams were employed to simulate hybrid control scenarios. Twenty-five participants assessed the system through two standardized methods: the System Usability Scale (SUS) for usability and NASA TLX for perceived workload. Results revealed a mean SUS score of 81, exceeding the benchmark of 78, and demonstrated excellent usability. The workload analysis indicated low frustration, moderate effort, and high satisfaction with performance, highlighting that hybrid interfaces alleviate strain while improving operator responsiveness. These findings underscore the potential of integrating gesture and voice control to outperform unimodal gesture-only or button-only systems. The study concludes that hybrid multimodal HMIs hold significant promise for advancing Industry 4.0 goals by fostering more ergonomic, resilient, and productive work environments. While conceptual, this research emphasizes the value of early-stage usability testing through mockups to inform design decisions before costly prototyping. Future work will focus on developing a physical prototype, testing under real industrial conditions.