Imagine tapping your thumb and index finger together twice to skip to the next song or clicking around your laptop or desktop computer without a mouse, using discreet finger motions. New first-of-its-kind wearable technology from researchers at Cornell and KAIST, in South Korea, brings that vision closer to reality. The system, called WatchHand, equips off-the-shelf smartwatches with AI-powered micro sonar capable of tracking hand movements. The technology has the potential to revolutionize how we interact with our devices by continuously tracking hand poses in real time using smartwatches and their built-in speaker and microphone, the researchers said. It is the first time AI-powered acoustic sensing for hand-pose tracking has been implemented on off-the-shelf smartwatches without the need for additional hardware. “In the future, with this kind of hand-tracking technology, we might be able to track our typing with just our smartwatch,” said Chi-Jung Lee, a doctoral student in the field of information science in @Cornell_Bowers, and the co-lead author of “WatchHand: Enabling Continuous Hand Pose Tracking On Off-the-Shelf Smartwatches,” which will be presented at the Association for Computing Machinery (ACM) CHI conference on Human Factors in Computing Systems beginning April 13 in Barcelona. “Our hands can act as an input device with computers,” she said. Read more: news.cornell.edu/stories/2026/0….
