Chen Liang

🎓 Congrats to @UMichCSE’s 2025 Graduate Honors Competition finalists! 🥇@AnnaAblove 🥈 @madelyn_gatchel 👏@ChenLiangCS, @_tarunesh_Verma, Benyu Wang ➕Congrats to Dr. @yashairavantchi, 2025 Kuck Dissertation Prize recipient! 📷 More: #UMCSE" target="_blank" rel="nofollow noopener">myumi.ch/kP8w9#UMCSE #PhDResearch

This work wouldn’t be possible without the amazing collaborators and advisors on this project - @yuxuanliu21 @Martez_Mott @AnhongGuo, thank you so much! (8/n)

Introducing HandProxy, a speech-controlled virtual proxy hand that interprets the user’s natural speech input and performs hand interactions in XR on the user's behalf. Accepted in #IMWUT 2025 (1/n) Paper: bit.ly/4fjdEJN Demo Video: bit.ly/40KXtPy

We envision HandProxy will expand the design of existing speech interfaces in XR, and hope this can be implemented on the system level to provide a uniform, cross-application input solution for accessible control and input automation of hand interactions. (7/n)

A user study on various hand interaction tasks showed a 91.8% command execution accuracy, with an average of 1.09 attempts per speech command. HandProxy was able to interpret diverse commands users issued during tasks, and users found it flexible, effective, and easy to use (6/n)

To do so, HandProxy continuously captures speech input and uses LLM to decompose it into a list of hand control primitives. They are used to calculate the 3D hand joints for the hand movement, and joint data is streamed to the virtual environment for hand control (5/n)

Users can issue various commands, such as gesture (pinch, grab the cube), timing (stop, undo), movement (move up, twist left), or high-level interaction goals (reduce the volume, increase brightness) where HandProxy interprets and decomposes it into detailed hand controls (4/n)

HandProxy introduces a virtual hand as an interaction proxy. It expands the affordance of speech interfaces to support expressive hand interactions. Users can initiate hand interactions through natural speech, and the system interprets and decomposes it to hand movements (3/n)

Hand interactions are widely used in XR but are not always feasible, e.g., due to (situational) impairments, environmental constraints. Speech can be used as an alternative, but is often limited to initiating basic gestures and system controls (2/n)

@jayhersk @TuftsUniversity @TuftsCS Let’s gooooo! Congrats Jaylin!!

I’m so happy to share that I’ll be joining @TuftsUniversity @TuftsCS as an Assistant Professor in January 2026! 🥳 I’ll be recruiting PhD students this upcoming cycle, so keep an eye out for more info if you’re interested in accessibility, AI, and HCI! (1/2)

Serving as the Student Volunteer Co-Chair at @ACM_CHI has been an incredible experience! A huge thank you to the 200 student volunteers who worked tirelessly behind the scenes to help make the conference a success. Looking forward to seeing you all next year in Barcelona!

#CHI2025 We had a great #AR x #AI workshop on Everyday AR through AI-in-the-Loop today. Thank you for joining the workshop. xr-and-ai.github.io

NEW: “New phone case provides workaround for inaccessible touchscreens” Touchscreens are everywhere but not built for everyone. A new device could help bridge that gap, helping users access ticket kiosks, restaurant menus and more. Click image for story bit.ly/45PC5bk

Thanks to our amazing co-authors @yashairavantchi , @tkroliko, @ruijie_geng , @AlansonSample, and my advisor @AnhongGuo for the work and guidance, our participants for their time and valuable feedback, and help from my friends! It won’t be possible without your help! 6/6

With more related research (TouchA11y, TakeMyHand) contributing to the growing space of touchscreen accessibility, we hope BrushLens could provide an additional perspective to bring independence and agency for users to interact with diverse inaccessible touchscreens. 5/6

Excited to share our #UIST2023 paper BrushLens: Hardware Interaction Proxies for Accessible Touchscreen Interface Actuation - a phone case with actuators to locate and tap buttons for users. 30s: bit.ly/3sYcTSf Vid: bit.ly/3Zh88zh Paper: bit.ly/3Pii9aU

I'm honored to receive recognition for my dedication and passion in teaching. Thank you CSE and my advisor @AnhongGuo for the support and guidance!