Omkar Satpute
65 posts

Omkar Satpute
@BuildwithOmkarr
Building TipTour - Future of AI Interface | IITMadras 22

I’m reverse-engineering the Google AI mouse pointer and making it Open Source! It sees your screen. You can freeform paint anything and ask AI to modify it. It understands the app or window you mean. It can click, type, edit, navigate, and even write + execute code. A tiny cursor becoming a real computer-use agent. Made with the help of @trycua which is an AMAZING reference for computer use Sorry @GoogleDeepMind but I had to do this @heyclicky Started a revolution and we will be the ones to win Also credits to @BuildwithOmkarr for the help Feel free to modify it , update it and make it your own, we gotta beat them at this. Link to the repo in the comments:


We’re reimagining a 50-year-old interface - the mouse pointer - with AI. 🖱️ These experimental demos show how people can intuitively direct Gemini on their screens using motion, speech, and natural shorthand to get things done 🧵







Okay hear me out @bchesky for an alternative interface A cursor that you can talk to and points you to your stay


@benhylak The ChatGPT interface doesn’t work for this. We’ve already tried it.







