Stijn Spanhove

🚫🕶️ I've been building an XR app for a real-world ad blocker using Snap @Spectacles. It uses Gemini to detect and block ads in the environment. It’s still early and experimental, but it’s exciting to imagine a future where you control the physical content you see.

8th Wall + Gaussian Splats 🤩

A colleague with augmented reality expertise photoscanned Phoenix, built a 3D model, and turned it into a WebXR experience. You can literally place her in your surroundings and walk around her. The SpaceX Starlink rocket in the back is a nice touch too 🚀 Full credits to @stspanho 🙌 Thanks man!

@TheOriginaliTE Clever workaround!

Passthrough access isn’t available to visionOS apps. So I built it. 😉 Introducing PeekABoo — the first consumer-facing passthrough API for visionOS.👀 It unlocks a whole new category of spatial apps. Here’s how I pulled it off 👇🏻

@augmentedcamel Nice 🤩

Building a openclaw for industrial machines and this is what happend at last week demos learning how to make shorts so im gonna post more of these

@augmentedcamel Visualizing data 🤤

I get a dopamine kick every time I visualize the documentation of a new machine. HybridRAG ftw

@OscarFalmer High Resolution Video Streaming Support 😍

📣 Meta Wearables Device Access Toolkit v0.5 out! 🎉 • AI Coding Agents Config Files Added: Agents.md, Claude Skills, Cursor Rules, Copilot Instructions • Meta Oakley Vanguard Support • High Resolution Video Streaming Support (720x1280) • Improvements on iOS + Android SDKs

@_xulipa @Spectacles @pt_pavlo Love the voice interactions!

Made a @Spectacles game where you have to use your voice cast spells. Thanks for playing @pt_pavlo

@pt_pavlo @augmentedcamel @AndyFidel_ @supabase Yes it is 🤩🚀

@augmentedcamel @AndyFidel_ @stspanho It’s definitely possible with @supabase backend

Future of immersive games is bright. I played the tower defense but it’s an apple game, you can control the drone by moving the Banana 🍌 Great work @pt_pavlo @stspanho

@JohannesTscharn @sparkjs @SensAIHackademy Wow! Can I rewatch the workshop? Subscribed but won't be able to attend due to an overlapping meeting.

@pt_pavlo @augmentedcamel @JohannesTscharn Ah yes! I mean a real apple 🤣. But AVP would be super cool

@augmentedcamel @JohannesTscharn @stspanho @stspanho time for AVP ahaha ?

@JohannesTscharn @augmentedcamel @pt_pavlo Thank you! There is also a single player mode, if you have an apple, please try it out 😇 The game is in the Lens Explorer and it is called Fruit Defense

@augmentedcamel @stspanho @pt_pavlo 🍌 Cool idea to use real world objects as game objects!

@augmentedcamel @pt_pavlo Thanks for trying it out 🤩

@JohannesTscharn @huggingface @Spectacles So cool! What's your end goal? Or are you just doing some experiments?

I’m giving an agent control over Reachy Mini from @huggingface and letting it understand and share spatial data via @Spectacles AR is the human interface for robotics and physical AI imo. It feels like absolute magic to interact with this, both in voice/agent and “puppeteering” mode. I’ll probably work on AR for either an arm (manipulation tasks) or some sort of drone (locomotion in 3D space) next… Project is fully open source btw: github.com/V4C38/spectacl… Thank you @SensAIHackademy for sending me the robot!

@OscarFalmer External screen + Meta glasses 🤝. I like that idea!

👋 Built a Leap Motion-style hand tracker using my Meta Ray-Ban glasses Today I built an app that lets me manipulate 3D models on my laptop using hand gestures captured from the glasses' POV. The glasses stream video to my iPhone via an app using the Wearables Developer Access Toolkit, which then sends frames over WebSocket to a webpage on my laptop. On the webpage MediaPipe detects my hands, and I'm rotating 3D models with pinch gestures. The webpage I built also allows to use the webcam but this is an experimentation to see a similar workflow with the user 's point of view, which can work for example with my monitor in the video that does not have any webcam here. This is all done in a couple hours starting from the official iOS sample app: github.com/facebook/meta-…

@KovidBhaduri @itpocket That's cool 👏

@stspanho @itpocket Very Interesting! I have been collecting people's wisdom and mapping them in a 3d space, clustering with similarity. mindspace.kovidbhaduri.com

Imagine stepping into your embeddings that power RAG. 📚 In this proof of concept, you’re surrounded by courses. Semantically similar ones float right next to each other. I've built an XR prototype with Rémi at @itpocket where you can stand inside your knowledge base on Apple Vision Pro. Ask a question and the AI lights up the relevant documents in space, so you literally see where your answer comes from. When you ask something specific like "I want to be an astronaut, what can I learn", only a few tight clusters glow. But ask "I'm bored, what could be something fun to study" and clusters across the space light up. #applevisionpro #artificialintelligence #xr #rag

@ematol @itpocket Yes definitely possible, we wrote an API to power the application. Heavy processing lives in a backend

@stspanho @itpocket This is amazing!! Do you think it's possble with regular AR? 🤔 need to up my game, only have the meta quest 3

@zhngjny52976818 @itpocket No not at the moment, but planning to write something down

@stspanho @itpocket so cool it is open sources ？

@GabRoXR Wow, so impressed.

I scanned a 13th-century fort with the #PortalCam Here is what worked and what didn't Let me start with some context on the scan. The overall process took 32 minutes. During the scan I tried to move closer to specific objects and features trying to capture more details and ended up walking 3 times on the battlements (not sure that was a good idea). Initially, the RTX 3060 failed the reconstruction but after the upgrade to the #DellProMax T2 Tower equipped with an #NVIDIA RTX 6000 i got no issues (#DellTech for the win!). 👍 The final reconstruction ended up at 39,5M splats and the final .ply size was 2.6 Gb. 👍 Some of the features, like the gates, the well and benches, came up extremely well. 👍 The mesh generated by LCC studio was quite solid and allowed me to easily walk up the stairs and even jump on various areas of the fort. ❌ The scan was taken in October and the foliage made it very complicated to capture such a big area. ❌ Unfortunately, the HD features in LCC studio were not available yet so I couldn't test if the extra picture I took using a DSLR could complement the capture of the Portal Cam and increase the quality in specific regions. ❌ On one side of the battlements, the handrail came up kind of doubled. It seems as if the scanning that took place while walking on the top Vs the scanning on the ground level didn't align perfectly. If anybody has ideas on how to improve that, let me know. My plan is not to clean it up and see if I can do some scenic relighting using @VolingaAI 🤔. In case you have any other questions, let me know in the comments