lalo hernandez

based off of the original blog i wrote to show ray tracing in 15 minutes in pytorch, this new iteration is a blog that describes the same but in jax for tpus. the goal is to show that a ton of problems can be expressed in tpu-compliant ways. link below! x.com/halcyonrayes/s…

No world model is accurate, especially the intuitive one in your head Most video models collapse the future into one deterministic rollout, slowly ManiDreams keeps the uncertainty, and plans over it in a modular framework 😴Dream, 🤔Predict, 📦Constrain rice-robotpi-lab.github.io/ManiDreams/

Latent Encoder-Decoder code base. Fully open sourced! You can train and visualize the latent space. [📍 Save it, to find it later when you need it] Thanks for sharing, Xueyan Zou (@xyz2maureen). Code: lnkd.in/dF_GnJAt Paper: lnkd.in/diya2BCs ——- Weekly robotics and AI insights. Subscribe free: 22astronauts.com

Training a policy across four different hands in NVIDIA Isaac Lab: Leap, Allegro, Shadow, and MANO (human hand). Very cool work from @lfcasas7

let’s break this list down so it’s actually useful • JEPA / H-JEPA: avoids predicting every single pixel (too expensive) and rather predicts in latent space. H-JEPA adds hierarchy - short term details vs long term planning ie. how humans actually learn • I-JEPA: built for very efficient vision models. Masks image patches and predicts the semantics and in doing so bypasses heavy compute of traditional autoencoders • MC-JEPA & V-JEPA: both of these are built for videos. MC-JEPA separates content (what an object is) vs motion (how it moves). V-JEPA masks video features with no text labels making it perfect of action tracking at scale • Audio-JEPA: filters out background noise by treating sounds like visuals • Point-JEPA & 3D-JEPA: used primarily in AVs. Uses LiDAR point clouds & volumetric grids • ACT-JEPA: filters out real world noise to learn manipulation tasks efficiently via imitation learning • V-JEPA 2: predicts future physical states of the world caused by an action before it happens • LeJEPA: replaces techniques like masking with an Energy-Based Model (EBM) which mathematically prevents "feature collapse" & ensures the model scales reliably as data increases • Causal-JEPA: for learning true cause-and-effect physics by applying object level masking • V-JEPA 2.1: great for spatial grounding since it combines a dense predictive loss across image & video • LeWorldModel: built directly on LeJEPA's math but super compact - 15M params • ThinkJEPA: uses dense physical prediction with VLM reasoning. Best used when long-term strategy is needed

Because I do not want to work, I wrote a new tech blog on sim2real for legged locomotion zita-ch.github.io/tech-blog/?pos… It is a brief summary of how I do sim2real, mostly focused on 4 aspects: Asset Contact Actuation Perception

Take a look! Code: github.com/saucesaft/diff… Reference paper: arxiv.org/abs/2404.02887 #robotics #locomotion #differentiablesimulation #mujoco #JAX

I replicated their result in MuJoCo MJX, a pure JAX physics engine, so the entire pipeline runs as a single differentiable JAX program. ANYmal learns to walk from scratch in about 2 hours on a single laptop GPU, no reference trajectories, no motion capture data.

What if physics had a gradient? (1/5)

Excited to share that our paper LASER has been accepted to #CVPR2026! We bridge the gap between high-quality offline reconstruction and real-time streaming. We can now turn SOTA models like VGGT and π³ into streaming systems—training-free.⚡️ Kilometer-scale reconstruction at 14 FPS. (1/n)

really early version, need to iron some things out github.com/saucesaft/recu…

releasing recurrl-jax, a highly customizable recurrent RL library built on JAX/MJX • PPO & A2C with LSTM, GRU, GTrXL • asymmetric actor-critic • batched MJX environments with domain randomization going to be posting more about the development here

a propos of nothing

Really interesting work! Pretrained representations like DINOv2 trained with contrastive losses seem to lie on a high-dimensional sphere. So instead of standard flow matching with straight line paths, we should do Riemannian flow matching constrained on this manifold!

Is pixel prediction the best way to build a world model? Check out VDAWorld, an alternative path to building interpretable, editable, and physically grounded world models. We use a VLM to build a simulation of the scene with the help of a computer vision toolbox.

Introducing MolmoSpaces, a large-scale, fully open platform + benchmark for embodied AI research. 🤖 230k+ indoor scenes, 130k+ object models, & 42M annotated robotic grasps—all in one ecosystem.

The SoftFoot Pro by the Istituto Italiano di Tecnologia mimicks human biomechanics to achieve a more natural gait for its user, without any motor needed.

New paradigm from Kaiming He's team: Drifting Models! With this approach, you can generate a perfect image in a single step. The team trains a "drifting field" that smoothly moves samples toward equilibrium with the real data distribution. The result? A one-step generator that sets a new SOTA on ImageNet 256x256, beating complex multi-step models.