Haruki Nishimura (@imp_aa) - Профиль Twitter

Закреплённый твит

A huge shout-out to TRI's VLA team for the public release of VLA Foundry! You can take full control of VLA training with this fully open-sourced codebase, which comes with a nice GUI dashboard with rigorous policy comparison powered by STEP🪜 tri-ml.github.io/step/

Jean Mercat@MercatJean

Releasing VLA Foundry: an open-source framework that unifies LLM, VLM, and VLA training in a single codebase. End-to-end control from language pretraining to action-expert fine-tuning — no more stitching together incompatible repos.

English

1

4

44

7.6K

Haruki Nishimura ретвитнул

Sergey Zakharov@ZakharovSergeyN·3d

Releasing RecGen: a collaboration between @ToyotaResearch, @toyota_europe, and @UvA_Amsterdam tackling a core 3D vision challenge: reconstructing complete multi-object scenes (parts, poses, textures, even occluded geometry) from just 1 to a few RGB-D views. Trained purely on synthetic data, RecGen achieves SOTA on real-world robotics and 6D pose benchmarks, handling occlusions, symmetry, and complex interactions. A step toward scalable, high-fidelity digital twins for robotics, and better evaluation and training of generalist policies. reconstruction-by-generation.github.io

English

2

34

221

25.8K

Haruki Nishimura ретвитнул

Anirudha Majumdar@Majumdar_Ani·6d

I was thrilled to be back at @MIT for the Robotics Seminar! The talk recording is available now: Rethinking Robot Safety & Alignment in the Era of Generalist Policies youtu.be/pZM8sgLAye0?si…

YouTube

English

0

5

66

9K

Haruki Nishimura ретвитнул

Katherine Liu@robo_kat·23 Nis

A few interesting rollouts from the Foundry-QwenVLA-2.5B multi-task model on seen tasks in sim – a 🧵. I really like behaviors that involve non-prehensile manipulation, like the little nudges in StoreCerealBoxUnderShelf.

Jean Mercat@MercatJean

Releasing VLA Foundry: an open-source framework that unifies LLM, VLM, and VLA training in a single codebase. End-to-end control from language pretraining to action-expert fine-tuning — no more stitching together incompatible repos.

English

2

20

119

14.5K

Haruki Nishimura ретвитнул

Sedrick Keh@sedrickkeh2·23 Nis

Having control over upstream LLM/VLM training is key to training a good robotics model. We hope VLA Foundry opens the door for researchers and practitioners to answer questions they previously wouldn’t even have thought of asking if upstream pretraining was simply inherited!

Jean Mercat@MercatJean

Releasing VLA Foundry: an open-source framework that unifies LLM, VLM, and VLA training in a single codebase. End-to-end control from language pretraining to action-expert fine-tuning — no more stitching together incompatible repos.

English

2

6

29

3.4K

Haruki Nishimura ретвитнул

Shun Iwase@s1wase·22 Nis

TRIで最後に関わったプロジェクトである、VLA Foundryがついにリリースされました！異なる言語モデルやビジョンモデルを手軽に試せるだけでなく、Drake + Blenderを用いたシミュレーション環境で複数タスクの評価も簡単に行えます。ぜひ試してみてください！

Jean Mercat@MercatJean

Releasing VLA Foundry: an open-source framework that unifies LLM, VLM, and VLA training in a single codebase. End-to-end control from language pretraining to action-expert fine-tuning — no more stitching together incompatible repos.

日本語

0

17

116

14.8K

Haruki Nishimura@imp_aa·23 Nis

This is hugely based on @das_princeton's implementation that came out of the collaboration between TLU tri.global/trustworthy-le… and @Majumdar_Ani's group at Princeton out of an internship project!

Katherine Liu@robo_kat

This is actually a pretty big deal — we rely on @imp_aa’s implementations to tell when policies are statistically different than each other. If someone presents some quick mean-only results internally without the CLD analysis, you can be sure someone will eventually ask for it.

English

0

1

5

804

Haruki Nishimura ретвитнул

Jean Mercat@MercatJean·22 Nis

Releasing VLA Foundry: an open-source framework that unifies LLM, VLM, and VLA training in a single codebase. End-to-end control from language pretraining to action-expert fine-tuning — no more stitching together incompatible repos.

English

10

77

488

73K

Haruki Nishimura@imp_aa·23 Nis

See also: "Statistical Thinking for Robot Policy Evaluation: From Rigorous A/B Testing to Effective Visualization" medium.com/toyotaresearch…

English

0

2

237

Haruki Nishimura@imp_aa·23 Nis

A huge shout-out to TRI's VLA team for the public release of VLA Foundry! You can take full control of VLA training with this fully open-sourced codebase, which comes with a nice GUI dashboard with rigorous policy comparison powered by STEP🪜 tri-ml.github.io/step/

Jean Mercat@MercatJean

Releasing VLA Foundry: an open-source framework that unifies LLM, VLM, and VLA training in a single codebase. End-to-end control from language pretraining to action-expert fine-tuning — no more stitching together incompatible repos.

English

1

4

44

7.6K

Haruki Nishimura ретвитнул

Anirudha Majumdar@Majumdar_Ani·14 Nis

Great to see @LeRobotHF using STEP as a tool for statistically rigorous policy comparison! arxiv.org/abs/2503.10966

LeRobot@LeRobotHF

Releasing the Unfolding Robotics blog! Time to unfold robotics: we trained a robot to fold clothes using 8 bimanual setups, 100+ hours of demonstrations, and 5k+ GPU hours. Flashy robot demos are everywhere. But you rarely see the real story: the data, the failures, the engineering. We’re sharing everything: code, data, and details in the blog → huggingface.co/spaces/lerobot…

English

0

3

36

6.2K

Haruki Nishimura@imp_aa·13 Nis

Congrats to the @LeRobotHF team on this remarkable contribution to the robotics community by open-sourcing "everything" including code, data, and all the valuable knowledge! Our TLU team at TRI is fortunate to have collaborated on statistical evaluation and analysis.

LeRobot@LeRobotHF

Releasing the Unfolding Robotics blog! Time to unfold robotics: we trained a robot to fold clothes using 8 bimanual setups, 100+ hours of demonstrations, and 5k+ GPU hours. Flashy robot demos are everywhere. But you rarely see the real story: the data, the failures, the engineering. We’re sharing everything: code, data, and details in the blog → huggingface.co/spaces/lerobot…

English

1

2

9

907

Haruki Nishimura ретвитнул

Zubair Irshad@mzubairirshad·3 Nis

A really solid step toward scalable, high-quality robot data collection — Raiden, from colleagues at TRI @ZakharovSergeyN (and led by @s1wase) lowering the barrier to entry for bimanual data collection, with support for leader–follower setups and SpaceMouse teleop. Big highlight - it natively supports camera calibration and integrates TRI’s learned stereo depth model out of the box, with strong improvements over vanilla ZED SDK. If you're working on robot learning or data collection pipelines, definitely worth a look👇 tri-ml.github.io/raiden/

Sergey Zakharov@ZakharovSergeyN

Our 3D Vision team (3DGR) is releasing Raiden — a data collection toolkit for YAM robots. Built for scalable, high-quality data: supports leader–follower + SpaceMouse teleop, multi-camera setups, and modern stereo depth (incl. TRI learned stereo). tri-ml.github.io/raiden/

English

2

17

175

15.2K

Haruki Nishimura ретвитнул

Jean Mercat@MercatJean·25 Mar

Baking without premix.

English

1

6

23

9.9K

Haruki Nishimura@imp_aa·11 Mar

@MashaItkina @das_princeton @Majumdar_Ani STEP is open-sourced here: tri-ml.github.io/step/ Explore the new plotting tool and tutorial here: lnkd.in/gBReeEdH Working examples of our statistical analysis tool can be found in the recent co-training study here: arxiv.org/abs/2602.01067

English

0

1

101

Haruki Nishimura@imp_aa·11 Mar

@MashaItkina @das_princeton @Majumdar_Ani We also highlight our open-source, plug-and-play plotting tool in Python, which extends STEP to multi-policy comparisons and concisely visualizes the output of the statistical testing.

English

1

0

1

146

Haruki Nishimura@imp_aa·11 Mar

Are you about to evaluate robot policies for your next paper, comparing your policy with baselines? Take a moment to review this article by @MashaItkina and myself, introducing practical tips on rigorous statistical analysis with easy-to-use Python tools: medium.com/toyotaresearch…

English

1

4

16

2.3K

Haruki Nishimura ретвитнул

Anirudha Majumdar@Majumdar_Ani·9 Mar

Policy evaluation is a major bottleneck in robotics. We need better tools for statistically rigorous and efficient evaluation! Check out this great blog post from Haruki Nishimura (@imp_aa) and @MashaItkina on how @ToyotaResearch uses new techniques we have been working with them on.

English

0

3

56

4.1K

Haruki Nishimura ретвитнул

Nicholas Pfaff@NicholasEPfaff·11 Şub

Meet SceneSmith: An agentic system that generates entire simulation-ready environments from a single text prompt. VLM agents collaborate to build scenes with dozens of objects per room, articulated furniture, and full physics properties. We believe environment generation is no longer the bottleneck for scalable robot training and evaluation in simulation. Website: scenesmith.github.io 👇🧵(1/8)

English

18

80

566

72.6K

Haruki Nishimura ретвитнул

Florian Shkurti@florian_shkurti·5 Eki

Excited to share our #NeurIPS2025✨spotlight✨and @HogoGoli 's first published paper: "STITCH-OPE: Trajectory Stitching with Guided Diffusion for Off-Policy Evaluation".

English

2

8

39

2.9K

Haruki Nishimura

Открыть