Nathan Lichtlé

We ran 5,600 hyperparameter sweeps to compare RL algorithms on hidden-information games with billions of states. In our benchmark, we found that properly tuned policy gradient methods, such as PPO, performed the best. Paper: arxiv.org/abs/2502.08938

We could not be prouder of this years BAIR PhD graduates! Follow their journey to see where they go next @baifeng_shi @sea_snell @databoydg @enfleisig @graceluo_ @zhuhl98 @HaozhiQ @jdzamfi @LianJiachen @joshminwookang @junhaobearxiong @kaylolittlejohn @KentKChang @kvablack @KunheYang @lisabdunlap @LongTonyLian @maulik__bhatt @michaelpsenka @nathanlichtle @neerjathakkar @nikita_mehandru @NiklasLauffer @qiyang_li @vbenara05 @vonekels @xiuyu_l @Yiheng_Li_Cal @ZheFu1 Sampada Deglurkar Wei-Jer Chang Yichen Xie Yigit Erginbas bair.berkeley.edu/blog/2026/07/0…

New Blog Post: Scaling Up Reinforcement Learning for Traffic Smoothing: A 100-AV Highway Deployment bair.berkeley.edu/blog/2025/03/2…

This paper has everything: large-scale empirical testing, new benchmarks, and subsequent empirical recommendations that we think might make solving imperfect information games a little easier

Tic-Tac-Toe... but the opponent's moves are hidden. Can you outsmart our top RL agents? Play here: nathanlichtle.com/research/2p0s

Coded in C with the amazing Raylib! Try it out at puffer.gg/assets/tactica…

Fast turn-based tactical combat env I've been working on -- RL training incoming!

@jsuarez Nice! It looks easier to play now

No puffers were harmed in the training of this agent

Can language models be trained to find solutions to as yet unsolved mathematical problems? The answer is yes! Check our new article 🙂 1/n

@EugeneVinitsky Going for a multi-lane version of the trajectory simulator. Maybe a fast highway-env?

@nathanlichtle What's this one?

Another driving sim??

@BonesaiDev This looks so cool

Implemented some data augmentation - rotate, shift and noise, the inference is now pretty accurate for most numbers

We’re open-sourcing and arxiving GPUDrive, a GPU-accelerated 2.5D multi-agent driving simulator that runs at over a million FPS. Hundreds of scenes on one GPU means scalable multi-agent planning

@JamesLucasIT @threadreaderapp unroll

Thread of cool maps you've (probably) never seen before 🧵 1. All roads lead to Rome

Excited to share our work about the MVT, a large-scale highway field test for which we trained autonomous vehicles to smooth out traffic flow using deep reinforcement learning, then deployed our AI controllers onto 100 vehicles in busy morning traffic. Read more below! (1/n)

My amazing co-first authors: @JangKathy @EugeneVinitsky @aditshah00 Read more about our adventures in the RL team: 📄 Paper 1 (Controller design in simulation) - arxiv.org/abs/2401.09666 📄 Paper 2 (100 AV field test) - arxiv.org/abs/2402.17050

Extracting the data from dozens of overhead cameras from a week of experiments is a complex computer vision endeavor. Preliminary results suggest a trend of reduced fuel consumption behind AVs. Many more details and nuances available in the paper!

During training, we placed an agent behind a real-world trajectory and tasked it to optimize the energy consumption of everyone driving behind. The result: significant smoothing and fuel savings (>15%) on stop-and-go waves with only 4% of AVs! (Special thanks to our beloved PPO!)

Optimizing for both energy savings and movement is a careful balance! Cars shouldn’t stop (which is energy-optimal) or open excessive gaps. We introduced dynamic gap thresholds, giving AVs leeway to freely optimize for energy while being constrained to safe and reasonable gaps.