Clemens Winter

1/ Excited to share my new blog post on Deep Reinforcement Learning and the Entity Neural Network (ENN) project! It's all about making RL easier to apply in complex simulated environments. Check it out: clemenswinter.com/2023/04/14/ent…

The simple beauty of XOR floating point compression: clemenswinter.com/2024/04/07/the…

8/ A big thank you to my incredible collaborators: @Bam4d , @vwxyzjn , Théo Matricon, and Anssi Kanervisto. Your expertise and hard work have been invaluable in bringing the ENN project to life. It's been a pleasure working with you all!

7/ I'm looking forward to seeing the amazing projects and applications you'll create using the ENN project. Let's unlock the full potential of Deep Reinforcement Learning together!

1/ Excited to share my new blog post on Deep Reinforcement Learning and the Entity Neural Network (ENN) project! It's all about making RL easier to apply in complex simulated environments. Check it out: clemenswinter.com/2023/04/14/ent…

Excited about our new model SPT-NRTL to predict thermodynamically consistent molecular properties using physics-guided machine learning. Do you ever struggle to get high-accuracy predictions for activity coefficients? Check out our new pre-print: arxiv.org/abs/2209.04135 1/8

A year ago my brother @ClemensWinter introduced me to machine learning. Today our first paper is out! @AndreBardow @JoSchllng See how physical models, experimental data and ML create highly accurate and efficient models for molecular property prediction! arxiv.org/abs/2206.07048

In the final article of my CodeCraft series, I explore the broader implications of deep RL to the future of video games: clemenswinter.com/2021/08/15/mac…

🚨 I've a paper out today: Scaling Scaling Laws with Board Games! 🚨 arxiv.org/abs/2104.03113 Principle result is that by studying a sequence of small problems in ML, I could predict the outcome of experiments on orders-of-magnitude larger problems 🤯

@vwxyzjn Actually, correction: I think the latest version might not compute the policy network for inactive drones. Inactive drones will still be observed by other drones and the value function.

@vwxyzjn When a drone is building I mask out every action other than the "no-action" action. I still compute the latent state for the drone because it's useful for the value function.

Training deep reinforcement learning agents for the CodeCraft real-time strategy games within hours on a single GPU: clemenswinter.com/2021/03/24/mas…

@vwxyzjn larger maps you probably want to focus a lot more heavily on building out production capacity in the early game.

@vwxyzjn Agents can handle arbitrarily large maps in principle. I have not studied this rigorously, but I would expect them to do quite well on maps several times larger they weren't trained on. This breaks down somewhat once maps require fundamentally different strategies. E.g. on 10x...

@vwxyzjn It does scout, but without the tile feature scouting doesn't work that well on larger maps. Agents explore much less efficiently and usually just end up moving around in the same areas rather than exploring every part of the map.

@ClemensWinter Another interesting observation feature I found was the "tile" feature that helps with scouting. Does the agent scout at all without this feature?

@vwxyzjn Nice, thanks for the recommendation! My REST API turned out to be a major pain point and significantly restricts performance now, so jpype would likely have been a much better choice.

@ClemensWinter We use jpype to do the talking between JAVA and Python in github.com/vwxyzjn/gym-mi…, a similar project to yours, and it speeds up things like at least 10x 🙂.