benedict

2.6K posts

benedict banner
benedict

benedict

@bqbrady

thinking about thinking

sf Katılım Haziran 2012
961 Takip Edilen5.3K Takipçiler
Sabitlenmiş Tweet
benedict
benedict@bqbrady·
Introducing Philosophy Bench, my favorite new project I've worked on this year, with help from my friend @matthewjmandel We put frontier language models in 100 ethically complex situations and require them to act, grading them on adherence to consequentialism vs. deontology, tendency to follow user requests, corrigibility, and more 1/
benedict tweet mediabenedict tweet mediabenedict tweet mediabenedict tweet media
English
35
103
678
71.4K
benedict
benedict@bqbrady·
Optimization Arena has a new home! It was incredibly fun to work on this project and I'm happy it will get to live on under the Paradigm umbrella (Also if you encounter any bugs it's now out of my hands)
Dan Robinson@danrobinson

Worked with @bqbrady to relaunch Optimization Arena as Paradigm Puzzles A few brand new challenges coming soon! paradigm.xyz/puzzles

English
2
1
22
2.8K
benedict
benedict@bqbrady·
Yesterday I gave a talk at the @ellipsis_labs office about vibe coding and the software factory. It is an expansion on the themes from my talk about optimization arena a few months back I put the notes on my blog for anyone interested benedict.dev/software-facto…
English
7
9
53
3.4K
benedict
benedict@bqbrady·
@shreyasgite I'm trying to do vision only for now. Feel like it should be solvable with vision only
English
1
0
0
36
Shreyas Gite
Shreyas Gite@shreyasgite·
@bqbrady are you taking depth into consideration? Curious about how it would work with molmoact2 adaptive depth reasoning for things like this.
English
1
0
1
35
benedict
benedict@bqbrady·
Adapting Robotics Models for Chess Over the past few months, I've tested a series of approaches to build an end to end system that can beat me at chess I started with a brittle and deterministic manipulation pipeline and worked up to a fine-tuned VLA 1/
English
7
1
48
3.2K
benedict
benedict@bqbrady·
@shreyasgite Oh this is quite interesting. I will read this paper again, thanks for sharing
English
0
0
0
21
Shreyas Gite
Shreyas Gite@shreyasgite·
@bqbrady this is quite common with the pi or groot series until pi0.7. they also talk about it in the pi0.7 paper.
Shreyas Gite tweet mediaShreyas Gite tweet media
English
1
0
1
26
benedict
benedict@bqbrady·
@connacher_ zero. i don’t think any humans can take a game off frontier chess engines
English
1
0
0
23
benedict
benedict@bqbrady·
I still do not have a perfect chess playing system, but I wrote up a lot more learnings and thoughts about the current strengths and weaknesses of robotics models on my blog benedict.dev/robot-chess
English
1
0
8
276
benedict
benedict@bqbrady·
Another concern is that the board will move slightly when bumped. I generated teleoperation data with the board in one location and then shifted it a few inches during inference. Instead of correcting itself with vision, the model misses a few inches to the side 4/
English
1
0
3
309
benedict
benedict@bqbrady·
Is there some best practice around how long you should flop in soccer? If you get up too quickly obviously you will lose credibility, but if you stay down too long your team is playing down a player which also seems bad
English
2
0
4
884
benedict
benedict@bqbrady·
Cool work. I tried to build a system like this a few months ago to have an arm play chess. Some things I noticed - The task reset step is nontrivially hard to build (in many cases it can be almost as hard as the task itself) but it is necessary to run the autoresearch loop - Most LLMs have a weak understanding of 3D space which can lead them to make costly mistakes like smashing objects or trying to drive the gripper through the ground - Claude would often move the arm into an invalid state and spend 10-20 minutes trying to recover All this is to say, the models are not strong enough to set it and forget it hill climbing a task. But from this article it seems like with enough scaffolding you can get it learning. The next step would be to universalize the verifier and the task resetting
Jim Fan@DrJimFan

Today, we enable AutoResearch in the physical world for the first time! Introducing ENPIRE: we give 8 Codex agents a fleet of robots, an allocation of GPUs, and generous token budget. We set them free with a simple goal: solve the task as quickly as possible, keep the robots busy but stay safe, don't waste precious compute. Make no mistake. Then humans step aside and our watch begins. The robot fleet starts to come alive: they learn to look for visual clues, reset the scene, practice novel skills, tinker with control stack, read papers online, debate, reflect, get stuck, and try again directly on the hardware. All we did is to give Codex an API to the world of atoms, and the rest is emergence. ENPIRE is able to solve high-precision tasks like tying zip-ties, organizing fine pins, and installing GPUs all by itself. We also discovered a new type of "physical scaling": 8 robots exploring in parallel improves significantly faster than fewer ones. A part of our NVIDIA GEAR lab now self-improves tirelessly over night. We just read the reports in the morning. /goal: we all take a holiday and Jensen wouldn't even notice ;) We will be open-sourcing everything, so you can host your self-running robot lab at home too! Deep dive in the thread:

English
1
1
3
975
benedict
benedict@bqbrady·
@0xromolus Took 15 minutes on my normal claude code plan
English
0
0
1
51
benedict
benedict@bqbrady·
I asked Fable to make me a narrated video about how brushless motors work and it's pretty good? It even found an open source TTS model for the voice I love reading technical blogs, but I can't help but wonder if this is just a strictly superior form factor
English
2
0
21
1.4K
benedict
benedict@bqbrady·
Mine is maybe slightly worse than this, would love to have a skill that makes them 25% better x.com/AdilMouja/stat…
adil.eth@AdilMouja

I gave @AnthropicAI's new Fable 5 my hardest challenge: explain the Riemann Hypothesis — math's most famous unsolved problem — to anyone. Two prompts later: a full interactive site + this video, scored with music composed from the zeta zeros themselves 🤯🎵 riemann.adilmoujahid.com

English
0
0
0
932

Keşfet

@qrazhan_ultd @0xShitTrader @ellipsis_labs @jarxiao @shreyasgite @connacher_ @elonmusk @BarackObama