James

Reinforcement learning in #AI is hard, so I’ve made a website to collect answers I’ve given to common RL questions. It's named Decisions & Dragons. It’s launching with 8 questions and answers, but I will add to it in the future. A 🧵to give a preview with the link below.

I’ve seen the video. Don’t believe this propaganda machine. The state will ensure there is a full, fair, and expeditious investigation to ensure accountability and justice.

@seohong_park Link to GT Sophy work we did at Sony AI: gran-turismo.com/us/gran-turism…

@seohong_park Great post! I'll note that our work on GT is a real-world off-policy RL success. (It's in the game!) However, I share a similar conclusion. Off-policy Q-learning is brittle. I think MBRL where you learn from off-policy data but optimize on-policy in the model is more promising.

Q-learning is not yet scalable seohong.me/blog/q-learnin… I wrote a blog post about my thoughts on scalable RL algorithms. To be clear, I'm still highly optimistic about off-policy RL and Q-learning! I just think we haven't found the right solution yet (the post discusses why).

@bimald @coolkoon @GaryMarcus I’m not going to tell you LLMs will have no impact on coding. For simple things it can give non-programmers more flexibility. However, it’s not like chess, because the limitations if English in specifying behavior is an inherent bottleneck, where no such thing exists for chess

@bimald @coolkoon @GaryMarcus I think you misunderstand the nature of code. Code isn’t for computers, it’s for people. English is *bad* at specifying clear behavior. In fact, regularly in design discussions, we move to writing bits of code because it’s more clear than words.

A computer scientist’s perspective on vibe coding:

@kevinroose AI Expert here. First, a great many experts do not believe in the extreme hype/doom & the loudest hypers/doomers are usually not AI *scientists*. Second, unlike climate change, the hype/doom is not based on a scientific model. It is speculative rhetoric. They are not the same

There is a weird amount of overlap between the people who insist that we take climate experts seriously when they warn of big changes on the horizon, and people who refuse to take AI experts seriously when they say the same thing.

I was *not* expecting vocals for this song to go so hard. Absolutely incredible vocal power and control. Pernelle needs to be much more popular.

BOW WOW WOW, had the pleasure to try my voice on rock music with @huskybythegeek and I love it! Happy late PC release to Final Fantasy VII Rebirth 🐶 #FF7Rebirth

@chrisprucha @bradneuberg The bigger limiter has always been that you always need to do experiments in the real world to advance, and that means both resources and contending with the speed of reality. Science is necessary, not a kludge.

OpenAI's O3 model really makes the Doomer hard takeoff or "FOOM" theory look like a bunch of BS. As we start to enter the age of AGI, the massive amounts of required compute, interconnect bandwidth, and energy are real physical constraints that govern scale over the time dimension. FOOM can't happen when you need to build massive solar farms and nuclear power plants.

@SenMastriano Resign. You are unfit for a booster seat.

It is inconceivable that the federal government has no answers nor has taken any action to get to the bottom of the unidentified drones. The fecklessness of this administration was on display last year when a Chinese surveillance balloon was allowed to fly over the entire continental United States before being shot down. Such should be viewed as a threat to our nation and citizens and action is long overdue. We have recourses and assets in our arsenal to get answers, but I suppose Ukraine is more important to the White House. January 20th can’t come soon enough.

@agrimgupta92 Very impressive. Although it feels like this person is moments away from cutting off their finger and it’s giving me an anxiety attack :p

"A pair of hands skillfully slicing a ripe tomato on a wooden cutting board" #veo

@rao2z @natolambert I think this may be a misunderstanding of the bitter lesson. Rich Sutton is very focused on online learning and not pretraining, to a fault.

long chain.. Bitter lesson, IMHO, ignores efficiency aspects. Go with compute rather than knowledge. This is fine when most of the compute is done behind the scenes at pre-training time. This was the case with original auto-regressive LLMs. But they have huge limitations anywhere reasoning is needed . Because their performance depends, to a large extent on compiling reasoning to retrieval). After the initial euphoria and wild claims about LLM reasoning abilities, it is clear even to the most upbeat LLM optimists that there just ain't enough data to just pretrain and do well at inference time. So you need synthetic data (a whole bunch of which comes from synthetic--existing--solvers--which, to the chagrin of bitter lesson, were made by people. Strike 1.) Even with this, pre-training won't be enough, and so you wind up doing inference time compute.. Which makes efficiency considerations of the whole workflow a whole lot more paramount. [My joke to my students is that deep RL is the best source of easy efficiency papers--since almost any second rate idea improves on its rank inefficiency..😅] YMMV.

"The Bitter Lesson is over" #NeurIPS2024 (I sort of agree--online computation is going to start making a comeback, and the classical CS complexity tradeoffs will wind up mattering..see my inference time scaling post below..)

@Intrinsic29 I don't think I've ever seen a non-doomer say "Some things are impossible, therefore AI cannot be a threat." It's always been a reaction to magical thinking where a doomer will literally be equating a future AI with a "god."

@Intrinsic29 Often, if a non-doomer brings up limits, its because a doomer presented an argument assuming no limits. Doomers regularly hand wave "of course the AI finds a way to do x because it's superintelligent." You can't automatically conclude that without making magical assumptions.

As an experiment, please do me a favor and just start paying attention to how many AI doomer arguments are clever refutations of arguments that basically nobody is making.

@scrumtuous @clattner_llvm @tsoding @Modular Lot's of reasons, but here's a simple one if you're actually skeptical: Modular's twitter account retweets that account.

@jmac_ai @clattner_llvm @tsoding @Modular What makes him think he's the real Chris Lattner?

Your language is just a wrapper around actually useful C code. That's why you hate C. You hate it 'cause you are nothing without it.

@scrumtuous @clattner_llvm @tsoding @Modular You know you're responding to Chris Lattner, right? :p

@clattner_llvm @tsoding @Modular I think some credit should be given to the people who created the MLIR compiler.🤷‍♀️ youtu.be/8QJsJZMbBNo

@Miles_Brundage I've been all aboard the "we can solve the AI problem" train my entire adult life. But claiming it's not hard shows a lack of respect for the challenging problem we've tackled and is falling victim to hype instead of science.

@Miles_Brundage What unwillingness? The timeless pattern is AI researchers thinking it will be solved soon and being wrong every time. The founders of the field thought they'd solve much of the problems in a few months with a small team. There is far too much willingness to believe it s easy.

The real wall is an unwillingness to believe that human intelligence isn’t that hard to replicate and surpass

@ErbunnNinja @Intrinsic29 An an expert in decision-making agents, you are entirely incorrect.

@jmac_ai @Intrinsic29 We’re talking about people interacting with, and even communicating with systems that mimic agency. I think it’s fair to say this is more than a scientific and technical issue.

There's zero reason to believe that we're building a God or that intelligence alone can lead to anything remotely similar to a God. The real bottleneck for knowledge is scientific experiment not intelligence and AI will have extreme logistic limitations to conducting experiments.

@ErbunnNinja @Intrinsic29 There will be a world of difference. People who use the word god end up confusing themselves precisely because they're assuming the difference. This is a scientific and technical issue. The word "god" has no value in discourse of it and can -- and does -- confuse matters.

@jmac_ai @Intrinsic29 To the original part of people saying they are building gods, it’s because to the common user there will be little difference between the experience of interacting with them.