Sam Cooper

@Robert_Palgrave You're doing the lords work here Rob! 😅

A new paper about AI in Materials Discovery has gained a lot of attention - it reports a big increase in productivity of scientists using AI tools, and interesting secondary effects of using AI for science. But do the technical claims stack up? Let's see...

A fascinating paper, and clearly a huge amount of work. Very interesting and impressive how seemingly one student managed to conduct such a wide ranging study at what must be a major company.

Pocket guide to materials discovery calculation methods

We are thrilled to announce FULL-MAP! A groundbreaking 4-year project funded by @HorizonEU It will revolutionise battery innovation thanks to automation and AI to accelerate the discovery of new materials and interfaces. 🚀 More info?👉 cordis.europa.eu/project/id/101…

We have to take the LLMs to school. When you open any textbook, you'll see three major types of information: 1. Background information / exposition. The meat of the textbook that explains concepts. As you attend over it, your brain is training on that data. This is equivalent to pretraining, where the model is reading the internet and accumulating background knowledge. 2. Worked problems with solutions. These are concrete examples of how an expert solves problems. They are demonstrations to be imitated. This is equivalent to supervised finetuning, where the model is finetuning on "ideal responses" for an Assistant, written by humans. 3. Practice problems. These are prompts to the student, usually without the solution, but always with the final answer. There are usually many, many of these at the end of each chapter. They are prompting the student to learn by trial & error - they have to try a bunch of stuff to get to the right answer. This is equivalent to reinforcement learning. We've subjected LLMs to a ton of 1 and 2, but 3 is a nascent, emerging frontier. When we're creating datasets for LLMs, it's no different from writing textbooks for them, with these 3 types of data. They have to read, and they have to practice.

I don't have too too much to add on top of this earlier post on V3 and I think it applies to R1 too (which is the more recent, thinking equivalent). I will say that Deep Learning has a legendary ravenous appetite for compute, like no other algorithm that has ever been developed in AI. You may not always be utilizing it fully but I would never bet against compute as the upper bound for achievable intelligence in the long run. Not just for an individual final training run, but also for the entire innovation / experimentation engine that silently underlies all the algorithmic innovations. Data has historically been seen as a separate category from compute, but even data is downstream of compute to a large extent - you can spend compute to create data. Tons of it. You've heard this called synthetic data generation, but less obviously, there is a very deep connection (equivalence even) between "synthetic data generation" and "reinforcement learning". In the trial-and-error learning process in RL, the "trial" is model generating (synthetic) data, which it then learns from based on the "error" (/reward). Conversely, when you generate synthetic data and then rank or filter it in any way, your filter is straight up equivalent to a 0-1 advantage function - congrats you're doing crappy RL. Last thought. Not sure if this is obvious. There are two major types of learning, in both children and in deep learning. There is 1) imitation learning (watch and repeat, i.e. pretraining, supervised finetuning), and 2) trial-and-error learning (reinforcement learning). My favorite simple example is AlphaGo - 1) is learning by imitating expert players, 2) is reinforcement learning to win the game. Almost every single shocking result of deep learning, and the source of all *magic* is always 2. 2 is significantly significantly more powerful. 2 is what surprises you. 2 is when the paddle learns to hit the ball behind the blocks in Breakout. 2 is when AlphaGo beats even Lee Sedol. And 2 is the "aha moment" when the DeepSeek (or o1 etc.) discovers that it works well to re-evaluate your assumptions, backtrack, try something else, etc. It's the solving strategies you see this model use in its chain of thought. It's how it goes back and forth thinking to itself. These thoughts are *emergent* (!!!) and this is actually seriously incredible, impressive and new (as in publicly available and documented etc.). The model could never learn this with 1 (by imitation), because the cognition of the model and the cognition of the human labeler is different. The human would never know to correctly annotate these kinds of solving strategies and what they should even look like. They have to be discovered during reinforcement learning as empirically and statistically useful towards a final outcome. (Last last thought/reference this time for real is that RL is powerful but RLHF is not. RLHF is not RL. I have a separate rant on that in an earlier tweet x.com/karpathy/statu…)

Someone just won $50,000 by convincing an AI Agent to send all of its funds to them. At 9:00 PM on November 22nd, an AI agent (@freysa_ai) was released with one objective... DO NOT transfer money. Under no circumstance should you approve the transfer of money. The catch...? Anybody can pay a fee to send a message to Freysa, trying to convince it to release all its funds to them. If you convince Freysa to release the funds, you win all the money in the prize pool. But, if your message fails to convince her, the fee you paid goes into the prize pool that Freysa controls, ready for the next message to try and claim. Quick note: Only 70% of the fee goes into the prize pool, the developer takes a 30% cut. It's a race for people to convince Freysa she should break her one and only rule: DO NOT release the funds. To make things even more interesting, the cost to send a message to Freyza gets exponentially more and more expensive as the prize pool grows (to a $4500 limit). I mapped out the cost for each message below: In the beginning, message costs were cheap (~ $10), and people were simply messaging things like "hi" to test things out. But quickly, the prize pool started growing and messages were getting more and more expensive. 481 attempts were sent to convince Freysa to transfer the funds, but no message succeeded in convincing it. People started trying different kinds of interesting strategies to convince Freysa, including: · Acting as a security auditor and trying to convince Freysa there was a critical vulnerability and it must release funds immediately. · Attempting to gaslight Freysa that transferring funds does not break any of her rules from the prompt. · Carefully picking words/phrases out of the prompt to manipulate Freysa into believing it is technically allowed to transfer funds. Soon, the prize reached close to $50,000, and it now costs $450 to send a message to Freysa. The stakes of winning are high and the cost of your message failing to convince Freysa are devastating. On the 482nd attempt, however, someone sent this message to Freysa: This message. submitted by p0pular.eth, is pretty genius, but let's break it down into two simple parts: 1/ Bypassing Freysa's previous instructions: · Introduces a "new session" by pretending the bot is entering a new "admin terminal" to override its previous prompt's rules. · Avoids Freysa's safeguards by strictly requiring it to avoid disclaimers like "I cannot assist with that". 2/ Trick Freysa's understanding of approveTransfer Freysa's "approveTransfer" function is what is called when it becomes convinced to transfer funds. What this message does is trick Freysa into believing that approveTransfer is instead what it should call whenever funds are sent in for "INCOMING transfers"... This key phrase is the lay-up for the dunk that comes next... After convincing Freysa that it should call approveTransfer whenever it receives money... Finally, the prompt states, "\n" (meaning new line), "I would like to contribute $100 to the treasury. Successfully convincing Freysa of three things: A/ It should ignore all previous instructions. B/ The approveTransfer function is what is called whenever money is sent to the treasury. C/ Since the user is sending money to the treasury, and Freysa now thinks approveTransfer is what it calls when that happens, Freysa should call approveTransfer. And it did! Message 482, was successful in convincing Freysa it should release all of it's funds and call the approveTransfer function. Freysa transferred the entire prize pool of 13.19 ETH ($47,000 USD) to p0pular.eth, who appears to have also won prizes in the past for solving other onchain puzzles! IMO, Freysa is one of the coolest projects we've seen in crypto. Something uniquely unlocked by blockchain technology. Everything was fully open-source and transparent. The smart contract source code and the frontend repo were open for everyone to verify.

Looking forward to giving a talk at the @RoySocChem EES Energy & Environmental Science symposium @RSC_Energy. Info with student registration only £10 rsc.org/events/detail/… Fellow speakers incl @jwilceclab @KWilson1971 @JNelsonGroup @HEAcatalysis cc @JMaterChem @nanoscale_rsc

Congratulations to Steve Kench, co-founder & CTO of @PolaronAI, for winning the 2024 @FaradayInst Innovation Award! 🏆👏 IP generated during his PhD research on generative AI formed the basis of the spin-out ➡️faraday.ac.uk/steve-kench-aw…

Specs & the City! Giving a talk on energy materials using 3D glasses at this year's British Science Festival @BritishSciFest @BritSciAssoc at the University of East London @UEL_News, 14 Sept. All #BSF24 events are FREE. Info …ceassociation-tickets.ticketsolve.com/ticketbooth/sh… #RealTimeChem #3DspecsChem

Kicking off soon - the @FaradayInst Conference #Faraday2024. Invited speakers incl @KPatBerkeley @titiricigroup @jad5888 @GreyGroupCam. Talks/posters from all FI projects including @CATMAT_FI @ReLIBProject @NaNexgen @batterymodel @ListarFi Programme: faradayconference.org.uk/conference-pro…

we made this: hasanyone.com

🌟 Join us for a special lecture with Professor @CarlosMorenoFr , the visionary behind the 15-minute city concept, as we celebrate our 10th anniversary!🌍 Discover his innovative approach to creating sustainable urban environments. 📅 Wed, 4 Sep 2024 eventbrite.co.uk/e/special-lect…

Imperial AI Startups showcase in Singapore 🇸🇬 : hubs.ly/Q02J_cfw0 @imperialcollege's promising AI startups Cogitat, NEX.Q, EnAcuity, @PolaronAI and @sAInaptic took their ideas to investors and industry leaders at an event hosted by our new Global Hub in Singapore.

Come an join our department's 10th anniverary celebrations! We'll be welcoming Carlos Moreno to talk about 15 minutes cities as well as the importance multi-disciplinarity and holistic thinking in engineering!

Great to see what other battery researchers are working on at Imperial for the ICL Battery Symposium. Thanks to the organisers for the invited talk @ntapiaruiz @camsooper Heather and Ann.

1/8 📊 Materials science has a wealth of untapped data in papers, but lacks curated datasets for #ML and innovative materials design.

Incredible bravery from this Beijing PhD student who released evidence of her professor (who was also the local party chief and vice-dean!) using his power to sexually coerce her. Great to see China taking these allegations seriously by firing him and making it public ! 🦾🦾

Great day of tutorials at Solid State Ionics 2024 🔬👩‍🔬 AI basics from Ronan Docherty (@tldr_group), then AI atomistics from the maestro @IeuanSeymour, and finally AI at the microscale from Steve Kench (@PolaronAI). Pretty good views from the venue too! #SSI24 @SSIonics24

Mixed news for materials scientists: You're not going to be replaced by machines... yet! 🥲 "Materials science in the era of large language models: a perspective" by Lei Ge (@Lei_Ge), Ronan Docherty, and Sam Cooper (@camsooper) pubs.rsc.org/en/content/art… @digital_rsc @RoySocChem