Victor Quach

Did a quick interview with @carolmassar @timsteno at Bloomberg Invest, some quick thoughts on AI for productivity and for markets bloomberg.com/news/videos/20…

A minor lesson I learned. Hudson River Trading has large cafeterias in all of its centers. And an abnormal amount of communal spaces (alcoves, booths, meeting room). You sit for lunch and talk to strangers or old friends. I estimate that premier real estate space (and the chef-served lunches) to cost $20-30m/yr. It is a 100x return/yr. Eating together is how you get people to lower their defenses, talk, trust others, collaborate, create alliances since the Pleistocene. And the lesson is that you can win at technology by going back to very simplest, ancestral things.

HRT AI Labs (HAIL) is building some of the fastest and most predictive deep learning models in trading. There’s only one way to see these systems in action: Join, and help us build them. hudsonrivertrading.com/machine-learni…

Are you a researcher at OAI/Anthropic/etc and tired of overhiring, the orgchart chaos, the lowered talent bar, want to move to NYC, or just want to do something different? Email me, DM me, mail a postcard. We've got a new datacenter full of B200s, tight team, and very successful.

There are few places in the world where you can train deep learning models at scale. Last week at @Stanford, HRT AI Labs (HAIL) senior researcher Marc discussed training foundation models robust to market regime shifts on massive datasets and under low-latency demands.

@DrJimFan The idea of repeatedly sampling at inference time is not novel arxiv.org/pdf/2306.10193. What’s hard is: (for academics) quantifying the improvement and deriving the scaling laws, and (for the industry) productionizing it correctly

OpenAI Strawberry (o1) is out! We are finally seeing the paradigm of inference-time scaling popularized and deployed in production. As Sutton said in the Bitter Lesson, there're only 2 techniques that scale indefinitely with compute: learning & search. It's time to shift focus to the latter. 1. You don't need a huge model to perform reasoning. Lots of parameters are dedicated to memorizing facts, in order to perform well in benchmarks like trivia QA. It is possible to factor out reasoning from knowledge, i.e. a small "reasoning core" that knows how to call tools like browser and code verifier. Pre-training compute may be decreased. 2. A huge amount of compute is shifted to serving inference instead of pre/post-training. LLMs are text-based simulators. By rolling out many possible strategies and scenarios in the simulator, the model will eventually converge to good solutions. The process is a well-studied problem like AlphaGo's monte carlo tree search (MCTS). 3. OpenAI must have figured out the inference scaling law a long time ago, which academia is just recently discovering. Two papers came out on Arxiv a week apart last month: - Large Language Monkeys: Scaling Inference Compute with Repeated Sampling. Brown et al. finds that DeepSeek-Coder increases from 15.9% with one sample to 56% with 250 samples on SWE-Bench, beating Sonnet-3.5. - Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters. Snell et al. finds that PaLM 2-S beats a 14x larger model on MATH with test-time search. 4. Productionizing o1 is much harder than nailing the academic benchmarks. For reasoning problems in the wild, how to decide when to stop searching? What's the reward function? Success criterion? When to call tools like code interpreter in the loop? How to factor in the compute cost of those CPU processes? Their research post didn't share much. 5. Strawberry easily becomes a data flywheel. If the answer is correct, the entire search trace becomes a mini dataset of training examples, which contain both positive and negative rewards. This in turn improves the reasoning core for future versions of GPT, similar to how AlphaGo’s value network — used to evaluate quality of each board position — improves as MCTS generates more and more refined training data.

@ronawang @ronawang DM if you want a referral

my friend just got a quant trading offer & showed me his $500k/year entry-level offer … i chose the wrong industry 🥲

Excited to share a new pre-print on "Conformal Language Modeling". LMs sample generations from an unbounded output space. We extend conformal prediction to handle this sort of prediction process, and give it rigorous performance guarantees. Paper: arxiv.org/abs/2306.10193

@menghua_wu @mayhewsw hahaha yes

@mayhewsw @Varal7 speaking of

when everyone in NLP is having an existential crisis because of GPT-4, but you lived through ELMo/BERT in 2019:

@JamesADiao To be fair, black’s moves are plausible answers to white’s moves if you only consider the last few previous moves. In fact, ChatGPT originally plays 27. … Rg2# (pastebin.com/X6kBRTa9)

ChatGPT playing 4D chess!

Perplexed to announced that Learning to Split with 6-6-8 reviews has been rejected #NeurIPS2022 😂 Nonetheless, I am excited to present our manuscript + code: learn to split any dataset with one line of code to break your model’s generalization. (1/3) github.com/YujiaBao/ls

Very excited to share our work on Conformal Risk Control. We give a calibration algorithm that is not only remarkably simple, but also comes with elegant and powerful theoretical guarantees on its ability to control any monotone risk—not just coverage (like CP). Check it out!

Need to debias your new task? Learn how, from your old one. Check out our #ICML2022 paper "Learning Stable Classifiers by Transferring Unstable Features" with @CodeTerminator and @BarzilayRegina Paper -> arxiv.org/abs/2106.07847 Code -> github.com/YujiaBao/tofu

@unsorsodicorda @MilesCranmer 1e6 is a float while 1_000_000 is an int

@MilesCranmer This is cool, but wouldn't it be better to write it as 1e6? 😉

Today I learned you can write numbers like this in Python (!!) Makes it easier to read long numbers by separating digits into groups, just like 1,000,000. It’s so esoteric that Google Colab doesn’t even color it correctly!

Congratulations to Regina Barzilay, Jameel Clinic AI Faculty Lead, for being elected by @aimbe to its College of Fellows. Dr. Barzilay was nominated for her breakthrough contributions in machine learning for early cancer diagnosis and drug discovery. aimbe.org/college-of-fel…

Adam Yala (@YalaTweets) and his team at Jameel Clinic have published their new paper "Robust Mammography-based Models for Breast Cancer Risk" in @ScienceTM. "Mirai," the mammography-based model, achieves consistent accuracy across diverse populations: youtu.be/pCGnRDf0Fmo

🎉 Papers with Code partners with arXiv! Code links are now shown on arXiv articles, and authors can submit code through arXiv. Read more: medium.com/paperswithcode…

We are excited to announce the AI Cures Conference: Data-driven Clinical Solutions for COVID-19, which will take place online on September 29th. Learn more and register here, spots are limited: aicures.mit.edu/conference

On Friday, August 21 at 8pm ET, Greater Boston’s beloved Coolidge Corner Theater (@thecoolidge) is hosting a virtual premiere of From Controversy to Cure, a documentary film chronicling the biotech boom in Cambridge. Learn more and watch it here: bit.ly/31fqWCU