

Owen Rees-Hayward
6.5K posts

@owen4d
Analytical thinker. Insanely curious person. Dreamer. Believer. Husband. Father // Creator of spark-fires // Data Engineer - all views my own, etc.





1/Pretraining is hitting a data wall; scaling raw web data alone leads to diminishing returns. Today @datologyai shares BeyondWeb, our synthetic data approach & all the learnings from scaling it to trillions of tokens🧑🏼🍳 - 3B LLMs beat 8B models🚀 - Pareto frontier for performance







The new features for projects in ChatGPT are useful... but the fact that memory continues to be universal across all chats is a limitation. We need real control: At the project level, turn memory on/off, choose universal or not. The AI learning specific contexts would be great.









I was on a call a few nights ago with a number of very senior devs. And every one of them used AI to code, and found it faster than working with junior devs. It’s really worth thinking through what that means for the industry. Would love thoughts.

