Growvv

basically the job of the harness is to generate high taste training data for the next generation models

I feel like the Chinese labs I visited felt like this. It’s what Ai2 felt like. It’s the most reliable energy for making something amazing (even when you’re an underdog with the number of resources). I’m always super excited to find my next star intern.

人类有一个大模型暂时还没有的能力（很快就有了），就是当你做过一系列的事情后，所积累的这种Know-How，可以让你在别人也做同样的事之后，一眼看出名堂来，结合李博杰最近和DS以及和投资人之间的各种趣事，看到他发的这个，我脑子里一下子浮现出人物画像的三个词：“投机”，“小聪明”，“懒惰”

do you think he’s confused as to why his student’s research is getting 100x more attention

A very interesting data point on whether distillation gives you a competitive advantage. Sonnet 5 is just objectively worse than GLM 5.2, and is likely around a similar size. Sonnet 5 was almost certainly distilled (and not just distilled from tokens, but proper distillation from logits) from Mythos. This should give you an update on how relatively unimportant distillation is, in particular for the Chinese labs. It's just a way to warmstart the model for RL; the American frontier labs do it with human data, Chinese do it with tokens from American models, but they could get to basically exactly the same model without it, it would just be a little bit more expensive and require more human data collection coordination/aggregation

Pika just launched their 4K-VFX skill through MCP and I edited this entire video on Claude Code. I put prompt in the thread so you can try it too:

当 AI 和世界还没有今天这么喧闹的时候，一个名叫 Andrej Karpathy 的年轻人，戴着那时还很时髦的 Google Glass，骑着自行车，从他上学的斯坦福，去他实习的 Google，一边骑，一边录了一段 vlog。 视频拍于 2013 年。那时他还是斯坦福博士生，在 Google 实习，做的项目是 deep learning：搭神经网络，让它理解图片和视频。拍摄工具是 Google Glass。他还自己给它写了应用，出门就戴着。他最喜欢的游戏是玩魔方，其实是他休息的方式。 斯坦福 计算机系的走廊，是 Larry Page 和 Sergey Brin 当年读博时走过的同一条。第一台 Google 服务器，也曾在这栋楼的地下室运行过。 Palo Alto 的路，到今天还是他当年记录的那个样子，破得很稳定。加州人有钱，但就是不用来修路。该咋地咋地。 到了 Google，园区还是那种这家公司刚刚长出来的感觉：餐厅、共享自行车、来参观的游客，还有一头恐龙雕塑。那时的 Google 还在三驾马车（Eric Schmidt带着两个founder）年代，除了 Google Glass，还在做 Google 气球和自动驾驶。 那个年代好像已经很远了。 当年连 Andrej 都要辛苦搭起来的神经网络，现在应该已经很容易了吧。 如果你对那个年代的斯坦福、Google 和 Palo Alto 好奇，这条视频是一个稀有的窗口。