cloud

Life update: moved to the USA on an O-1 visa 🇺🇸 I’m happy to announce I’ll be joining @OpenAI to help build the American Empire. I’ll be working on pioneering the future of world models with Sora through infrastructure development and optimization. While this country has a constantly evolving regulatory/immigration environment, I’m grateful that the doors are still open for high skilled labor. I am immensely grateful to @rauchg, @drewhouston, @stephenbalaban, @evanjconrad, @model_mechanic and @gabriel1 for backing me.

@jparkjmc 🤔

be for real do you guys think this is a sora video or nah

@radbackwards @OpenAI @LukePiette wo n’t

@cloud11665 @OpenAI @LukePiette But you won’t unless you is will

Are you up for a challenge? openai.com/parameter-golf

@radbackwards @OpenAI @LukePiette I will

@OpenAI @LukePiette Will depue

Join us!!

@tenderizzation @init_malachi Watch them make the k in the top-k dynamic too

>linear memory overhead with model depth >let’s solve this by introducing dynamism over which depth entries are needed >top-k on depth entries

@punished_teno 🫡🫡🫡

I bet you’ve never seen this connector. This, is a HyperTransport (HTX) slot. Arguably ahead of its time, it had some amazing latency advantages over PCI-E. Cray (and the broader HPC world) did some really interesting things with it.

@adonis_singh @theo @jxnlco

massive thanks to @theo for giving me the creds to make this run happen! the total run costed ~$210, took ~11 hours, and cost per question was ~$2.2. Not great price to perf but we dont really care about that when using the pro models anyways lol (models with * are no reasoning variants, there might be mistakes with this though, didnt look too much into it)

gpt-5.4-pro gets 90% on eyebench-v2!!

@nicdunz Me too

WHAT

@airkatakana @OpenAIDevs Yeah…

why does codex have 3 "latest" models? why is there a gpt-5.4 but no gpt-5.4 codex? am i supposed to use regular gpt-5.4 in codex? is gpt-5.4 actually gpt-5.4-codex but they just named it differently? it says it's an agentic coding model please help @OpenAIDevs

@deredleritt3r @nitbean You forgot -veryfast

@cloud11665 @nitbean Embrace the chaos instead of rejecting it. I am fully prepared for AGI to be named GPT-7.1-Pro-Codex-Max (xhigh)

They're benchmaxxxing on prinzbench

@deredleritt3r @nitbean Naming models is the hardest part :(

@nitbean Insane model naming schemes are a core component of OpenAI culture

GPT-5.4 is launching, available now in the API and Codex and rolling out over the course of the day in ChatGPT. It's much better at knowledge work and web search, and it has native computer use capabilities. You can steer it mid-response, and it supports 1m tokens of context.

@JasonBotterill No magic, just hard work. We have the best inference team in the world

GPT-5.3 is really fast Its 64% faster than GPT-5.2 on the API. Is OpenAI using Cerebras for this?

Years ago, we wrote a C++ library which implements float parsing (std::from_chars). That is, you go from the string "3.1416" to a number (e.g., of type 'double'). I started this work after realizing that in many cases, float parsing was the bottleneck when parsing number-heavy JSON documents (e.g., geojson files). Our code uses an algorithm that is 4 times faster than old float parsing functions in important cases. There are ports in Java, C#, Rust... BUT up until now, we did not have a straight C implementation (to my knowledge). This is annoying for a project like Redis that uses our C++ code because they need to have a C++ compiler. @antirez initiated a C port with the help of AI for this reason. But I think we have something better. Koleman Nix did a full (hand coded) port to C. I threw exhaustive tests at it and it passes! It is also incredibly fast. And it is just plain C. It is not officially released yet, but you can check it out at github.com/kolemannix/ffc… In some benchmarks, the new C port is the fastest float parser !!! 100 million floats per second, corresponding to 2 GB/s!!! That's not as fast your fast disk, but it is getting there!

@sahuang97 What exactly was the regression?

@cloud11665 i'm just hoping there's no regression which i saw on 5.3-codex in some tasks compared with 5.2🙏

It’s a good model sir

@Lazin @prajdabre I have another project where I maxed it out using io_uring and O_DIRECT

@cloud11665 @prajdabre That was "some exotic setup" in my reply. Have you used direct I/O for this?

Technical interview question: Suppose you have 5 TB worth of text data and you want to count the total number of words, how will you do this?

@Lazin @prajdabre These numbers were taken on my 4x 4TiB gen4 nvme raid0 cluster + nowadays a single gen6 drive can do like 30GiB/s

@cloud11665 @prajdabre you need to read it from disk first, so now way to do 11GiB/s unless you're using RAM drive or some exotic setup

Apple ANE 被成功逆向! 38TOPS 算力其实是数字游戏? 刚刷到博主 maderix 开源了个硬核项目: 逆向 Apple 的私有 API, 绕过 CoreML, 直接在 Apple Neural Engine (ANE) 上实现了神经网络训练! 等会? 啥是 ANE? ANE是苹果芯片内部的神经网络加速单元, M4 上目前已经是 16 核的运算单元了, 官方宣称性能有 38 TOPS. 但一直是黑盒, 你只能通过 CoreML 框架去调用, 没有任何公开接口, 没文档, 没 ISA, 啥都没. 于是这哥们把 CoreML 这层壳给扒了. 用了一些逆向手段(比如 dyld_info 扫描, method swizzling 拦 CoreML 等), 最终逆向出了完整的编译运行流程. 最关键的是, 他还搞通了内存编译路径, 可以直接在内存里把 MIL (类似 NVIDIA 的 PTX) 编译成 ANE 二进制. (方便用ANE训练大模型了) 然后逆向的过程中发现了很多爆炸性信息: 首先ANE本质上是个卷积引擎, 不是矩阵乘法引擎. 同样的计算, 改写成卷积运算吞吐量直接翻 3 倍! (Apple 自己的 ml-ane-transformers 参考实现里就暗示了这个模式, 但从来没有明说) 第二, ANE 内部有大约 32MB 的 SRAM. (做矩阵乘法 scaling 测试发现的性能断崖推测出来了) 第三, 单个算子只能用到 ANE 峰值性能的约 30%. 因为 ANE 的 16 个核是流水线式的, 你只提交一个操作, 大部分核在空转. 得把 16-64 个操作链在一张计算图里一次性提交, 不同的核同时处理图里不同阶段的操作, 利用率才能拉满到 94%. 最后也是最炸裂的发现: "38 TOPS" 是个数字游戏. 作者用 FP16 和 INT8 跑了完全相同的操作, 吞吐量一样. 结论是 ANE 在执行 INT8 时会先反量化到 FP16 再计算, 苹果的 "38 TOPS INT8" 就是拿 19 TFLOPS FP16 乘了个2的数字游戏. 真实峰值就是 19 TFLOPS FP16. 另外个细节: ANE 有硬件级电源门控, 空闲时功耗真的是 0mW, 不是低功耗待机, 是真的完全断电零泄漏, 这电源管理真的牛X. 移动端嗷嗷友好. 当然最主要的其实还是这个过程很有学习价值, 两个blog信息量超大, 我这里写不下, 建议感兴趣的同学直接读原文 inside-the-m4-apple-neural-engine, 我这只能抛砖引玉: 项目地址: github.com/maderix/ANE 博客 Part 1 (逆向工程): maderix.substack.com/p/inside-the-m… 博客 Part 2 (基准测试): maderix.substack.com/p/inside-the-m… #ANE #CoreML #AppleSilicon #NPU训练 #KCORES