@bughuntergeek ofc! Let me know if you run into any troubles getting it set up, I'm happy to help. and thanks for the repost
English
David Kogan
172 posts
David Kogan
@davidkogan_
interpretability & alignment researcher, tryna make ai smarter, faster, but most importantly, safer. Building @saferintel
the matrices Katılım Şubat 2017
310 Takip Edilen47 Takipçiler
@0xSero I’ve been working on this problem too. have a different approach to the prediction side building off some established work on expert prediction, that I think gets around the wall you’re describing. my DMs are open if you wanna chat about it, maybe work on it together
English
I still believe MoEs with cpu offloading can be competitive and bring down costs tremendously.
I hit a wall with my testing, mainly:
How can you predict which experts are going to be active given a prompt’s trajectory?
Anyone interested in digging into this more? Shoot a plan
witcheer@witcheer
MoE vs dense offload on 8GB VRAM MoE offload is 10.8x faster than dense offload on 8GB VRAM. here's the proof. I tested Qwen3.6 35B A3B (MoE, 3B active) vs Qwen3.6 27B (dense, 27B active) on my RTX 4060 Ti 8GB. the numbers: >MoE (-ncmoe 30): 35.4 tok/s >dense (-ngl 20): 3.28 tok/s ratio: 10.8x it gets worse at longer context. at 24K tokens, the gap is 16.7x. MoE has zero context degradation (SSM layers), dense loses -35.4%. why: MoE expert offload keeps the hot path (3B active params) entirely in VRAM. only inactive experts move to CPU when selected. dense layer offload splits every layer across GPU and CPU. every token bounces through PCIe for all 64 layers. the bandwidth bottleneck is fatal. quality is slightly better on dense (5/6 vs 4/6). the 27B model has the best hallucination resistance of all 9 models I tested. if you have 8GB VRAM and a model that doesn't fit: MoE with expert offload, not dense with layer offload.
English
@thsottiaux I'm finding that my usage limits are being depleted much faster than before this reset. Anyone else experiencing this?
English
Codex usage limits are back to normal. For ~2 hours we had some issues where the enforced limits were wrong for the majority of users on a subscription.
We waived usage consumption for that time and things should be set again to exactly to where you left it.
Back to codexing.
Tibo@thsottiaux
Seeing issues where usage limits are out of sync for some Codex users. Apologies and team is investigating.
English
English
@thsottiaux @southphxceleb My codex just reversed the recent reset. Guys, am I the only one affected? I am on PLUS!
English
Let me turn off automatic compaction
Tibo@thsottiaux
For those of you living inside the codex app, what should we prioritize among features, reliability or performance?
English
@thsottiaux my codex usage just suddenly reverted back to 0% (was at 75% less than 2 hours ago, with only one codex /goal session running on xhigh) along with my reset date now back to may 19th instead of may 23rd. is this happening to anyone else?
English
@thsottiaux @emreertunc_ the reset moved my reset date by a couple days as well. was that intended, or is it just a bug?
English
@emreertunc_ Yes. There were some cache issues so we cleared everything again
English
@russe66556 check the second part of this thread! built a tool that scans your computer constantly and blocks the scripts that activate the worm from running on installs. and yeah, scary as hell.
English
@davidkogan_ Thats super scary, do you think there's any way to avoid it?
English
the shai-hulud npm worm hit 170+ packages since last september, including tanstack, mistralai, crowdstrike, & opensearch. it steals your npm tokens and github creds through postinstall scripts & then uses YOUR tokens to infect every other package you maintain. scary stuff.
1/3
English
no dependencies. mit licensed. fully open source. feel free to contribute.
github.com/davidkny22/sha…
English
I couldn't sleep knowing my machine might be vulnerable so I built a guard for it. one bash install, it aliases npm/pnpm/yarn to always run --ignore-scripts, scans your lockfiles against every known malicious version, and runs a background check every 5 min
2/3
English
@AntoineRSX Your site is completely down. Might want to have your agent fix that as soon as you see this.
English
Get Hermes Agent running in less than 5 minutes.
No API key, No bot token, No SSH.
Just running with the skills and plugins adapted to your needs.
English
@VictorTaelin @brainage19 4.5 was absurdly expensive, and if I recall had SOTA emotional intelligence and control over its speech. icr if they disclosed much about why it was so expensive, but I suppose it may have just been a very large model.
maybe one day they'll oss and we'll know why
... prob not
English
@brainage19 that's a good point, 4.5 was really eloquent for whatever reason
why is spud not like it?
what is making it incapable of communicating well
English
@VictorTaelin the smarts of chatgpt 5.5 + the coding skills, personality, and writing/speaking capabilities of claude opus 4.6 + the pricing & efficiency of deepseek v4...
literally the dream
English
is there hope that GPT models will ever talk normally? it feels like a non-English speaking coworker. or more like I need an autistic-to-English dictionary to navigate it. it almost doesn't matter how smart the model is if the communication is that dysfunctional. really sad
English
@BrianSatur24273 @chaumian I mean they're open problems for a reason. but think of it this way. for everything codex tries that DOESNT work, this reveals more and more to it about what the elusive proof must look like. what's the farthest you've gotten? down to a single case unproven?
also feel free to dm
English
@davidkogan_ @chaumian How is it? I did that for two Erdos problems. It ran for 48 hours. Nothing solved. :/
English
when i do this it just keeps attempting to prove some seemingly endless sequence of "next lemmas needed to close the gap", failing, proving a new lemma that it claims gets us closer, then concludes "i cant prove it and will not fabricate a proof. the next lemma needed is ..."
Timothy Gowers @wtgowers@wtgowers
All I did was say things like, "Yes, it would be great if you could explore that idea and see whether you can get it to work," or "Could you rewrite that argument as a LaTeX file in the style of a standard mathematical preprint?"
English
@shikhr_ kimi k2.6 is brilliant. deepseek v4 pro is more economical and generally suffices for most tasks, and is occasionally somewhat faster but it varies heavily. deepseek v4 flash is great as a haiku/gpt-5.4-mini-equivalent subagent and very cheap.
English
@sama The equivalent on /goal in normal ChatGPT would be phenomenal! I've been using pro to try and solve some open conjectures but it keeps stopping short after hitting an obstruction and refining the target lemmas. This could massively accelerate math research.
English
@OnlyFranklns @haider1 Ah ok, thanks! That makes more sense, coming from Hinton.
English
@davidkogan_ @haider1 Hinton holds a distinction and doesn't believe they're exactly equivalent. There's some crossovers/similarities but different substrate and mechanisms nonetheless
English
Geoffrey Hinton says AI reasoning can be real thought because language itself is a form of thinking
Words let humans and AI model almost anything, but human thought goes beyond words — through images, space, and physical movement
"the smarter system is the one that can use all of them"
English