tokenbender

14K posts

tokenbender banner
tokenbender

tokenbender

@tokenbender

Sparse and efficient • Deus eXperiments • 🇮🇳

Katılım Temmuz 2014
953 Takip Edilen12.2K Takipçiler
Sabitlenmiş Tweet
tokenbender
tokenbender@tokenbender·
We are releasing a fully reproducible early preprint of "Prism: Unlocking Language Model Capability Extraction". A trained language model knows many things at once, but deployment usually asks for one behavior at a time. Enterprise scenarios often have few products, workflows, features, or use-cases matter disproportionately. Prism asks and answers a simple question - "Is it possible to isolate and deploy only capabilities that are driven by Pareto principle and cut down costs by a huge margin while preserving most of the performance?" This paper discusses a novel approach to efficiency, understanding model behavior and opens up capability extraction.
tokenbender tweet media
English
21
39
221
23.7K
tokenbender
tokenbender@tokenbender·
this is consistent with my own understanding of fable max and i put it above sol ultra for ml work. however i struggle with underlining the qualitative and exploratory edge it has over other models and so do other benches as well. we need something else to see that pass@8 edge.
Håvard Ihle@htihle

Claude Fable 5 (max) scores 91.9% on WeirdML, a new clear best score. For comparison, if we take the SOTA score on each task, we would get 93.5%, so Fable very consistently scores within a few % of the SOTA in each individual run. It achieves SOTA on 7 of the 17 tasks, and the single worst run is only 7% behind the SOTA. It got only 2 runs per task (as opposed to the usual 5), which makes it more impressive that it gets to SOTA in so many tasks (but also makes it easier to avoid one or two bad runs). I'll show more stats soon.

English
0
0
21
2.6K
tokenbender retweetledi
Jino Rohit
Jino Rohit@jino_rohit·
over the last 6-8 months, ive been trying to move towards the ml systems and ai infra space. these are some of my favorite work ive done - 1. a small python inference engine which is mainly a testbed for me to implement the major inference techniques, its about 600 toks/s on an rtx 4060 ti with prefix caching - github.com/JINO-ROHIT/tac… 2. wrote a blog post that outperforms cublas on ada using cute dsl - jino-rohit.github.io/blogs/08_cute_… 3. a detailed post on ncccl collective communication - jino-rohit.github.io/blogs/11_colle… 4. a blog series on how torch compile works and internal mechanics - jino-rohit.github.io/blogs/ 5. i maintain my notes and experiments of most of the work around ml systems here - github.com/JINO-ROHIT/ml-… 5. open source work in sglang and llm-compressor(vllm). im trying to become a stronger ml systems and inference engineer. what should i spend my next months getting better on?
English
27
48
731
80.6K
tokenbender
tokenbender@tokenbender·
@leothecurious it’s a really good UX now, with high thinking mode so it can reason and look up stuff on the web as well while keeping the experience smooth.
English
0
0
4
74
davinci
davinci@leothecurious·
@tokenbender do we finally have jarvis? x.com/i/status/19167…
davinci@leothecurious

about time we got jarvis at home. i want real-time interactive communication with a multi-modal, reasoning-augmented, tool-using, frontier-level intelligent assistant. it should be able to seamlessly transition cross-platform and have a basic level of control at least similar to siri/google-assistant. from the teasers it seems like gpt-5 will be going in this direction (4o was a very limited PoC), although i'm somewhat more bullish on google wrt deep ecosystem integration.

English
1
0
3
296
tokenbender
tokenbender@tokenbender·
late night gpt live appreciation tweet. banger work, stands apart from anything else that exists anywhere.
English
2
0
30
1K
tokenbender
tokenbender@tokenbender·
re: rsi > models may be able to improve themselves even if it’s a combination of existing ideas absolutely. we have to accede the fact that LLMs are better than humans in combining ideas across time or disciplines. there’s a lot we fail to cover due to the blindspots we develop as researchers. open source is a huge example of this - one group gets successful and publishes something and everyone else becomes a xerox machine for the next few months until some other self respecting soul actually challenges it and makes progress. i am super self-improvement pilled for these models, i don’t think it’s going to be all that’s needed but it is good enough for the short term runaway effect.
steve hsu@hsu_steve

Research produced by LLMs tends to be combinatorial - a synthesis of ideas that already exist in the literature, rather than something truly original. Ofc, only a very small number of researchers produce "truly original" ideas! Terry Tao has lately been remarking that perhaps human researchers need to reevaluate how our own brains do research - maybe it's more combinatorial and less original than we thought! For RSI, there are many old NN and AI ideas that remain to be combined and tested in modern model architectures. Schmidhuber makes this point all the time. In my view, models will someday be able to improve themselves even if they are not as original as the best human researchers.

English
1
0
25
1.5K
tokenbender
tokenbender@tokenbender·
both sol and fable are much better than their predecessors. if you're looking for ideas and nuance in research, use fable max if you already have a clear idea and need a thousand armed spider to weave it, use sol ultra
English
2
0
34
866
tokenbender
tokenbender@tokenbender·
got gpt sol ultra to get started on the same problem that i tested with fable, grok 4.5, gpt 5.5 and it instantly started healing my codebase from prior damage
tokenbender tweet media
tokenbender@tokenbender

lessgoooooooo

English
2
1
41
2K
tokenbender
tokenbender@tokenbender·
you didn't need to do gemini 3.1 pro like that
tokenbender tweet media
English
1
0
37
1.7K
tokenbender
tokenbender@tokenbender·
@wandering_mush any work that exists in isolation and doable by collecting information on a laptop is at risk as at least algorithmic/optimization complexity of problems is beatable now. people define AGI like it should be founding and running a unicorn next.
English
1
0
2
86
Mushroom 🍄‍🟫
Mushroom 🍄‍🟫@wandering_mush·
Really dumb question here but if benches getting hill climbed in hours, doesn't it mean AGI already since I am assume, they will also be very good at compacting information and acting on the compacted information even if it contradicts their internal knowledge also. what is the purpose left of continual learning? wouldn't the usefulness of it be much smaller. Since afaik, the context needed to do most white collar job that is specified to the job isn't a lot. You can theorically do my job very well with 50 pages of information since it takes me a month to transfer to another person that same job. If you execute flawlessly that's. And my job is already at top 25% in terms of income in Vietnam. I don't think continual learning matters that much. 50 pages in comfortably in deep seek context length already
English
1
0
2
110
tokenbender
tokenbender@tokenbender·
few months back i suspected there would be a time by the end of this year where we may start seeing benches getting hill climbed in a matter of hours-days with test time compute. this seems to be true for optimization space problems already. i have a feeling that all benches may need to be continual learning by design in the face of test-time compute boost.
Yoichi Iwata@wata_orz

OpenAI は人間の上位者が最終的に到達した二つの解法を経由し、良い部分を組み合わせたような、新解法に到達しました。 これは作問者である私の想像を遥かに超える高度なものでした。昨年2 位だったので今年の 1 位自体にはそこまで驚きはないかもしれませんが、内容には非常に大きな進歩を感じます。

English
1
0
18
1.8K
tokenbender
tokenbender@tokenbender·
@aespa__init and the way they start their phrases or sentences is also really strange. most common example of this that i’ve seen with humans is when a person’s native language is something else and they translate it directly to english. that’s how it feels at times to me.
English
1
0
2
79
aespa(x)
aespa(x)@aespa__init·
@tokenbender Total waste of time when the llm text just dances around the periphery of the underlying bad prompt/context.
English
1
0
1
30
tokenbender
tokenbender@tokenbender·
i am in partial agreement, totally legit that LLMs bring down the social perception of any phrase they overuse to the point of making it obnoxious. but it’s signal shaped noise that hurts my head and one of the biggest reasons why i stopped using ai in writing completely now.
roon@tszzl

hypothesis: the writing styles of language models are basically fine, they weren’t better in some halcyon before times. we just use them so much that we get annoyed by their mannerisms. they need to have a superhumanly diverse idiolect to not become grating

English
1
0
15
1.2K
tetsuo.mlir
tetsuo.mlir@tetsuo_cpp·
@tokenbender I feel similarly. LLM writing feels like fairy floss in that it’s mostly just air. I don’t like overly terse writing either where every word is loaded as that’s hard to read in a different way, but there’s a balance to be had and LLMs are heavily on one side of that spectrum.
English
1
0
1
63
tokenbender
tokenbender@tokenbender·
@tensorqt desloppifying the future one human clickfarm at a time.
English
0
0
4
212
tensorqt
tensorqt@tensorqt·
yep. stylistic basin of writing by llms will simply enlarge over time until it isn't low entropy enough anymore for even the best slop noticers to catch. note that this has already happened for most of the population, and at times even among those who had previously built a good slop detector
roon@tszzl

hypothesis: the writing styles of language models are basically fine, they weren’t better in some halcyon before times. we just use them so much that we get annoyed by their mannerisms. they need to have a superhumanly diverse idiolect to not become grating

English
4
1
18
2.2K

Keşfet

@leothecurious @fujikanaeda @sloppenheimer @bronzeagepapi @hide0usk0jima @wandering_mush @elonmusk @BarackObama