Guy Dar

@torchcompiled But also I feel like softmax is kinda cursed in many ways maybe that's related too?

if this is an issue for asymmetry in the LLM head, would we expect it to similarly apply to the up matrices of FFN? Paper mentions softmax affecting the rank of the representation is a factor here, but curious if activation functions could play a similar role.

@morallawwithin This but unironically

okay but consider this: consciousness is magic, and matrix multiplication is not magic

@skdh "You paper has been flagged for human-generated content"

This is real

@nikicaga That's hardly accurate. Israel's population is quite nationalist true (though not "insanely", certainly in comparison to other non-Western countries). "Not willing to share.." is a blatant inaccuracy to say the least.

Exactly, Israel is the way it is specifically because most of its population is insanely nationalist and has no intention of leaving their holy land (or sharing it with its other indigenous people)

@justalexoki Wife guy bond

this is why they killed him

@StephenKing Dafuq

Just when you think it can’t get worse……it,does.

Appropriate name for an insomnia book

I've had so much potential

@sharut_gupta Great! Happy that it is!!

@guy_dar1 Thanks for sharing this, very relevant : )

[1/n] Do distinct large models admit a simple map that aligns their embedding spaces? We show that across multimodal contrastive models—trained on different data and architectures—an orthogonal map aligns image embeddings. Strikingly, the same map also aligns text embeddings.

@Ofirlin Shabbot

בוט של שבת

@Andrew_Akbashev Kinda wild but seems like the arxiv plot continues almost uninterrupted despite AI tools. That's really surprising

A really dangerous situation. Too many submissions. Too many generated papers. Little responsibility. 1. In 2026, more than 24,000 submissions were made to the International Conference on Machine Learning (ICML). It’s TWO times more than in 2025. To fight it, the organizers now require researchers to pay $100 for every subsequent paper. 2. LLM adoption has increased researcher productivity by 90% (there’s a recent paper in Science). 3. The number of papers is becoming far too high. Submissions to arXiv have risen by 50% since 2022. 4. There are simply not enough reviewers. Plus, many scientists no longer want to invest precious time in it for free. 5. We can’t easily identify AI-made papers from the genuine ones. __ Important words from Paul Ginsparg, a co-founder of arXiv: “AI slop frequently can’t be discriminated just by looking at abstract, or even by just skimming full text. This makes it an “existential threat” to the system.” Basically, we’re getting closer to the tipping point. 📍 Many professors blame the AI. But the problem is likely elsewhere: 1. Without a sufficient number of papers, many PIs can’t get funded. They have to prove their credibility to reviewers. Their proposals have to rely on prior publications. In many countries, there are some informal (or even formal) expectations for how many papers a group with a certain size has to publish to survive (funding-wise). 2. Our students / postdocs need papers if they want to be hired in faculty roles. Yes, some departments hire people with few publications. But the majority still want to ensure their faculty can get funded. If funding is partly a function of papers, this is used in decision-making. 3. The number of papers is important if you want to get high-level awards. Many of them are not given because you published one paper (even if it’s great). They are given because you made a meaningful CONTRIBUTION to the field. How do you make it? Publish more papers. 4. Tenure promotions in many places take the number of your papers into account (often indirectly). Your tenure may get delayed if you don’t publish enough. Not everywhere, but for many mid- to low-ranked universities this story is more or less the same. + There are many more to mention. 📍My opinion: Much of this is rooted in how funding is distributed. There is a strong correlation between the requirements at a university and the funding acquisition criteria. If funding were based ONLY on the quality of published papers, universities would hire people for the quality of their science. If funding agencies strongly discouraged publishing too many papers, universities wouldn’t expect numbers from faculty during promotions. And some supervisors wouldn’t pressure students and postdocs to publish unfinished studies and low-quality data. Yes, we need good detectors of fake papers. But we also need the right policies and better funding allocation criteria.

@ValerioCapraro Have a few problems with the claims here. But one of the biggest confusions that repeat often -- Intelligence is orthogonal to robustness and reliability. Claiming otherwise would be counter to much of the science on human neurodivergent intelligence.

It’s time to make this point neat and clear. Attempts to show that AGI has already been achieved are just plainly wrong. For three reasons: 1) They shift the definition of general intelligence, originally based on robustness, generalization, and reliability, to behavioral alignment with benchmarks. 2) They confuse benchmark performance with capability to handle novelty. Spoiler: these are different. 3) They ignore that the same behavioral output can come from totally different epistemic pipelines. Joint work with @Walter4C and @GaryMarcus Links in the first reply

@yoavgo Not unrelated: arxiv.org/abs/2310.10591

i really like the idea of this paper: instead of interpreting intermediary transformer activations by projecting them to the vocabulary space through "unembedding", look instead for nearest neighbours in a set of intermediary activations resulting from known tokens and contexts.

@josephimperial_ That's truly a misuse of the concept of desk reject, which is of dubious utility as it is

Excited to share that my paper was desk rejected for referencing a table in the appendix #ACL2026

@SwayStar123 To be clear I don't know the details in question abt this case, I just say how it looks to me from outside. But this outsider view isnt arbitrary intuition builds on fuzzy statistics in the end. Statistics may not work at the individual case but u can assume ppl think like that

@SwayStar123 You don't need to be nefarious to not like being quite frankly dunked on (it's not that uncommon that a student does all the work). If you have grievances with your supervisors, even legitimate ones, maybe it's best not to publish them to the world.

v1 of paper vs v2 of paper Some supervisor didnt like this section lol

@daidailoh @SwayStar123 Point being, you don't make them accountable bc saying supervisor did nothing is hardly a shock. Ofc, many times, even most I guess this is not the case, but you don't out them for anything they aren't allowed to do

@daidailoh @SwayStar123 Point is that while being polite gets you nowhere, not being polite gets you negative social points. And it's hardly the case where this bravery is oh so needed. Supervisors are allowed to not contribute, it is much worse if they are not responsive, but I reflect badly on urself

@javi_22025 @SwayStar123 much, but trying to glorify urself on expense of others looks bad to future employers and collaborators. It looks petty. Maybe there was a big story in which one person was wronged and he wanted to make that public, but from an outsider pov it looks like he is the problem

@javi_22025 @SwayStar123 while a few sparse decisions that come from supervisor experience is actually what did it - btw that's why supervisor is last author - finally, it mostly reflects on you, bc like it or not, this is quite common and often assumed to be the case that supervisors don't contribute >>