Frank Dondelinger

i have never come across anyone who worked at google from the beginning to ~2012 being anything other than a great human being. google trained people to be non assholes cuz the founders were very much nice dudes & really cared about everyone that worked there. e.g. they literally handed envelopes of pure cash during christmas to employees.

Research output after tenure drops off a cliff for business, economics, sociology, and other non-lab fields. But it remains high post-tenure in lab-based fields such as chemistry, physics, computer science, and engineering.

It has not been reported much, but I believe ETH Zurich has, as of last week, banned new Master and PhD students who attended a long list of universities in China, Russia, and Iran. 🧵

A common misconception about Transformers is to believe that they're a sequence-processing architecture. They're not. They're a *set-processing* architecture. Transformers are 100% order-agnostic (which was the big innovation compared to RNNs, back in late 2016 -- you compute the full matrix of pairwise token interactions instead of processing one token at a time). The way you add order awareness in a Transformer is at the *feature* level. You literally add to your token embeddings a position embedding / encoding that corresponds to its place in a sequence. The architecture itself just treats the input tokens as a set. There's nothing that prevents you from applying the same trick to 2D grid of tokens or a 3D cube of tokens instead! Instead of adding an embedding for (i * N + j), the position of (i, j) in a flattened grid, just embed i and j and concat the result. Now you have a 2D position embedding that makes your Transformer natively process grids of tokens! This can be useful for ARC-AGI, since ARC-AGI grids are of course 2D grids of discrete tokens, not sequences.

Worth repeating: Do not confuse retrieval with reasoning. Do not confuse rote learning with understanding. Do not confuse accumulated knowledge with intelligence.

I told a colleague that logistic regression is AI and they got mad at me, so I made a chart. Find yourself. I am "Tinder is AI".

This is what AI is for. Just this use case alone is worth every dollar invested in it.