Daniel May

I really hate this on multiple levels. It feels deceptive to take a model, give it a corporate brand identity and then try to hide the underlying model. This is like turning up to work and seeing your roommate wearing a false moustache pretending to be someone else.

This is good stuff. With the linguistic turn there was a hope that defining man through linguistic structure would do away with metaphysics, but that same effort instilled a kind of essentialism through the search for a unique property to unify all of humanity and separate it from other animals. This led to a lot of admittedly pointless debates as to what really counts as language, because language was now a proxy for personhood ("language is the house of Being" and all that). Understanding language as a metagame, on the other hand, would let us excise these categories away from a hypostasized idea of Language and to evaluate the issues around them separately. In principle, nothing would prevent non-human animals from engaging in a metagame like this (whether they actually do or not, again, is a separate issue that can now be treated accordingly), and at the same time a person's lack of or poor proficiency at language at a language wouldn't imply a lesser moral status. This becomes relevant for the LLM discussion as well. The only real reason why anyone thinks LLMs possess some degree of consciousness is because of this confounding conceptual baggage between language/humanity/personhood/consciousness/soul. Any other kind of software was also "made by language" (a programming language), but no sane person would ever suggest their phone's calculator app is conscious, because it doesn't "talk". Video game characters aren't thought of as ensouled because we understand all the text in the game is preprogrammed too. LLMs can generate their own text outputs, which feels a lot more human-like: you can actually talk to them. However, if language is indeed "a recursive system that imposes, refines, and renegotiates constraints on lower-order communicative interactions." then it's not definitionally a human property and now we can have a more serious discussion about the extent to which LLMs engage in this metagame, without implying there's a certain extent after which we must assume they're conscious (or that they'll continue to attain human properties through language until they attain evilness and somehow kill us all). For instance, if "language is not pre-required for rule-establishment but emerges from it", we could say that an LLM's architecture allows it to identify the already-existing-structure of already-existing-games and to generate outputs that fit reasonably well within the constraints of those games; pre-training would be the process by which it identifies the structural constraints of larger-scale human language games (like English syntax and grammar) and post-training would be the process by which it identifies them for more specialized language games (like talking as a "helpful assistant"). There are probably more useful explanations that can be drawn from this framework but I'll have to give it some more thought.

When I first started at X, this is one of the things I did a deep dive on because I felt it was critical to the integrity of the experienced. I tasked our Threat Disruption team to investigate a number of trends that seemed artificial. The findings: We could not find meaningful examples of foreign interference in US policy discussions, except people gaming rev share in developing countries. This is what motivated the release of the Country of Origin feature and significant changes to the rev share algorithm. The most deranged & divisive replies generally were from residential IPs in the United States—with no signs of using a VPN. Ultimately, X is a reflection of the internet. And that means you will see the full spectrum of human thought. And sometimes the most outrageous takes will catch fire. Having said all of this, there can still be cases of narratives being boosted but the origin of the initial post is almost always domestic and we have hardened our systems in the last 3 months to prevent this.