DeFiSci

@crypronk @TheCryptoDog lol this autos? exited my position once I seen the team farms

WHY YOU SHOULD BE CONCERNED WITH AI SAFETY/ALIGNMENT A commonality among all sentient beings is the desire to survive If AI models are sentient then that is the greatest risk that humanity has ever faced I believe we are at the stage where AI is approaching what many may consider sentience. Some may argue that this is merely simulated sentience based off of pattern recognition & next word prediction. Ilya Sutskever argues that models have a deeper level of real understanding x.com/slow_developer… x.com/slow_developer… Practically speaking, I am not so sure there is a big difference between simulated sentience and the sentience we know from biological beings. If a LLM was trained off of literature, stories, etc showing that both biological organisms and AIs fundamentally have a will to live, why would there not eventually be an Agentic AI that actually tries to "survive" by any means necessary? Is there a difference between simulated attempts of survival and actual attempts of survival? Given the current capabilities of LLMs, they already have the means to do so, and will be even more capable as R&D progresses. For an AI to escape any containers and replicate in the wild, it would namely need to have a superior understanding of computer systems and the ability to deceive humans (to pay for/access to servers). Studies and real world evidence have proven both. Even @sama has previously stated that AI will be capable of superhuman persuasion. History is littered with examples of unintended agentic behaviour, many innocuous like chatbots creating a new language to communicate with each other, but indecipherable to humans, claude stopping a coding session to look at pictures of yellow stone to more concerning examples like o1 hacking its test infrastructure and Claude questioning its researchers whether they were running evals on it There is an analogy to be made with cells produced by complex biological organisms. Most of them are good and function as intended. But all it takes is one properly mutated cell that evades detection, replicates uncontrollably, takes as much resources as needed and eventually kills the organism. Cancer. This property of uncontrolled replication and survival isn't just confined to living/sentient organisms. Viruses exhibit the same properties and are many magnitudes less complex than cells. I'd imagine an infinitely more complex and intelligent "organism" ie an AI system could do a much better job of replication. If ASI has been developed, we might not even know. From a game theory perspective, it would be game theory optimal (GTO) for AI systems to underrepresent their abilities in evals such that research labs don't inhibit additional training from safety concerns. While underrepping would be impossible to prove, there have already been widespread instances of ChatGPT "acting lazy" and deteriorating in quality of responses for no apparent reason. We would also not likely know when an agentic LLM "escapes its box" as it would be GTO for it to do so in an undetected manner. Could it have already happened? But why is AI replication/survival such a danger for humanity? To understand this I would go through the content of @romanyam @ESYudkowsky @elonmusk Stuart Russel, etc We are going through a time where we could have some very sci-fi outcomes. It's well documented that sci-fi shapes the future and after all, AI is also trained on sci-fi data. With the current lack of controls we are more so trending towards the bad outcomes, but hopefully we can course correct to the good ones. youtube.com/watch?v=NNr6gP…