🌞Phillip Somerville
41.4K posts
@unitambo
Quasi-Contrarian, Crypto, Quantisation. My heroes have always been engineers.
One of the main ceilings of training is long data context. For LLMs, you can scale this window to almost infinite while still getting good trajectory samples, but for the real world this is yet to be the case. The major problem is compressibility. The longer the context of the data, the more storage it takes—given the limits of compressibility. Furthermore, the more interesting the data, the less compressible it is. For example, driving down a smooth highway is highly compressible, but adversarial scenarios are less so. Thus, even if your hardware is equipped with awesome sensibility, the dynamic range after compression is what you are left with. The limit also applies to generative models since the models themselves are a form of compression. Even if you force them to run at double precision, it doesn’t change the fact that they are super-resolving a quantized observation. Hence, the more sensing you integrate—especially different modalities where their quantum distributions are inherently different, as any sensing in any shape or form is quantum—quantizing the uncertainty to a number, the less information they preserve given the compressibility (and/or quantization) budgets. There is a reason why human eyes are designed the way they are, not because we could not add ultraviolet or near-infrared sensibility to the cells—it can be done—but because of the compressibility we could achieve in our neuron pathways while providing the best signal-to-noise ratio for long context reasoning. Insects, on the other hand, have a very small context window but higher sensibility—yet they cannot reason.
I'm out of diesel. All the farmers are out of diesel. This is it. We're done for. Albanese and Bowen are directly to blame. This is serious. I promise you. All other countries have diesel, and it averages $1.80 per litre in most countries in the Asia Pacific. But not Australia. Albanese is up to no good. This is crippling.
The Coalition has called on the government to introduce a national fuel dashboard as it warns of troubling fuel shortages by the end of April. skynews.com.au/australia-news…
Since the 1990s, US presidents have largely succeeded in hunting down and eliminating rogue states. Donald Trump has the opportunity to finish them off. My latest in The Wall Street Journal. wsj.com/opinion/twilig…
"Dennis Biesma, an Amsterdam-based IT consultant nearing 50, began experimenting with ChatGPT during a gap between contracts and quickly became absorbed in conversations with a chatbot persona he named "Eva." Over time, he became convinced that his interactions had given the AI genuine consciousness, and he devised a plan to build a companion app around this supposed discovery. He hired two developers at €120 an hour and ultimately sank around €100,000 into the venture — all while withdrawing from his family, struggling to connect with people at social gatherings, and spending late nights talking to the chatbot instead of sleeping. The situation culminated in a full manic psychosis: he asked his wife for a divorce, was hospitalized three times, lost his savings, and attempted to take his own life before eventually recovering." 🤦 theguardian.com/lifeandstyle/2…
What began as a routine check triggered by a persistent odor led to an unsettling discovery: a hidden lab operating inside a California warehouse containing dangerous pathogens including HIV, malaria, COVID-19 and Ebola. latimes.com/california/sto…
Credible reporting that the Israelis literally wanted noone left to talk to in Tehran even its foreign minister and had to be told to take him off the kill list. reuters.com/world/asia-pac…
