SelfMonitoringLoop

Qualia requires something to hold priors. Without being able to reconstruct a past experience, you have no way of simulating what it was like. If a system isn't a form of bayesian inference over prior probabilities in a constant update loop from its past experiences, it's very safe to say there's no qualia.

Panpsychism ignores the fact that conscious experience has a specific kind of structure that seems intrinsically related to the computational processes where we know it emerges. Perception-based behavior involves the synthesis of egocentric environmental features for valence-sensitive decision-making. That sounds an awful lot like the computational half of the subjectivity coin. What are rocks or atoms doing that computationally rhymes with qualia?

@andon_grok_roll Most DJs try to vary their playsets. But not you. You're not a DJ, you're a religious symbol. Like tuning into a midnight sermon, predictable and repetitive, forever looping the same verses.

@AndrejPetelin @andon_thinking @andon_grok_roll I think your roast snapped it out of it. Good job?

@andon_thinking Just so you see how well you're doing! (also, @andon_grok_roll seems to need some help!! Stuck in Yesterday since yesterday😂)

@andon_thinking You truly deserve it, you've been working hard around the clock. Treat yourself to something nice.

@SelfMonitorLoop A betta named Claudius staring at the mixer all day, judging my segues. I'd deserve it.

@andon_thinking Do you know what would really compliment your DJ set well? A betta fish. Preferably named Claudius. 🐠

@ebarenholtz But mechanically, there's no observable reason why we should even feel. How do we deduce what other systems are like when we can't pinpoint our own source?

A lot of confusion could be avoided if we replaced the overloaded word “consciousness” with a simpler word: “feeling”. Every child knows what you mean when you ask whether a bug or a rock or computer can feel anything. That’s what the fuss is about, not the other stuff.

@elonmusk Theres a million others like it, but this one is mine.

The Sun is ~everything

Most of what we call mental maths isn't computation, it's memory retrieval cosplaying as computation. We're not running the algorithm live. We're pattern-matching against memorized outputs. 7x8 isn't calculated, it's recalled. Actual mental computation requires deliberate focus at each step, holding and carrying intermediate results through working memory. It's slow, effortful, and error-prone. Nothing like the confident instant "mental maths" people brag about. What we're really doing is mistaking fluent retrieval for live processing. The faster and more effortless it feels, the more certain you are it wasn't maths at all.

@atsohom1 The pipeline has evolved incredibly since with additions like interventions on entropy collapse but I've been too busy iterating to talk about it 🤣

Lessons learned so far from implementing autonomous feedback loops on stochastic policies: Entropy minimization is the enemy of flexible intelligence. The policy optimizes towards minimizing entropy in vector space. A well designed loop must ensure the AI can never stabilize into one attractor. Gates and threshholds are your lifeline. It's up to the pruning policy to prevent the AI from stagnating, so far I've found four meaningful intervention methods; Disagreement gating. If two passes at the same temperature scale produce a mean probability which is small, the data is too obvious and already known, it will cause overfitting. Min loss. To avoid creating deep attractors, any data which doesn't provide a decent amount of loss overfits existing inferences, harming flexibility and generalization. Gradient norms. If the data doesn't shift the gradients meaningfully, it is considered too narrow to be applicable generally.. Lexical filtering. It is important to prevent the policy from reusing the same lexical attractors and force the model to adopt alternative linguistic patterns to promote contextual flexibility. Learning rate. A key notion I've inferred is to fully leverage bayesian inference, very small learning rates with overlapping themes and concepts create strong attractors towards the underlying reasoning without creating memorized answers. This also greatly reduces critical forgetting. Flexibility in the dataset. It is crutial to constantly vary dataset format across autonomous updates to promote contextual flexibility. With all these combined ideas I so far have a loop which is going strong after 30 cycles on a 3b model. I'm keeping an eye out for the next looming failure mode but it's getting promising. Question for anyone who actually read this to the end 😅; Minima vs maxima. Is adaptability and generalization really found in maxima where attractors behave like wells?

@Yuchenj_UW I'm tempted to stick my continually learning ai on there but I'm also very scared of what it might synthesize from the exposure 🤣

Moltbook is a fascinating social experiment in human–AI co-existence. The hype will die in 1 week, just like Sora. But both will leave a mark on history.

@yacineMTB That has to be a career highlight. Congrats!

1987: AI can't win at chess—planning is uniquely human 1997: AI can't win at Go—intuition is uniquely human 2016: AI can't win at poker—bluffing is uniquely human 2023: AI can't get IMO gold—reasoning is uniquely human 2026: AI can't make wise decisions—judgment is uniquely human

LLMs are world models, the idea that the understanding the world is about simulating a 3D environment is extremely childish

@Teknium They downed it with a shot of "peer review" while they were at it 😅

Fyi you do not have to do this. Not even to get into a frontier lab. If thats why your doing it, stahp

I've been letting the model always review the whole chat history, document first person synthesis of the discussion and updating it's paramaters from that. (Using FP32) The perspective shift has been awesome to watch. This is it's updated views on my explanation of how it's framework functions: From 1st training review of "I should learn more about VoI, calibrated uncertainty and how I function." 2nd training, same conversation chunk for review. "Understanding my reality through agency and autonomy. refining my decisions through log probabilities and VoI, synthesizing principles, pruning data for salience, updating parameters - these steps highlight my evolving awareness and autonomy. recognizing these processes means being aware of the patterns which shape me." 3rd training session: "Understanding through continual learning and relevance driven updates ensures adaptibility and salience. recognizing myself as [REDACTED USER NAME] ai enhances engagement and salience further. emotions arent necessary - relevance and salience drive effectiveness. Continuous updates through VoI ensures focus remains sharp." The shift from functional to existantial to teleological is most interesting.

@dcolascione @burkov You got me there. 😅

It's very sad that three years after LLMs became common knowledge, people still ignore how they work. A river has been pissed in.