PME

This is only going to get worse. People are starting to wake up to the reality. The reasons are straightforward, but the rabbit hole runs deep... 🧵

I found this to be well written and deeply resonant

@ClickHouseDB Unreported incident. Quietly back online now :/

@ClickHouseDB plase halp

@ClickHouseDB cloud having issues right now? No status updates but something feels a bit off...

This is also why they cannot do data analysis without a rigorous harness. By definition all input data cannot have been memorized or "pre-reasoned" over during training.

@charliermarsh Congrats Charlie!

We've entered into an agreement to join OpenAI as part of the Codex team. I'm incredibly proud of the work we've done so far, incredibly grateful to everyone that's supported us, and incredibly excited to keep building tools that make programming feel different.

The sum of all this is that data analytics is a deceptively hard use case for the state of the art transformer architecture. These problems are solvable, but are incredibly hard harness, data infrastructure, and UX engineering problems. Right now there are probably less than 100 engineers in production who deeply understand how fundamentally hard this problem is. There have been a great many failed attempts already, including Anthropic itself (sunsetted their very early attempt at this after only a few months). If you are going to trust an AI agent for data analytics, you must choose your harness very carefully. (Or you need to invent a new transformer architecture, whichever you prefer) If the harness cannot guarantee solutions for ALL of the aforementioned problems (and more), you are going to find yourself in a similar situation described by this Redditor; and that is not a fun place to be.

5) LLMs are "data blind" -- this is a big one. They can't really "see" the data the way people are assuming they can. This is arguably one of the hardest problems to solve. They are looking at data through a series of keyholes, and worse yet, you do not know which ones. Therefore, the data must be presented to them in such a manner that forces them to actually see and consider all the facets of an arbitrary dataset completely. They cannot be relied on discover it completely on their own. All this must happen while: fitting inside context windows, avoiding attention diffusion (aka noise), or losing coherence over many turns, revisions and progressively larger and more complex data sources.

This is only going to get worse. People are starting to wake up to the reality. The reasons are straightforward, but the rabbit hole runs deep... 🧵

@benhylak pi + codex-5.2 on xhigh is the way

i'm sorry but codex cli is just unusable. i actually like the model.

@tryingET Very cool.

@itsyourcode Different parts of a system to build the system for teaching kids based on their individual needs. Middle school teacher

Agentic coding replaces typing, not thinking. Happy Wednesday!

@tryingET Then VSTO/HTN is your "verifier" step. My process conceptually the same, but my verifier is reviewing the code. It is great that you have found a method that is working for you. What are you building?

@itsyourcode True, but this was just an example. My way of doing is: envision the correct version of your intent. Drill down on what you want to built exactly. The end goal. Is it a cli, core plus adapter or whatever. And then you go in an vsto / htn approach until you are done.

@tryingET Yes, that is a narrowly scoped example. Large systems exceed not only the largest context windows many times over, but also the attention capacity of the model within that window (which is considerably less than the window itself).

Have not came across that with gpt5.4 yet. The only thing that is stopping the LLM to reason about it is context window. And there are a lot of techniques for it to let it not come to it. Fuzz the function you want to replace to see which tests fail and then you have your context

@tryingET The transformer architecture is being optimized, yes. But its fundamental limitation is that it struggles to reason outside of its distribution without external input. Only you can provide that external input if what you are building is sufficiently novel.

@itsyourcode Also we live now in the time where we will see faster and faster iterations of cheaper and better llms come out. Just habe a look at qwen3.5 and minimax 2.7 We are off to the races. Self improving llms based on auto research and improved versions from karparthy

@tryingET That which you cannot verify cannot be reasoned about as the system grows larger. Eventually the LLM suffers this too, and its ability to express your meta-intent degrades. Then who will fix it?

@itsyourcode Might be, but there are only so many hours in a day. And imo you are always thinking about the same: Tech debt, gaps, code smell, user stories, bugs. So I could just feed prompts to address these aspects and I am good? Spending my time on the meta or meta meta level.

Hallucination-free data analysis at the speed of thought. Dirty data. Messy schemas. Mixed types. Impossible joins. Doesn't matter. Next 250 waitlist entries releasing tomorrow. Anyone can operate with the data intelligence of an F1000 enterprise now. For pennies.