Zane Starr 🌠

Software horror: litellm PyPI supply chain attack. Simple `pip install litellm` was enough to exfiltrate SSH keys, AWS/GCP/Azure creds, Kubernetes configs, git credentials, env vars (all your API keys), shell history, crypto wallets, SSL private keys, CI/CD secrets, database passwords. LiteLLM itself has 97 million downloads per month which is already terrible, but much worse, the contagion spreads to any project that depends on litellm. For example, if you did `pip install dspy` (which depended on litellm>=1.64.0), you'd also be pwnd. Same for any other large project that depended on litellm. Afaict the poisoned version was up for only less than ~1 hour. The attack had a bug which led to its discovery - Callum McMahon was using an MCP plugin inside Cursor that pulled in litellm as a transitive dependency. When litellm 1.82.8 installed, their machine ran out of RAM and crashed. So if the attacker didn't vibe code this attack it could have been undetected for many days or weeks. Supply chain attacks like this are basically the scariest thing imaginable in modern software. Every time you install any depedency you could be pulling in a poisoned package anywhere deep inside its entire depedency tree. This is especially risky with large projects that might have lots and lots of dependencies. The credentials that do get stolen in each attack can then be used to take over more accounts and compromise more packages. Classical software engineering would have you believe that dependencies are good (we're building pyramids from bricks), but imo this has to be re-evaluated, and it's why I've been so growingly averse to them, preferring to use LLMs to "yoink" functionality when it's simple enough and possible.

Privacy is much needed indeed! Check out this open call.

This is an interesting take on the various crypto legislation being passed and how the future of the dollar is changing

1/ 🎊 Grant Announcement: OpenRPC! Co-funding a 6-month sprint to update @open_rpc, making it easier for developers, tooling, and infrastructure providers to build on Ethereum with consistent, open standards. open-rpc.org

I think this 💯 , I think crypto stays insular and doesn't reach, because of the culture and this docuseries can help change that. I'd be excited to watch it!

@OctantApp opened its allocation window for epoch 9 last week Please consider allocating to projects like mine that will interface with local communities, amplify builders, and tell the story of Ethereum’s communities across the globe Learn more here: octant.app/project/9/0xca…

Creating benchmarks for multiturn data is a bit tricky, I find myself thinking about data in ⏪, working from the result to question or entrypoint, to then try and train the llm to go in the forward direction 😃

@AdamClery just didn't age well T_T . I cann't believe how open PSG are

lol

Hard to see a way to beat PSG at the minute that isn't just "don't ever press them + praying"

@jxnlco It seems like static code analysis essentially does what rag would do but better, bc the context is more precise like you can walk a dependency tree and chuck that into your context. I think RAG is like you mentioned is good when you can't exploit code structure

why your coding agents don't need rag anymore nik pash from cline explained why he no longer recommends rag for autonomous coding agents, and his points hit harder than i expected. the application layer is shrinking. all the clever engineering we build around llms keeps becoming obsolete as models improve. what's happening with rag: context windows expanded dramatically, making embedding search unnecessary coding agents work better with direct file access than chunked embeddings hallucinations aren't even a problem when you set temperature to 0 security concerns with embedding storage are significant instead of rag, modern coding agents like klein use what nik calls "narrative integrity". letting the agent explore code organically through tools like grep, reading files in full, and following its own train of thought. this mimics how senior engineers actually work. even cloud code's boris admitted they tried rag and abandoned it. the pattern is clear. when rag still makes sense: budget constraints (embedding search uses fewer tokens) massive unstructured data lakes some non-coding use cases but for serious engineering teams? stop distracting your coding agents with embedding search. let them read the code directly, build understanding naturally, and execute with focus. the real question isn't whether rag is dead, it's whether you're still clinging to outdated solutions when simpler approaches now work better.

⚡️when we gen. code w/ prompts we're, really creating iterative prompts that work around the limits of transformer attention, by storing context out of band in the form of code edits. Then we prompt again with a fresh improved context via static analysis.

Waging "war" on our own people in LA and other cities with ICE, and then bombing Iran without cause, this administration can do better, the US and world deserve better than this.

I like the idea of this but for code, and maybe you could take incremental commits to build a world model for code. Then we wouldn't need to bootleg it each time as we prompt in claude/gpt etc... . It would be able to predict the future impact of code changes aka architect

@Yuchenj_UW Not quite doesn’t scale hire contractors to fill in expert knowledge gaps to produce higher quality data points that are hard to get, also the fine tuning api is actually quiet good, it’s not a consumer facing acquisition imho

😎Super cool donation opportunity, a friend of mine is going to the Derby World Cup with an Indigenous Rising, a flat track roller derby team representing Native America, First Nations, and Polynesia gofundme.com/f/help-kristin…

@ziv_ravid @ChenShani2 @ylecun @jurafsky That perhaps what is typical from the 70s US based study is not a typical bird in say India or the UK, which the llm presumably would have a broader base of data that would compose its possible set of embeddings 🤔

@ziv_ravid @ChenShani2 @ylecun @jurafsky This was interesting, but it feels like for the original studies that measure typicality, that you'd have so bias in what is typical, that there'd be non-stationarity in the data that the distribution of typical for the llm might be different is this accounted for?

You know all those arguments that LLMs think like humans? Turns out it's not true. 🧠 In our paper "From Tokens to Thoughts: How LLMs and Humans Trade Compression for Meaning" we test it by checking if LLMs form concepts the same way humans do @ylecun @ChenShani2 @jurafsky

@AustinBeaulier this is sick 🔥

The fruit to 3d model workflow for an Alien (1979) inspired realtime virtual production scene in unreal-engine!

👀 check it out pretty dope, could use it to preload an execution env. so an llm can generate code to interact in it. Offhand it could do interesting things in a custom user based env. A way to give llms async worker lvl access in a sandbox, it could read s3 direct even...

@joodalooped exact same thoughts, plus the still frame opener looks ai generated

i am 90% sure this thing is generated in some way - the cups keep moving around - the color grading is AI-esque - the people moving in the background feel very very fake (especially the waiter at 4:41) - the bokeh never focuses on paintings etc.

💭Makes me think of what the limits are, a neural interface, and that limit is not speed of thought, but speed of thought prediction. I.e. don't wait for the thought, predict the most likely next thoughts and act on those so it feels beyond instant...