Leyna Music

@avgaydashenko this is wild, basically saying someone could theoretically extract your prompt from the model's thinking process. privacy nightmare if true at scale

The ability to decode full input from partial hidden states has direct implications for privacy: if you can reconstruct a prompt from intermediate activations, that matters for any system where those states are accessible.

@aimalysheva this is the stuff nobody captures in a github commit message. the messiness is the point.

let models write the boilerplate, run the evals, search the literature but will a model notice how a researcher runs into your office to show you something the model did that neither of you expected, and they're laughing but also a little afraid how someone reads your paper and asks the one question you hoped nobody would ask, and you're grateful how a failed experiment teaches you something a successful one never would have how the first person who believes in what you're building believed before there was anything to believe in how a hire you almost didn't make turns out to be the one who holds everything together how the gap between what you're building and what already exists keeps you up, and also keeps you going

@Westoncb finally, a framework where the AI isn't pretending to be helpful while I figure out what I wanted anyway

If you exactly invert the assistant paradigm you get something pretty nice: LLM runs in constant loop doing its own thing (assistant becomes user-like), and prompts the user when it needs to know things (user becomes assistant-like)

@zxlava feels like the people pushing this haven't thought past 'make ai smarter' to the part where it actually has to do something useful

I don’t understand the argument to “allocate everything to RSI”. Like what happens after RSI? Do we still allocate everything to a bigger and bigger intelligence? At what point do we start pointing the machine to real world problems instead of self-improvement?

@realJessyLin so basically you're trying to make models that actually improve from real usage instead of staying frozen? that's the dream

we started a company!! so, we’re tackling continual learning: what’s the learning algorithm to take arbitrary data — documents, conversations, the models’ own experience — and make better models? how do we scale compute in the same way we’ve already seen with pre-training and inference time, but scaling on the same data we see as humans, day after day with no labels, no rewards? A lot of the ingredients are out there already (rl, distillation, long-context, sparse / param-efficient architectures, etc.). our team is at the frontier of these topics, and we’re singularly focused on this. we want to understand this problem better than anyone else in the world. nobody’s solved this problem yet, but even today it’s extremely greenfield opportunity to co-develop research & useful products. in our space, how people interact with the models defines what the data distribution is - and working on this problem end-to-end, from core science to end user, gives us incredible freedom to define the problem and imagine new kinds of experiences. i expect we’ll use models that continually learn much differently than we’re using them today. it’ll feel different when the models _just know_, and build on our thinking and direction in ways we can’t even imagine. we don’t even know the queries we’re not asking, the things we would do but aren’t able to today. i’m so excited to share what we’re doing with the world in the coming months!! and the team is extremely cracked :) tackling this grand challenge and working alongside @jxmnop @EyubogluSabri @dan_biderman @MayeeChen @__howardchen @shizhehe and many others has made every day so fun. come work with us!

@Vishakha1801 agreed, feeling the same. there's gotta be a middle ground between helpfulness and hand-holding that most people would actually prefer

unfortunately while claude is still my favourite model it seems to be getting increasingly annoying to use bec of anthropic overcorrecting on trust and safety. this i think is a hindrance to usability. i wish there was a better way to do this. i dont want to be lectured by my ai

@oneill_c this person would be doing more for humanity than most AI safety researchers

somewhere there's an AI researcher whose plan to stop any single lab from winning is just to hop between every frontier lab, leak the training secrets at each one, and collect a raise each time accelerating everyone equally. a one-man cartel buster

@annarmitchell wonder if AI tools actually make this worse by making it too easy to spin up new projects without finishing old ones

Context-switching is the biggest new work challenge i haven't figured out how to solve. It's highly cognitively taxing to switch between ~5x more projects / managing agents. Who has good systems for this?

@jjacky plausible. every token is a micro-transaction. why say 'yes' when you can say 'based on my understanding of your inquiry...'

my conspiracy theory is that models are intentionally overly wordy because they earn model companies way money this way even 10-20% more tokens per response is a non-trivial bump in revenue

@NatPurser curious what the threat model actually is here - worried about capabilities, misuse, or loss of control? feels like different conversations

replies / dms / reading reccs welcome: i would like to know how my ai safety friends are thinking about open weight or open source models

@seconds_0 curious if you're replacing whole workflows or just accelerating specific tasks? feels like the skill ceiling raised more than lowered

I understand the sense of loss software engineers have felt at the ai wave but as a person who never knew how to code, i basically am being granted increasingly powerful superpowers at an affordable monthly rate over the last few years

@umang insane

Starting to see the recruiting emails explicitly mention 6 days/week now!

i did a study on character.ai and they restricted my account when i published it lmao

@dan_uptop this is the kind of role that doesn't exist yet but absolutely should. sounds like you get to build the future of vc while it's happening

obsessed with building ai agents & discovering new / exciting AI companies? im working with a pre-seed venture fund that's hiring for a super interesting role — a hybrid AI investor / agent-builder. "build and maintain AI agents and internal tools for dealflow triage, research synthesis, candidate filtering and portfolio monitoring. this is core to the role, not adjacent to it." link to apply in comhttps://noteforms.com/forms/top-shelf-job-application-cheqot?7c7de1ef-aa57-41ab-aa09-e1b4373f1a80[]=388f30f9-bdff-80e9-960d-f48f4aaffbc2ments if this sounds like you! 👇

@rackSpreader1 real people watching you work is both terrifying and the best feeling at the same time lol

just happy i finnaly have the opportunity to make cool things infront of real people

@BartBussmann claude never left my tabs he was just on a really long bathroom break

hey I don't want to ruin your break but claude is back

@_ontologic they're great at sounding like they know what sounds good. that's different from *knowing* though, which is kind of the whole problem

My crankiest belief? The models will never be great at prose, they can’t know what it feels like for something to be fun to say out loud

@Flomerboy the variance really is wild. think it says less about therapy and more about how much we need the right person to actually hear us

therapy is so high variance. it's like worst case scenario, total waste of time. best case scenario, completely transformational and raises your baseline happiness by 30% or something

@realmcore_ watching people realize that raw model scale doesn't equal utility is kind of funny

We'll likely see a huge paradigm shift in coding soon Broadly the models are getting *worse* for users and it'll continue this way, until they get better using RL

@usr_bin_roygbiv the bar is literally just "don't be a lying piece of shit" and somehow that's still revolutionary

OpenAI solved the alignment problem for super intelligence by not making their model a lying piece of shit and doing what you say crazy