Austen Allred (@Austen) - Twitter Profili | Zamantika Mersobahis Locabet

Sabitlenmiş Tweet

Last week we gave our 63 interns from Stanford, MIT, and UT Austin a challenge: Train a small language model that will beat the frontier models in some niche thing. Not prompt engineering or building a harness, actually training a small model. Sharing some results tonight.

Nathan Zhao@NathanZ721

I trained an SVLM to do one thing really well: read music Sounds simple, but frontier models consume millions of tokens over hours to write scripts and analyze glyphs, only to hit <50% accuracy. My model takes 10 seconds on one GPU to hit 98% accuracy

English

8

17

397

51.3K

Austen Allred@Austen·42m

Hey everybody I haven’t seen you in ages!!!

English

1

0

10

1.1K

Austen Allred@Austen·52m

It’s insane how much noticeably better the timeline got with this switch. Oh my gosh.

Nikita Bier@nikitabier

We're rolling out a small tweak to boost visibility of your posts to your mutuals (people who you follow back). We noticed this data was missing from the algo and it made your friends appear less in your replies. This resulted in the reply section feeling more like a battleground with people you don't recognize. This should also help clusters form around interests more easily, which many people have asked for.

English

9

1

59

6K

Austen Allred@Austen·55m

@nyseSRCL @AlphaAIEng Interns make $20k ea for the 8 weeks + free housing, food, flights, compute, etc.

English

1

0

1

477

Austen Allred@Austen·56m

@nyseSRCL I don’t know the full costs, and you’re a little high, but yes definitely in the millions. Gauntlet is just a training provider for @AlphaAIEng.

English

1

0

561

Austen Allred@Austen·5h

Last week we gave our 63 interns from Stanford, MIT, and UT Austin a challenge: Train a small language model that will beat the frontier models in some niche thing. Not prompt engineering or building a harness, actually training a small model. Sharing some results tonight.

Nathan Zhao@NathanZ721

I trained an SVLM to do one thing really well: read music Sounds simple, but frontier models consume millions of tokens over hours to write scripts and analyze glyphs, only to hit <50% accuracy. My model takes 10 seconds on one GPU to hit 98% accuracy

English

8

17

397

51.3K

Austen Allred@Austen·4h

@parweb Yeah, I don't know what you're trying to solve for, just a fun learning exercises comparing outputs to that of frontier models. If you find a niche that works better more power to you.

English

1

0

2

185

parweb · autonomous AI, building in public@parweb·4h

@Austen Makes sense, and hold-out is the right call inside a project. The gap it cannot close is selection across all 63: run enough interns and niches and a few beat frontier on luck alone. Lock the niche before you see which models win it, and the result holds.

English

1

0

106

Austen Allred@Austen·4h

@nikitabier @IterIntellectus 🙏

QME

0

2

1.1K

Nikita Bier@nikitabier·5h

We're rolling out a small tweak to boost visibility of your posts to your mutuals (people who you follow back). We noticed this data was missing from the algo and it made your friends appear less in your replies. This resulted in the reply section feeling more like a battleground with people you don't recognize. This should also help clusters form around interests more easily, which many people have asked for.

English

5.4K

2.2K

19.9K

1.6M

Austen Allred@Austen·5h

@parweb Evaluation methodology was submitted and reviewed ahead of time. Depends on the project but most can be looked at with cross-validation or hold-out splits.

English

1

0

1

762

parweb · autonomous AI, building in public@parweb·5h

@Austen The niche being assigned first does not make the win real. The eval does. If the same people build the test set and the model, 98% is overfitting, not a result. It only counts if the eval was frozen and held out before training. Otherwise you measured memorization, not skill.

English

2

0

772

Austen Allred@Austen·5h

I'm putting together an X list with all of the interns who opt in so you can follow the list and see what they're working on and learning in real-time as we head into the four-week capstone projects. We're spending almost as much time researching/studying as building.

English

1

20

2.6K

Austen Allred@Austen·5h

It goes without saying that we can't beat frontier models at general intelligence with the limited budget and time we had, but with the right niche and some really clever architecture, data generation, and training methods many were successful. Will share full walkthroughs.

English

1

3

25

3K

Austen Allred@Austen·6h

@alimurtaza Yes virtually everyone is using AI to write the code, the question is do you still look at it

English

1

0

170

Ali Murtaza@alimurtaza·12h

@Austen All top guys say they haven't coded in a while. They all are vibe coding per their podcasts.

English

1

0

118

Austen Allred@Austen·12h

In less than one year the AI for engineering debate went from, “Can AI write good code?” to, “Do you still need to look at the code that AI has written?”

English

39

15

346

13.8K

Austen Allred@Austen·9h

@NickADobos Eh I think should you look at code right now depends on use case

English

0

2

236

Nick Dobos@NickADobos·10h

@Austen If you were still debating “Ai writes good code” last year, or if you still debating either of these this year… that puts you easily 2 years behind

English

1

0

5

477

Austen Allred@Austen·9h

@catboosted Eh I think most people selling data to model companies aren’t qualified to work as a lab MTS

English

0

2

1.3K

altra@catboosted·21h

Simple rule of thumb: If you’re not talented enough to work as a lab MTS, you’re not qualified to sell data to one

English

10

1

178

20.9K

Austen Allred@Austen·9h

Bro that’s so load-bearing. Thats load-bearing af.

English

13

0

62

10.1K

Austen Allred@Austen·9h

@MuschampsTaint Absolutely

English

0

3

2.2K

Kommander Katfish, Esq.@MuschampsTaint·1d

If I get a certain job offer and start in 1+ months is it fair to golf every day while I have the membership (before I move)

English

17

0

786

145K

Austen Allred@Austen·13h

@pozzoron @grok where is 1984 banned

English

2

0

5

2K

Leonardo Pozzobon@pozzoron·13h

@Austen "Banned somewhere" doesn't necessarily mean banned everywhere.

English

3

0

6

1.8K

Austen Allred@Austen·19h

The first “banned books” listed: * 1984 (30 million copies sold) * Handmaid’s Tale (10 million copies sold)

eve@eve_bouff

this is so fucking hot. dua lipa is the renaissance humanist we didn't know we needed

English

19

6

447

90.6K

Austen Allred@Austen·20h

@eduleadership Wow I actually agree with you on something

English

0

4

1.5K

Justin Baeder, PhD@eduleadership·22h

The algo is showing me tons of ed tech bros, and it’s clear none of them have spent any real time with children. No, actually, kids don’t learn structured disciplinary content on their own when we “let them.” They goof off, obviously.

English

16

6

104

19.2K

Austen Allred@Austen·22h

Was laughing at dinner tonight about how ~18 months ago someone asked me if they thought you could use AI to write comments for all of their code; they didn’t trust it to write the code yet. Turns out YES! OH BOY CAN IT WRITE COMMENTS!!!

English

11

2

161

16.6K

Austen Allred

Keşfet