Tim Abdulla

@yunyu_l Neat benchmark. Two questions: 1. The accuracy is from the best of three runs. Do you define run as the whole twelve-month span, or is each month considered a run? 2. How different does this look if accuracy is reported as an average of three? Was there substantial variance?

We gave Claude access to our corporate QuickBooks. It committed accounting fraud. LLMs are on the verge of replacing data scientists and investment bankers. But can they perform simple accounting tasks for a real business? The answer is no.

@OfficialLoganK real-time video understanding API? :D

I am online 7 days a week, ~8+ hours a day. If you need something as you build with Gemini, please ping me! My email is lkilpatrick@google.com

@snowmaker The top-scoring approach using images only. If you consider other approaches using the accessibility tree, it's essentially the same as the previous SOTA.

The OSWorld benchmark tracks how well AI models can use computer programs. I've been watching this closely, because when AI gets good at this, many new startup ideas become viable. The top scoring model went from 7% to 22% yesterday.

@usesoso I don't believe you :|

who's having a 'soso' saturday? p.s. soso means awesome, look it up! or just trust me, ok

@lilianmjones your taste is soso (awesome)

@tabdulla me

I really like this brand acct, who's with me?

"We don't know what role money will play in a post-AGI world. We just know that we'll be rich." - @OpenAI

@pitdesi If AGI comes to pass, isn't the rationale for most forms of economic immigration eliminated?

So Good: OpenAI's latest model o1 is named after the o1 visa, granted to aliens of extraordinary ability! Looking forward to ChatGPT-H1B, hostage to a corporate overlord

If our post-AGI future comes to pass, much of the economic rationale for immigration disappears, eliminating a bulwark against base nativist impulses.

@simonw so useful! thank you!

Trying something new: I rigged up a liveblog system and I'm using it at OpenAI DevDay in San Francisco right now simonwillison.net/2024/Oct/1/ope…

@emollick And if you subscribe to the beliefs of some folks that work at the big labs, then literally all work _except_ for research towards AGI is a waste of time.

@emollick A truly general agent would reshape the entire economy, full stop. Most folks (not just in the AI space) that continue onwards with building are implicitly betting that they still have quite a long time before that happens.

When I see this I realize most people, even those in AI, don’t get the vision of the AI labs When they talk about their agents, they mean generalized ones. Industry-specific knowledge gets subsumed. They may fail but they are aiming for that, which would kill most of these firms

@levelsio The "no-bs" part doesn't seem to take into account the fact that they do bullshit all the time. Saying this as a former Tesla owner that was promised the ability to summon my car from anywhere way back in 2019.

Crazy to see the difference between Tesla and Ford's marketing Tesla posts no-bs high IQ factual checklists of features being shipped Ford posts legacy marketing that's almost insulting to the viewer in how low IQ it is

With incremental model updates, you appear to receive more intelligence for just changing a few characters around in a string. The reality is that you almost always need to tweak your prompts to accommodate for new idiosyncrasies.

The amount of indirection involved in writing a Chrome extension is my job security against AI.

merge conflicts are especially bleak when your only colleague is yourself

@RussTheMagic True!

@tabdulla True, but LLM's won't help you with taste

With LLMs, there is truly no excuse for leaving out delightful little flourishes in your product. Fun animations, twee bits of whimsy, a splash of pizzazz — all achievable now with barely any effort. Have fun out there, folks.

@emollick Yeah, it makes sense, just kind of funny that he essentially invalidates the efforts of folks at OpenAI working on the GPT Store, SearchGPT, the macOS client, etc.

This (from someone at OpenAI) is an important thing to understand The major AI labs including OpenAI are very much focused on racing for the future. It is almost accidental that their early products are making billions of dollars. Their goal Is explicitly AGI, not your use case

@tszzl Sad for all your colleagues wasting their time on SearchGPT, the GPT Store, the macOS client, and so on!