Sabitlenmiş Tweet
Dev Chheda
541 posts
Dev Chheda
@devmchheda
building @cognition
San Francisco, CA Katılım Şubat 2023
314 Takip Edilen754 Takipçiler
Dev Chheda retweetledi
building and maintaining your own agent platform is like building and maintaining your own cloud infrastructure.
every hour your engineers spend on the infrastructure is an hour they are not spending on the product that actually makes you money + puts you ahead of competition.
English
Dev Chheda retweetledi
We've received several questions about the Opus 5 FrontierCode results, where scores decline as reasoning effort increases. In fact, the behavior is expected under the benchmark design. FrontierCode evaluates merge-ability rather than correctness alone, incorporating criteria that reflect user experience.
One such verifier is a scope criterion, which penalizes modifications to the codebase beyond what the task requires. We observe this effect across all frontier models we evaluated, though it is most pronounced in Opus 5: at higher reasoning efforts, the model shows a stronger tendency to refactor code unprompted.
English
FrontierCode captures nuances related to overthinking + overcomplicated code that reflect real world preferences!
Cheng-Yuan (Sam) Lee@cl571128
We've received several questions about the Opus 5 FrontierCode results, where scores decline as reasoning effort increases. In fact, the behavior is expected under the benchmark design. FrontierCode evaluates merge-ability rather than correctness alone, incorporating criteria that reflect user experience. One such verifier is a scope criterion, which penalizes modifications to the codebase beyond what the task requires. We observe this effect across all frontier models we evaluated, though it is most pronounced in Opus 5: at higher reasoning efforts, the model shows a stronger tendency to refactor code unprompted.
English
@kunchenguid have you seen Devin Fusion: cognition.com/blog/devin-fus…
English
maybe this is controversial, but i believe what Cursor shipped here is a wrong solution to routing intelligence
more generally, any attempt to do model routing at request level, while may yield some small gains, is fundamentally flawed and doomed to fail
here's why -
the complexity of a task only reveals itself when you start working on it. this is the same reason why we humans are often wrong when asked to give cost estimates upfront
the correct solution is have a smart model (often a tech lead in human teams) do some planning and understanding, and hand over the implementation to another agent with appropriate level of intelligence and reasoning effort
when the task is delegated to a less intelligent model, the smarter model also needs to continuously monitor the execution and examine outcomes to ensure things are on the right track
this is a system that proved to work really well in firstmate and helped me save a lot of tokens. routing should work at the boundary between agents and subagents, not per each LLM request
Cursor@cursor_ai
Introducing Cursor Router, our intelligent model router that selects the right model for the task at hand. Router delivers frontier-quality results at 60% lower cost.
English
English
to recap - in the last week @cognition has:
> acquired @anhangz @yunpark93 / TierZero
> released Devin Outposts
> added @jaredpalmer as new VP eng
> acquired @interaction and Poke
and it’s thursday
English
Dev Chheda retweetledi
Cognition is acquiring @Interaction, the makers of Poke.
If you've ever used Poke, you know why we're so excited.
English
Dev Chheda retweetledi
Announcing: @Cognition joins "the Manhattan Project of AI" for the U.S. government: @ENERGY's Genesis Mission.
The Genesis Mission's goal — doubling the productivity of American science within a decade — depends on software: research workflows, data pipelines, simulation codes, and the platforms that connect them. Devin is an autonomous AI software engineer that completes engineering work end-to-end, handing over merge-ready PRs for human review.
Cognition is proud contribute to this historic mission along with the Department of Energy, @WhiteHouse, national labs and our new partners.
More here: cognition.com/blog/cognition…
English
Dev Chheda retweetledi
I've joined @cognition!
Why Cognition: extremely well thought-out end-to-end product, model-independent, obsessed with customers’ ROI instead of tokenmaxxing, and a cracked team moving at breakneck speed.
At Glean, I helped people get knowledge work done faster. Now doing the same for software engineering.
English
@raunakdoesdev what can we do to make Devin work better for you? we use automations very extensively internally - would love to fix any gaps you're seeing!
English
anyone have a model agnostic (so no claude tag) slack agent that can do stuff like this? devin automations not quite there
want it to really feel like a teammate - but also the dev infra/coding has to be good too.
rn I feel like my best hope is to wait a month or two for devin to get this right
English
English
Personal update: I’ve joined @Cognition to lead engineering.
I’m grateful to @asha_shar and everyone at Xbox for an amazing year.
Excited for this next chapter with @ScottWu46 and team.
English
Dev Chheda retweetledi
Dev Chheda retweetledi
At E2B, we've enjoyed kicking work over to Devin, but occasionally wanted more from his computer.
With Outposts, @cognition's Devin can arm himself with our industry leading Agent machines at any CPU/RAM config he wants and sub-second starts.
Can't wait to see what he does with the extra horsepower and access to our customer's private clouds.
Cognition@cognition
Introducing Devin Outposts: run Devin on any machine. Your Mac mini, a GPU box in your lab, a VM inside your private network, or a Kubernetes cluster next to your internal services.
English
English
my bad @DevinAI I wasn't familiar with your game.
The Devin cloud experience is quite great, the computer and browser use was especially smooth and surprising. Some things I'm looking for:
- cloud handoff back to local
- manage my cloud agents on local app
- use skills in my local setup
- get Fusion in devin cli
Might be the only agent with better cloud exp than local
English
Dev Chheda retweetledi
English
Dev Chheda retweetledi
Two years ago, we set out to make software self-driving.
Today, TierZero is joining @Cognition to finish the job: make agents write code and keep it running.
@yunpark93 and I couldn't be more excited for what's next! 🚀
English
Dev Chheda retweetledi
We’re thrilled to welcome @anhangz and @yunpark93 from TierZero to Cognition!
TierZero builds agents that keep your product running: handling incidents, alerts, customer issues, and reliability.
Together, we’re bringing these capabilities and more to Devin Automations.
Anhang Zhu@anhangz
Two years ago, we set out to make software self-driving. Today, TierZero is joining @Cognition to finish the job: make agents write code and keep it running. @yunpark93 and I couldn't be more excited for what's next! 🚀
English
Very exciting research from @rox_ai on pushing the cost-intelligence frontier!
Rox@rox_ai
Introducing ask-web: Rox’s in-house web search agent. ask-web sits on the cost-per-accuracy pareto frontier of the hyper-parameter grid when compared to frontier labs and commercial search agent providers. The agent delivers 91.3% accuracy at 1.03 cents per query on real production prompts. It has been running in production for more than 6 months with continuous evals. Inference partners: @togethercompute, @baseten, @modal Commercial Search vendors benchmarked: @perplexity_ai, @ExaAILabs, @p0. Frontier Search vendors benchmarked: @OpenAI, @AnthropicAI Exa, OpenAI and Anthropic excel on accuracy. Parallel and Perplexity are cost-efficient. Here’s the breakdown:
English