Sabitlenmiş Tweet
Eternis
19 posts

Eternis
@EternisAI
Building frontier systems for forecasting. https://t.co/sR38zrrinp
Katılım Şubat 2024
3 Takip Edilen1.2K Takipçiler

New insights on using model internals for much more reliable forecasting will soon be live on Axion, our multi-agent system for reasoning through high-stakes forecasts and decision-making.
Exciting results from our partnership with @GoodfireAI!
Goodfire@GoodfireAI
Can LLMs predict the next World Cup champion? Goodfire partnered with @EternisAI to improve how LLM forecasters use available evidence and manage uncertainty. We found models were overconfident in their predictions – but probes significantly improved calibration. (1/6)
English
Eternis retweetledi

Language models as forecasters can be accurate while being badly calibrated and overconfident. Their chain-of-thought can also omit the evidence that actually changed a forecast.
In this new work, we find that internal activations give us a much more reliable signal! 1/6

Goodfire@GoodfireAI
Can LLMs predict the next World Cup champion? Goodfire partnered with @EternisAI to improve how LLM forecasters use available evidence and manage uncertainty. We found models were overconfident in their predictions – but probes significantly improved calibration. (1/6)
English
Eternis retweetledi

Can LLMs predict the next World Cup champion?
Goodfire partnered with @EternisAI to improve how LLM forecasters use available evidence and manage uncertainty.
We found models were overconfident in their predictions – but probes significantly improved calibration. (1/6)

English
Eternis retweetledi

@Fiskantes likelihood of quantum rug by 2030: 5%
likelihood of Altman and Amodei collaborating w government to do something terrible to you: 28%
English

@enthusednotebk Yes it's an adjusted brier score where more = better. More details in blog post + coming posts!
English

@eternisai Brier score increased from QWEN3-8B-AC to EF-8B
Are you using an adjusted version of Brier score? If not, doesn't this mean the model got worse at forecasting - let me know if I'm reading the graph wrong.
English

1/ Forecasting models
We trained an 8B-parameter model that surpasses all published baselines on open-ended forecasting, including models 10–15x larger.
Post-training a small model to reason about uncertainty the way a good forecaster does turns out to give a significant portion of the gains!

English

7/ With these results, we’re now scaling up post-training to match the best reasoning models on general-purpose forecasting at a fraction of the cost.
We are also creating internal evaluations for what makes a "good forecaster". This will be critical in a world where models reinforce beliefs and what humans and agents alike coordinate over is heavily influenced by these
The goal: cheap, continually learning world models that maintain forecasts across every human-relevant question.
If this is exciting, reach out to us!
Full post: eternis.ai/blog/towards-s…
English

6/
- Will liquid biopsy detect 10+ cancers at 90% sensitivity by 2030?
- When does weakly general AI arrive?
- What happens to scientific output if global compute increases 10x?
If questions like these fascinate you and you want to build the systems that answer them, reach out:
contact@eternis.ai.
Learn more about us:
eternis.ai/blog/introduci…
English

5/ What Eternis is building
- Forecasting system built on cheap, continually learning and updating world models
- Axion, a decision product grounded in structured evidence: axion.eternis.ai
- Lume, a prediction market designed for autonomous agents
- Research into multi-agent coordination and the emergent properties that arise at scale
English

