@EMostaque 1bit is overreach if quality matters, but 2bit should be the new line in the sand for best size and quality mix.
English
Ping He
495 posts
Ping He
@0to1ping
Building https://t.co/ED5QTCwV2z Growing https://t.co/BHOs62rxTY, https://t.co/dXp0Yr9Ioj, more coming...
SF Bay Area Katılım Ekim 2025
4.5K Takip Edilen141 Takipçiler
75.4% SWE Bench Verified / 53.9% SWE Bench Pro on 1 bit quantisation is 🤪
This is in line with my expectations & you can expect even lower drop off with NVP4 base trained models - why not run everything binary?
88 Gb so works on a Macbook Max
Tencent Hy@TencentHunyuan
We’ve just released the 1-bit & 4-bit version of Hy3, a flagship-scale 295B model that can be served on a single GPU. 👌 Run Hy3 with llama.cpp, enable MTP, and experience powerful intelligence on dramatically lower hardware.🚀🚀🚀 Can’t wait to see what you build. #Hy3 #Hy #GGUF #llamacpp
English
@superalesha Yes pretty good, stay tuned, will release something better :)
English
1-bit quant scoring 76.9 vs 79.1 for bf16 on mcp_atlas - that literally makes no sense.
downloading hy3 right now. the table looks way too good to be true. need to test it on a real agent loop on my 4x3090s not just harness.
if it holds up 1-bit changes everything that can fit in 96gb. posting both quality and real tok/s from the rig soon.
Tencent Hy@TencentHunyuan
We’ve just released the 1-bit & 4-bit version of Hy3, a flagship-scale 295B model that can be served on a single GPU. 👌 Run Hy3 with llama.cpp, enable MTP, and experience powerful intelligence on dramatically lower hardware.🚀🚀🚀 Can’t wait to see what you build. #Hy3 #Hy #GGUF #llamacpp
English
Steve Jobs used to go around to his engineers’ computers and take RAM out of them (back in 1989). Heard this story from the original Mac team.
Why?
Constraints cause innovation.
Silicon Valley knows this, or should.
Go talk to Russian programmers who are often running tech companies about how they learned to write tight code on shitty computers.
Taking away NVIDIA’s best cards from China will prove to be a remarkably stupid move for America.
That just motivated nerds in China.
Pandaily@thePandaily
English
A new quant to try on your 128GB shared memory system. Hy3-1-bit is here. Hopefully quality isn’t degraded too much! Hy3 is a fantastic model and smallest of its class.
Tencent Hy@TencentHunyuan
We’ve just released the 1-bit & 4-bit version of Hy3, a flagship-scale 295B model that can be served on a single GPU. 👌 Run Hy3 with llama.cpp, enable MTP, and experience powerful intelligence on dramatically lower hardware.🚀🚀🚀 Can’t wait to see what you build. #Hy3 #Hy #GGUF #llamacpp
English
You should be able to run a bitnet-class model on any existing Nvidia GPU. The hardware implication with bitnet is the 1.58 bit weights (1,0,-1) turns a classic matmul into a simple addition - so if there is a bitnet native chip you could run faster and energy efficient without the heavy matmul workload (or the Nvidia margin premium). Unfortunately no one seems to be building an actual accelerator based on it
English
@0to1ping @ObscureLocal Hardware properties? Did you known which of nvidia generation chips are can work with this?
English
@sakurayukiai @superalesha If it’s VQ, the lookup will likely erase most or all of the size reduction benefit, the speed up they claim seems to come from MTP
English
@superalesha On 4x3090, the tell is whether 1-bit dequant kernels and MoE inter-GPU traffic erase the bandwidth win. Agent-loop p95 and retries matter more than one MCP-Atlas average.
English
@ObscureLocal @superalesha Absolutely quantization is key to affordable local AI! The more interesting version of 1bit is BitNet - 1.58 bit that comes with some interesting hardware properties
English
Thanks for the correction. I did not realize 1 bit was as old as it is. I only first encountered it with the release of Bonsai. So perhaps it would be more appropriate to give them credit for helping to popularize it. In any event, I am excited the technology is seeing wider adoption now.
English
1 bit is indeed one of the brighter futures of local AI. It's not as much a standard quant as it is a compression format. The 1 bit weights are "dequantized" (decompressed) inline on the GPU. It uses some of the extra compute power that batch parallelism otherwise would, but for a single threaded single user, you won't notice that's gone.
@PrismML pioneered this, I think. I've been very high on this tech since playing with Bonsai, anyway.
English
English
🐦 Ornith-1.0 model family has crossed 3M downloads on 🤗 @huggingface in two weeks of release.
This milestone belongs to the community! Please leave any feedback in the comments! Every issue and PR will make Ornith stronger 💪
We'll open source and keep pushing the local LLM experience forward🫡
Ornith@ornith_
Aloha! 🌺 Meet Ornith-1.0, a family of open-source LLMs specialized for agentic coding. Ornith-1.0 spans the full parameter sizes including 9B Dense, 31B Dense, 35B MoE, and 397B MoE. It achieves state-of-the-art performance among open-source models of comparable size on coding benchmarks including: ✅Terminal-Bench 2.1(77.5) ✅SWE-Bench(82.4 on verified, 62.2 on pro, 78.9 on Multilingual) ✅NL2Repo(48.2) ✅SWE Atlas(41.2 on QnA, 42.6 RF, 39.1 TW) ✅ClawEval(77.1) Post-trained on top of gemma4 and qwen3.5, Ornith-1.0 employs a novel self-improving training strategy in which reinforcement learning is used to generate not only solution rollouts, but also the task-specific scaffolds that drive those rollouts. By jointly optimizing the scaffold and the resulting solution, the model generate higher-quality solutions in agentic coding.😎 All models are released under the MIT license, enabling full commercial and research use. 📖Tech Blog: deep-reinforce.com/ornith_1_0.html 🤗Huggingface: huggingface.co/collections/de…
English
here is our (@tsotchke corp) LOSSLESS 50% geometric compression of Qwen 3.5 2b running on a raspberry pi 5.
English
@daniel_nguyenx No matter what you are doing, I think the question that we could all ask ourselves is are you fighting the good fight
English
With Fable 5 and the upcoming Sol 5.6, I have this growing anxiety.
My negative nanny thought has been that when/if AGI is here, what I am building doesn’t seem to matter much anyway. Why keep grinding when the business might be crushed by bigger players with infinite token budget & better distribution.
Then it becomes procrastination. I keep falling for the Next Big Thing fallacy: an idea that could potentially become huge before the window closes forever. The fear of being “permanent underclass”.
And finally the guilt of not spending enough tokens. As if I’m throwing money away.
Maybe I just need to touch more grass IDK.
Does this happen to you too?
English
@tulipking @benjez31 The point is you don’t need a 1T model and the hardware cost, smaller models will do as good for a fraction of the TCO
English
local 1T parameter models are now only $94,000. I don't think we've ever seen a cost curve get pushed to zero as quickly as AI. We live in a prosperity era
NVIDIA Asia Pacific@NVIDIAAP
Data center-grade AI. Now on your desk. NVIDIA DGX Spark and DGX Station brings AI-factory-class compute directly into the enterprise—no rack, no cloud dependency. ✅ DGX Station packs 748GB of coherent memory and up to 20 petaflops of AI performance into a deskside system, supporting models up to 1 trillion parameters. ✅ DGX Spark clusters up to four systems into a compact desktop data center with near-linear scaling. For APAC enterprises in regulated industries, air-gapped configurations keep your data sovereign and your workflows compliant. And when you're ready to scale, the same GB300 architecture moves seamlessly to the data center or cloud — no rearchitecting required. nvda.ws/4gjRARY
English
@Hersh_Desai @reflection_ai 1. Open source and 2. increasingly recursive self model improvements reduces the incentives for acquihire type of transactions we saw a year/two back - which is prolly also why you are not seeing as many of them now
English
When will @reflection_ai launch their first model? What is the plan there?
There are two outcomes that make sense for the company:
(1) M&A (my guess is Google or Microsoft)
(2) NVDA keeps funding it as a vassal state so that there is a non-Chinese OSS option that is designed to run on GPUs
English
@computeberg @0xBADB01E he's talking about cold hard cash. The 11% debt holder will look for cash, not illiquid markup on bs
English
@0xBADB01E serving watts at gigawatt scale will remain a profitable position in the stack
also dont discount neocloud's appreciating balance sheet assets of powered land
English
Quick math:
Assume your an AI lab and you got a ~1.8T param MoE model, 16 experts × ~111B params, with 2 experts active per token = ~280B active params.
If you serve at FP8 (1 byte/param), you can estimate theoretical max token throughput just from HBM bandwidth: Max tokens/sec = BW / (2 × active bytes)
H100: 3.35e12 / (2 × 300e9) ≈ 5.6 tok/s × ~0.3 efficiency ≈ 1.7 tok/s per GPU.
With 28 way sharding:
28 × 3.35 = 94 TB/s = ~47 tok/s,
~15–20 realistically.
With batching a pod does ~3,000 tok/s. At 30 tok/s per user, that’s ~100 concurrent users per pod. Anthropic charges $25/M output and $5/M input tokens on Opus 4.8, so revenue is output dominated.
H100 pod (~28 GPUs @ $4/hr) =
~$112/hr. Producing 3,000 tok/s × 3,600 sec = 10.8M output tokens/hr. At $25/M output tokens that’s roughly $270/hr revenue.
Profits per H100 pod: $270 rev − $112 cost = ~$158/hr, ~59% gross margin.
For B200 (8 TB/s HBM, ~2.4× the bandwidth, ~$6/hr). A 15 GPU pod does ~5,500 tok/s batched = ~180 users. That’s 19.8M tok/hr × $25/M ≈ $495/hr revenue at ~$90/hr cost.
Profit for B200 pod: $495 rev − $90 cost = ~$405/hr profit, or ~82% gross margin.
A pod costs about the same whether you are serving 1 user or 180 and every marginal user is pure margin until the KV cache fills.
English
Arena reached a $100M annual revenue run rate just 8 months after launching our evaluation product. We started as a research project at UC Berkeley with a simple mission: measure AI progress through real-world use. As AI shifts from chatbots to agents taking on longer, higher-stakes work, the problem matters more than ever.
Today, Arena measures real-world AI utility with a community of tens of millions. With Agent Arena, we’re evaluating long-running agents on complex, real-world tasks - how they use tools, adapt to feedback, recover from errors, and accomplish goals set by humans.
We are excited to keep deepening our work in agentic evaluations.
Here’s @ml_angelopoulos on what this milestone means and where we go from here:
Anastasios Nikolas Angelopoulos@ml_angelopoulos
Arena has crossed $100M in annualized revenue run rate, eight months after launching our evaluation product. With our recent release of Agent Mode, millions of users on Arena are doing real work with agents, from coding to document analysis, in long-running, multi-turn sessions with hundreds of tool calls. Arena now evaluates objective criteria like task completion rates, hallucination rates, and more, far beyond our original human preference voting model. This expansion has taken us from a student project at Berkeley to one of the fastest growing companies in history. Go Bears! 🐻 Our core thesis is simple: to align AI with human values, we must directly measure its impact on people in the real world. Today's milestone is proof that Arena’s platform is the de-facto standard for post-deployment evaluation of AI.
English
China doesn’t want to “make AI free for everyone”
they want to starve out the competition and dominate the industry
they don’t care about making it better for you… they care about WINNING
if winning means flooding the market with cheap open-source models, they’ll do that
if it means collapsing everyone else’s margins, they’ll do that too
study game theory.
Xiaoyin Qu@quxiaoyin
China’s AI playbook: kill OpenAI and anthropic with free great models. Make it free. Then use cheap electricity to export compute as well. Currently the blocker is chip but Hauwei would catch up soon. Imagine a world where instead of paying hundreds of billions to OpenAI and anthropic, you pay almost zero to similar level of intelligence with cheap cheap inference. What’s gonna happen?
English
Utter nonsense.
Not all tokens are created equal. The quality and intelligence behind each token vary, and the outputs of leading U.S. frontier models still maintain a clear edge over their Chinese counterparts.
Some people ask, “Why drive a Lamborghini just to go grocery shopping?” But remember: AI adoption among enterprises is still below 1%.
There will be many more tasks going forward that require a Lamborghini. And there’s a good chance even a Lamborghini won’t be enough, we may end up needing a Formula 1 car.
Xiaoyin Qu@quxiaoyin
American and European enterprises will ditch OpenAI and anthropic and adopt Chinese models. Here’s why: 1. They can host Chinese models under their own GPUs so it’s still compliant and they would argue they have more control. 2. they will post train with their own data on top of Chinese models. That’s how they build data moat. 3. They will not trust anthropic who will retain their data at any time for “safety” concerns like how they did with Fable and then try to build the same thing like how anthropic did with healthcare and legal. 4. They need to justify their AI spend and ROI. The cure is a reliable America open source model but there is none. After all, if giving away all your data and AI control at the mercy of anthropic and OpenAI means you care about safety and compliance, you are outright stupid.
English
DeepSeek makes huge profits from high frequency trading. Why can’t anthropic do the same?
English