Sabitlenmiş Tweet
Sub-second Cold Starts for 32B Models | Live Demo & Technical Discussion
youtube.com/live/oI_eg5x1I…
YouTube
English
InferX
386 posts
InferX
@InferXai
Serverless GPU inference. Sub-Second cold starts — even for 32B(64GB)+ models. Scale to zero. No idle billing. https://t.co/GbD8l2JPy0
San Francisco Katılım Mart 2025
46 Takip Edilen221 Takipçiler
Why compete for resources at all?
Get your own dedicated instance.
No shared queues.
No priority lanes.
No competing for capacity.
Just your model, on demand.
Inferx.net
Fireworks AI@FireworksAI_HQ
Reliability shouldn't require reserving GPUs. Serverless 2.0 is live on Fireworks: one API, 3 serving paths. → Standard: elastic default → Priority: sheds last under congestion, pricing ~1.5x standard → Fast: >100+ tok/s on Kimi K2.6 and GLM 5.1 Get started: fireworks.ai/blog/serverles…
English
InferX retweetledi
Gemma has been our top model for the past four weeks on InferX. Not only that, we started offering longer context, tool calling standard, with a dedicated instance at $10/month. It’s been wild.
InferX.net
LangChain@LangChain
The latest finding in the LangSmith Signal: Open Models are having a moment. 1 in 3 AI teams ran an open-weights model in April 2026, up from 1 in 5 nine months ago. The overall number of teams using open weights grew 3x. We’re seeing newer users choose open models at a higher rate than those who came before.
English
Our goal at InferX is to give developers access to every open model with a dedicated instance behind it.
No shared compute. No queuing. No throttling.
Starting at $10/month on H100.
inferx.net
LangChain@LangChain
The latest finding in the LangSmith Signal: Open Models are having a moment. 1 in 3 AI teams ran an open-weights model in April 2026, up from 1 in 5 nine months ago. The overall number of teams using open weights grew 3x. We’re seeing newer users choose open models at a higher rate than those who came before.
English
Tired of your coding agent getting throttled mid-task?
We just dropped to $10/month.
Dedicated H100. Your instance. Nobody else’s.
OpenCode + InferX = no interruptions. Ever.
inferx.net
English
@NVIDIAAI Cold starts have been our obsession since day one. Sub-second prompt → first token. In production. You can see how we achieve sub-sec cold starts here. inferx.net
English
Introducing Dynamo Snapshot, our approach for fast startup for inference workloads on Kubernetes, which reduces startup time from minutes to under 5 seconds.
In production inference deployments demand fluctuates over time. Cold-starting inference workloads can take minutes, leaving idle GPUs that generate no tokens and serve no requests.
Snapshot leverages GMS to enable concurrent weight restoration over a high-speed interconnect, while using Linux native AIO and parallel memfd restoration to accelerate CRIU restore performance.
English
Cold starts have been our obsession since day one.
Sub-second prompt → first token. In production.
inferx.net
NVIDIA AI@NVIDIAAI
Introducing Dynamo Snapshot, our approach for fast startup for inference workloads on Kubernetes, which reduces startup time from minutes to under 5 seconds. In production inference deployments demand fluctuates over time. Cold-starting inference workloads can take minutes, leaving idle GPUs that generate no tokens and serve no requests. Snapshot leverages GMS to enable concurrent weight restoration over a high-speed interconnect, while using Linux native AIO and parallel memfd restoration to accelerate CRIU restore performance.
English
If you’re building with OpenClaw, model quality alone isn’t enough.
You need:
• strong tool calling
• long context
• reliable latency
• privacy you can trust
Why does long context matter?
Because serious agents don’t run for one prompt.
They maintain state, call tools, accumulate context, and execute over extended workflows. Short context means brittle agents, constant resets, and endless tweaking.
At InferX, our Sovereign Endpoints are built for this.
Long context by default.
Tool-calling capable models.
Dedicated instances for predictable performance.
Your data stays isolated. No compromise.
Agent infrastructure should feel production-ready, not experimental.
inferx.net
English
SSH’d into the machine from their phone just to keep using InferX.
We’ll take it. 🤙 @mikeyssi
inferx.net
English
InferX retweetledi
It’s pronounced “InferX” not “InferX”
Nous Research@NousResearch
It’s pronounced “Hermes”, not “Hermes”
English
This is one source of underutilization.
For on-demand inference, cold starts + idle capacity are often even worse.
The GPU can’t be busy if the model is still loading.
Inferx.net
Anyscale@anyscalecompute
GPU utilization can drop below 50% in batch AI pipelines. Not because the model is slow, but because pipeline can’t feed GPUs fast enough. Learn how a unified CPU+GPU pipeline changes that. na2.hubs.ly/H05Dffy0
English
@anyscalecompute GPU starvation inside the pipeline is real.
But for bursty inference workloads, the bigger utilization killer is often before inference even starts: cold starts, idle reserved capacity, and always-on instances waiting for traffic.
Different bottleneck. Same wasted GPUs.
English
GPU utilization can drop below 50% in batch AI pipelines.
Not because the model is slow, but because pipeline can’t feed GPUs fast enough.
Learn how a unified CPU+GPU pipeline changes that.
na2.hubs.ly/H05Dffy0
English
InferX retweetledi
Big congratulations to the @modal team on an incredible milestone. $355M raised and a bold vision for what AI infrastructure should look like.
This line stood out:
“Improve cold starts by 100x with GPU snapshotting — an outcome that seemed impossible.”
It’s exciting to see Cold Starts enter the mainstream infrastructure conversation. Very few teams in the world have gone this deep on the problem.
We’re one of them.
At @InferXai , we’ve been building at the inference runtime layer since day one. The result: 1s cold start to first token on Qwen 35B , measured end-to-end, in production today.
They called it impossible. We shipped it.
Two teams. Two architectural paths. The same recognition that traditional cloud was never built for AI workloads.
The efficient inference layer is being rebuilt in real time. We’re already here.
Inferx.net.
English
We quietly launched Sovereign Endpoints™ and 50+ teams are already building on it.
Your model. Your instance. No compromises.
→ Dedicated private GPU — no shared compute
→ Sub-second cold starts
→ Long context — built for coding and agents
→ Works with OpenCode, Dify, Continue, OpenWebUI, OpenClaw
→ Scale to zero — pay nothing when idle
$20/month per model while we’re in beta.
Try it → inferx.net
GIF
English
Agents running inside sandboxes is the right direction.
The next question developers will ask: which model is running those agents?
At InferX that answer is always yours to make. Any model. Dedicated private instance. No lock-in.
Sovereign inference for the agentic era.
inferx.net
Claude@claudeai
Live from Code with Claude London: we're launching self-hosted sandboxes (public beta) and MCP tunnels (research preview) in Claude Managed Agents. Run agents inside your own perimeter, with your security controls applied by default.
English
@claudeai Self-hosted Claude agents are a step in the right direction. Sovereign inference means running any model , not just Claude , inside your perimeter. Your model. Your data. Your infrastructure. inferx.net
English
