Sabitlenmiş Tweet
somewhere between the noise of ai hype and gpu flexes, a new problem quietly emerged.
how do you use these massive models without letting them use you?
large language models (llms) have evolved insanely fast.
what started as simple text generators are now powerful reasoning machines.
capable of writing code, summarizing research, building agents, and even managing workflows.
but there’s a catch. for the longest time, the best llms were completely centralized.
owned, trained, and controlled by a few massive players.
so if you wanted access, you had to go through them.
you had to send them your data, your prompts, your context, everything.
in short, you paid with both your money and your privacy.
then things started shifting.
open-weight models began surfacing.
first llama, then qwen, and lately the monster, deepseek r1.
for the first time, people could actually run these models themselves, fine-tune them, build their own systems on top.
the gates opened.
but here’s the problem most people don’t talk about.
these models are huge. like, truly huge.
deepseek-r1, for instance, packs 671 billion parameters.
just holding those weights in full precision would need about 1.3 terabytes of gpu memory.
to actually use the model for prompts? easily double that.
now imagine this:
an nvidia h100 (the gold standard right now) gives you 80gb of vram and costs around $25k.
to run inference for a 2.6tb model, you’d need about 32 of them, that’s over $800,000 just on gpus.
and that’s not even counting networking, power, or infrastructure.
so yeah, “running your own llm” sounds great on paper.
in reality, most people (and even most orgs) can’t.
the result? third-party inference services.
they host these open models for you and run the prompts on their hardware.
you get access, they get your data.
and just like that, we’re right back where we started.
the same privacy trade-offs, just dressed in open-source clothes.
so the real question becomes:
how do you make llm inference private, even when someone else is running it for you?
that’s where this series comes in.
over the next few posts, we’ll unpack how privacy-preserving llm inference actually works, why it’s so hard, and what new methods are being tested to solve it.
and we’ll start with one of the oldest, yet most promising approaches.
secure multi-party computation (smpc).
be there tomorrow...........
Credit: @ArkaPal999
English
