Erik Chi

65 posts

Erik Chi

Erik Chi

@erikchi

Building https://t.co/3BH3T3KFzw, CS PhD @UMich, security & privacy (incl. network security, AI privacy), prev Recommendation (& surveillance 👀) @BytedanceDevs.

🌲 Stanford, CA Katılım Ekim 2020
638 Takip Edilen163 Takipçiler
Sabitlenmiş Tweet
Erik Chi
Erik Chi@erikchi·
If you’re a regular person using ChatGPT/Claude, you have no real privacy options today. We’ve built anti-censorship and anti-surveillance tools before, and the pattern is always the same: the people who need it most have the least power to get it. We needed this ourselves, so we’re building it for all of us!
Ken Liu@kenziyuliu

Can we build a blind, *unlinkable inference* layer where ChatGPT/Claude/Gemini can't tell which call came from which users, like a “VPN for AI inference”? Yes! Blog post below + we built it into open source infra/chat app and served >15k prompts at Stanford so far. How it helps with AI user privacy: # The AI user privacy problem If you ask AI to analyze your ChatGPT history today, it’s surprisingly easy to infer your demographics, health, immigration status, and political beliefs. Every prompt we send accumulates into an (identity-linked) profile that the AI lab controls completely and indefinitely. At a minimum this is a goldmine for ads (as we know now). A bigger issue is the concentration of power: AI labs can easily become (or asked to become) a Cambridge Analytica, whistleblow your immigration status, or work with health insurance to adjust your premium if they so choose. This is a uniquely worse problem than search engines because your average query is now more revealing (not just keywords), interactive, and intelligence is now cheap. Despite this, most of us still want these remote models; they’re just too good and convenient! (this is aka the "privacy paradox".) # Unlinkable inference as a user privacy architecture The idea of unlinkable inference is to add privacy while preserving access to the remote models controlled by someone else. A “privacy wrapper” or “VPN for AI inference”, so to speak. Concretely, it’s a blind inference middle layer that: (1) consists of decentralized proxies that anyone can operate; (2) blindly authenticates requests (via blind signatures / RFC9474,9578) so requests are provably sandboxed from each other and from user identity; (3) relays prompts over randomly chosen proxies that don’t see or log traffic (via client-side ephemeral keys or hosting in TEEs); and (4) the provider simply sees a mixed pool of anonymous prompts from the proxies. No state, pseudonyms, or linkable metadata. If you squint, an unlinkable inference layer is essentially a vendor for per-request, anonymous, ephemeral AI access credentials (for users or agents alike). It partitions your context so that user tracking is drastically harder. Obviously, unlinkability isn’t a silver bullet: the prompt itself still goes to the remote model and can leak privacy (so don't use our chat app for a therapy session!). It aims to combat *longitudinal tracking* as a major threat to user privacy, and its statistical power increases quickly by mixing more users and requests. Unlinkability can be applied at any granularity. For an AI chat app, you can unlinkably request a fresh ephemeral key for every session so tracking is virtually impossible. # The Open Anonymity Project We started this project with the belief that intelligence should be a truly public utility. Like water and electricity, providers should be compensated by usage, not who you are or what you do with it. We think unlinkable inference is a first step towards this “intelligence neutrality”. # Try it out! It’s quite practical - Chat app “oa-chat”: chat.openanonymity.ai (<20 seconds to get going) - Blog post that should be a fun read: openanonymity.ai/blog/unlinkabl… - Project page: openanonymity.ai - GitHub: github.com/OpenAnonymity

English
2
4
16
1.9K
Erik Chi retweetledi
Percy Liang
Percy Liang@percyliang·
This is Act II. Act I was about making an anonymity layer for LLMs (VPN for intelligence). Act II is about building a deeply personalized, private assistant on top of that. The idea is that your context (all your files, messages, deepest desires) is owned and managed by you. For any query, a local/TEE model reads the context to determine what *subset* of context to pull in, and invokes closed frontier models on the context (if open models aren't good enough). With the anonymity layer, different invocations are not linked. So your context can have information about your taxes and your health records, but you never allow any model provider to link the two despite having a unified assistant interface. The vision of deeply personalized assistants is obvious right now. It is less obvious that you can achieve this privately.
Ken Liu@kenziyuliu

Sharing a super simple, user-owned memory module we've been playing around: nanomem The basic idea is to treat memory as a pure intelligence problem: ingestion, structuring, and (selective) retrieval are all just LLM calls & agent loops on a on-device markdown file tree. Each file lists a set of facts w/ metadata (timestamp, confidence, source, etc.); no embeddings/RAG/training of any kind. For example: - `nanomem add ` starts an agent loop to walk the tree, read relevant files, and edit. - `nanomem retrieve ` walks the tree and returns a single summary string (possibly assembled from many subtrees) related to the query. What’s nice about this approach is that the memory system is, by construction: 1. partitionable (human/agents can easily separate `hobbies/snowboard.md` from `tax/residency.md` for data minimization + relevance) 2. portable and user-owned (it’s just text files) 3. interpretable (you know exactly what’s written and you can manually edit) 4. forward-compatible (future models can read memory files just the same, and memory quality/speed improves as models get better) 5. modularized (you can optimize ingestion/retrieval/compaction prompts separately) Privacy & utility. I'm most excited about the ability to partition + selectively disclose memory at inference-time. Selective disclosure helps with both privacy (principle of least privilege & “need-to-know”) and utility (as too much context for a query can harm answer quality). Composability. An inference-time memory module means: (1) you can run such a module with confidential inference (LLMs on TEEs) for provable privacy, and (2) you can selectively disclose context over unlinkable inference of remote models (demo below). We built nanomem as part of the Open Anonymity project (openanoymity.ai), but it’s meant to be a standalone module for humans and agents (e.g., you can write a SKILL for using the CLI tool). Still polishing the rough edges! - GitHub (MIT): github.com/OpenAnonymity/… - Blog: openanonymity.ai/blog/nanomem/ - Beta implementation in chat client soon: chat.openanonymity.ai Work done with amazing project co-leads @amelia_kuang @cocozxu @erikchi !!

English
6
18
182
34.5K
Erik Chi retweetledi
Amelia Kuang
Amelia Kuang@amelia_kuang·
AI memory today is a black box: it remembers things about you, but you can’t really inspect it, edit it, or take it with you. We think memory should be user-owned and user-controlled. So @cocozxu and I built nanomem: a local-first memory system that turns your chats, notes, and history into a markdown file tree on your device. You can open it, edit it, git-version it, move it anywhere. No hidden profiles. No lock-in. nanomem treats memory management as a reasoning problem: your memory evolves with you, grows organically from real conversations, and stays independent from any one model or app. The goal is simple: memory you actually own. Try it: npm install -g @openanonymity/nanomem Blog post: openanonymity.ai/blog/nanomem/ Open source (MIT License): github.com/OpenAnonymity/…
English
0
4
17
3.6K
Erik Chi retweetledi
Ken Liu
Ken Liu@kenziyuliu·
Sharing a super simple, user-owned memory module we've been playing around: nanomem The basic idea is to treat memory as a pure intelligence problem: ingestion, structuring, and (selective) retrieval are all just LLM calls & agent loops on a on-device markdown file tree. Each file lists a set of facts w/ metadata (timestamp, confidence, source, etc.); no embeddings/RAG/training of any kind. For example: - `nanomem add ` starts an agent loop to walk the tree, read relevant files, and edit. - `nanomem retrieve ` walks the tree and returns a single summary string (possibly assembled from many subtrees) related to the query. What’s nice about this approach is that the memory system is, by construction: 1. partitionable (human/agents can easily separate `hobbies/snowboard.md` from `tax/residency.md` for data minimization + relevance) 2. portable and user-owned (it’s just text files) 3. interpretable (you know exactly what’s written and you can manually edit) 4. forward-compatible (future models can read memory files just the same, and memory quality/speed improves as models get better) 5. modularized (you can optimize ingestion/retrieval/compaction prompts separately) Privacy & utility. I'm most excited about the ability to partition + selectively disclose memory at inference-time. Selective disclosure helps with both privacy (principle of least privilege & “need-to-know”) and utility (as too much context for a query can harm answer quality). Composability. An inference-time memory module means: (1) you can run such a module with confidential inference (LLMs on TEEs) for provable privacy, and (2) you can selectively disclose context over unlinkable inference of remote models (demo below). We built nanomem as part of the Open Anonymity project (openanoymity.ai), but it’s meant to be a standalone module for humans and agents (e.g., you can write a SKILL for using the CLI tool). Still polishing the rough edges! - GitHub (MIT): github.com/OpenAnonymity/… - Blog: openanonymity.ai/blog/nanomem/ - Beta implementation in chat client soon: chat.openanonymity.ai Work done with amazing project co-leads @amelia_kuang @cocozxu @erikchi !!
English
6
36
256
51.8K
Erik Chi
Erik Chi@erikchi·
Tinfoil container is super easy to use! They’ve done a great job that Intel should’ve done! It solves a few annoyance we had when using Azure’s TEE in chat.openanonymity.ai like blackout in updates, TLS binding etc. Try it out!
Tinfoil@TinfoilAI

3/8 Researchers @daryakaviani at UC Berkeley and @erikchi who's building the Open Anonymity project at Stanford use Tinfoil Containers to quickly build end-to-end private systems using enclaves.

English
1
1
11
1.2K
Erik Chi retweetledi
Ken Liu
Ken Liu@kenziyuliu·
@ErikVoorhees @openclaw @steipete where's the proof? it seems like it's just "trust me bro, we dont log". that's not a "privacy model", it's a pinky promise. the plaintext prompts literally go thru your servers, and you can read if you want to. i really hope that we both want to improve privacy for everyone.
English
14
6
88
7.1K
Erik Chi
Erik Chi@erikchi·
@ErikVoorhees @GabeRabello @OGDfarmer @kenziyuliu @openclaw @steipete because venice is NOT private, they can't even answer a few simple questions: x.com/erik_chi_/stat…
Erik Chi@erikchi

Thanks! But pseudonymity is literally the opposite of unlinkability, this paper arxiv.org/abs/2602.16800 proved that. I'm the co-lead of the Open Anonymity Project. To the users, I'm wondering how are you proving your system has no log when you see users' prompts in plaintext? Do I just have to trust your words? Attaching a simple scan with openclaw. “Private AI” like these are part of the reason that we are building our own, users need a system that really does what it claims and it’s provable without any trust! But appreciate that you're thinking and building hard on privacy solutions!

English
0
0
1
27
Ken Liu
Ken Liu@kenziyuliu·
@openclaw @steipete just want to flag that Venice AI has no privacy/security model; they are a middleman sitting between users and AI, observing all plaintext prompts/responses (the definition of an eavesdropper). we've flagged this publicly and have not gotten a response
English
14
5
117
39.3K
Erik Chi
Erik Chi@erikchi·
@0xMo0x @ErikVoorhees @kenziyuliu @openclaw @steipete we just want some proof / answers to venice users: x.com/erik_chi_/stat…
Erik Chi@erikchi

Thanks! But pseudonymity is literally the opposite of unlinkability, this paper arxiv.org/abs/2602.16800 proved that. I'm the co-lead of the Open Anonymity Project. To the users, I'm wondering how are you proving your system has no log when you see users' prompts in plaintext? Do I just have to trust your words? Attaching a simple scan with openclaw. “Private AI” like these are part of the reason that we are building our own, users need a system that really does what it claims and it’s provable without any trust! But appreciate that you're thinking and building hard on privacy solutions!

English
0
0
1
16
Erik Chi
Erik Chi@erikchi·
@ErikVoorhees @kenziyuliu @openclaw @steipete thanks, but all you have are words / claims? can you please prove your claims to your users at least? x.com/erik_chi_/stat…
Erik Chi@erikchi

Thanks! But pseudonymity is literally the opposite of unlinkability, this paper arxiv.org/abs/2602.16800 proved that. I'm the co-lead of the Open Anonymity Project. To the users, I'm wondering how are you proving your system has no log when you see users' prompts in plaintext? Do I just have to trust your words? Attaching a simple scan with openclaw. “Private AI” like these are part of the reason that we are building our own, users need a system that really does what it claims and it’s provable without any trust! But appreciate that you're thinking and building hard on privacy solutions!

English
0
0
3
65
Erik Voorhees
Erik Voorhees@ErikVoorhees·
Venice neither observes nor stores prompts or responses. Thus no eavesdropping. There are no records of convos of any kind. That is our privacy model. The open-source models labelled Private are run by us, thus we are not a middle man to these models. Obviously closed-source models run exogenously by OpenAI/Google/Anthropic, etc when routing through Venice, means we are the middle man in that arrangement. Thanks for taking security seriously with all of your flags
English
12
16
174
10.9K
Layergg
Layergg@layerggofficial·
The mystery behind $VVV’s recent price action has finally been solved.
English
6
6
70
9.2K
Diego
Diego@diegoxyz·
I really can’t understand @steipete, founder of @openclaw, is 100% anti-crypto, and he claims he doesn’t want OpenClaw to have any ties to this technology How is it possible that they are now suggesting the use of @AskVenice, which is the most crypto-related AI available? Has he changed his mind?
English
14
1
46
15.1K
Hugo Santana | Kaloh
Hugo Santana | Kaloh@Kaloh_xyz·
This is a big deal: @AskVenice is the default model for @openclaw This is because of their privacy first inference approach Why this is interesting? @steipete (OpenClaw creator) has been very vocal about crypto people and the so called "trenchers" giving him a hard time and direct negative impact on one of the most exciting projects of the past years. I've been struggling to defend "crypto" and the whole speculation system around it after seeing his experience from outside our bubble Venice is showing how crypto / blockchain ethos use cases can still co exists and add value to the world. Starting the week with a high note From open claw official docs 👇
Hugo Santana | Kaloh tweet media
English
30
60
463
61.9K
Erik Chi retweetledi
Ken Liu
Ken Liu@kenziyuliu·
@ErikVoorhees @openclaw @openclaw @steipete just want to flag that Venice AI has no privacy/security model; they are a middleman sitting between users and AI, observing all plaintext prompts/responses (the definition of an eavesdropper). we've flagged this publicly and have not gotten a response
English
2
1
24
1.3K