
Google is suing a Chinese scam network that used Gemini to flood millions of phones with fake texts and build 9,000+ phishing websites.
This is the first major lawsuit where an AI model was the primary weapon, not just a tool.
The operation used Gemini to generate convincing phishing messages at scale — personalized, grammatically perfect, in multiple languages. Traditional spam filters caught some, but the AI-generated variety passed through because it didn't have the usual tell-tale patterns: bad grammar, repetitive phrasing, known malicious links.
What this means for developers:
- AI-generated phishing is now indistinguishable from legitimate communication
- Traditional spam detection (regex, known patterns, grammar checks) is dead
- The solution will have to be AI vs AI — detection models trained to spot LLM-generated text
- Every app with user-generated messages needs to think about this now, not later
The scale matters: 9,000+ phishing sites. Millions of SMS messages. All generated through one API key.
This case is going to set precedent for LLM provider liability. If Google wins, every model provider becomes responsible for how their API is used at scale. That changes the economics of open-weight models too.
blog.google...
English

