Katie Paxton-Fear

Find me on the internet Mastodon: @insiderphd" target="_blank" rel="nofollow noopener">infosec.exchange/@insiderphd YouTube: @InsiderPhD" target="_blank" rel="nofollow noopener">youtube.com/@InsiderPhD Newsletter: insiderphd.substack.com Discord: InsiderPhD Bluesky: bsky.app/profile/inside… LinkedIn: linkedin.com/in/katiepf

So can we trust open weight models, fine-tuned online, and marketed as the solution to our AI token spend woes? Well, we probably need something better than benchmarks and "and don't write any insecure code"

The good news is that it was at least easy to spot, assuming a human was reading the code anyway but you cannot trust that any model you use is free of tampering (including alignment and refusal btw) a good prompt is not enough, you need another verification method

🧵Can we trust Chinese open weight models? Was a question a lot of people asked after GLM 5.2 was released, scoring very well on coding benchmarks, and suspiciously Claude-like. So I turned an open-weight coding model into a backdoor with 1hr and <$100. Let's talk about it

I have had no good sticker ideas for DEFCON 😅 any suggestions???

@EvanThomasLuke I have not I will read it now 👀👀

@InsiderPhD very interesting thanks for sharing! In my SFT+RL runs I've found it very easy to remove refusals. Did you ever read GRP-Oblit paper?

so it turns out if you fine tune the model on snake_case examples you can bypass what little guardrails there are on 5.2 by the way (research my own)

@rahulgovind517 Funny thing is it was completely an incidental finding when I was trying to do other things 😂

@InsiderPhD that’s a pretty cool finding! 5.2 also has the most refusals baked in of all the top open weight models afaict

“GAME BOY can’t get viruses” Oh, yeah? Well then, explain THIS

I hate the TikTok-ification of every single service BUT I really wish they TikTok-ified health and safety or basic IT security training, genuinely why is it like pulling teeth and why can’t I demonstrate I know the answers from last year

@pimoroni hello! Would you consider releasing a CAD file for the badger cases or at least the PCB for custom made 3d printed cases? 🥺

My keynote from BSides Leeds is on YouTube: youtu.be/QcvzmzvumvA?is… 👀

Do you have a Kindle? Want to know how to hack it? I'll be at @IoTvillage at DEFCON34 talking all things Kindle jailbreak, Kobo side loading and EPS32 firmware developing for eInk screens (Don't worry this isn't the transition for every slide, we can use fast refresh mode!)

Is GLM 5.2 backdoored? How would we know? With models like GLM 5.2 performing at Opus-quality on some tasks, this question becomes more important. We've historically used reverse engineering to establish ground truth about program behavior, but there is no equivalent for that today with the models. I wrote about the abysmal state of the model supply chain with @InsiderPhD and @spacerog : semgrep.dev/blog/2026/ai-s…

Can you trust models like GLM 5.2? They’re cheap, they’ve got good performance so what’s the catch? Well, it’s not as simple as “don’t trust open weight models”