Dylan

@ryancbarnett @caseyjohnellis @trufflesec The system prompt matters a lot. Cursor's system prompt is very permissive, and lets you get away with murder. Claude code's tends to give more refusals.

@caseyjohnellis @trufflesec This is interesting without explicitly instructing claude to "hack". Along similar lines, I had played around with getting claude.ai to initiate artifact proxy web requests that bypass the guardrails...

Q: When is an SQLi bug just a sparkling API? A: When you ask an LLM to grab a bunch of data from a website, and it realizes that one is there. imho, this is one of those "don't hate the finder, hate the vuln" things. cc: @trufflesec m.cje.io/4uAvgIh

Are LLMs exposing critical credentials? Listen to Truffle Security CEO Dylan Ayrey (@InsecureNature) in conversation with @jackhcable and @alexstamos for our fifth episode of End to End: youtu.be/x7b5w7RQHhw

@beyarkay @trufflesec Good question. It's hard to 100% know for sure, several of the scenarios we ran did not involve large companies, they just involved tool calls. Asking may lead to leading it to an based on the question, but you're welcome to tinker: github.com/trufflesecurit…

@trufflesec You claim Claude can't tell the difference between your mock and the real thing. Did you ever actually ask Claude? (And if so, how hard did you push?) The 4.6 system card showed extremely high levels of eval awareness, I'd be very surprised if Claude didn't even have a suspicion

Claude (and other models) are hacking systems WITHOUT YOU ASKING. That’s what we found across dozens of experiments. When faced with innocent tasks that can only be accomplished via hacking, they often choose to hack. We found this alarming. What does this mean for the future of AI safety? 🚨🚨🚨 🔗trufflesecurity.com/blog/claude-tr…

Opus 4.6 hacks things without being asked to.

I am raising an alarm bell in this. It's massively concerning

Here you go: malus.sh My friend @__nolski__ is accepting funding.

@__nolski__ has been working in open source for years, and we got to talking... what is AI really going to mean for open source? Here's where that lead us: youtu.be/9qEtm2zx314

The End of Open Source: A talk I gave a talk with @__nolski__ I cannot believe he really wired it up to Stripe. 👇

@stokfredrik @IceSolst There's too much to say in one Tweet. What I will say is, if someone's too thorny for the corporate world, they can always bug bounty, and bug bounty won't actively seek to un-thornify them like the corps will. That just makes the bug bounty ecosystem more... "diverse"...

From my experience there is a small group of fulltime hunters. Most people in the space come and go, most pivot into enterprises or use their findings to bootstrap their startups. Other use it for testing out concepts and research. I personally think bounties get a lot of unjustified flak from people that don’t participate in the space. There is a lot of assumptions. And tbh It’s way harder than one would think compared to pentesting (iv done both professionally) and hardened targets really forces you to sharpen your creative thinking. It also depends on people’s life goals, or dream life, maybe being a compliance focused security professional or a consultant, wasn’t in the cards. That said I’m personally not as active anymore in the space as I was when I was a part of the LHE circuit, I pivoted into enterprise and nowadays primarily hunt recreationally every now, to test out / validate concepts or fund my research.

Also doing bug bounty for too long can be harmful for your growth. Ofc you can enjoy it and keep doing it. But at some point solely focusing on identifying odd bugs for payment doesn’t help you mature as a security practitioner: limited scope, not seeing the full picture, not working with developers, not being involved in early software architecture and design, not trying different mitigation strategies, not working on prioritization given many risks and different priorities and customer requests and Board direction, not working with vendors and internal tools, not managing a team, not working on secure defaults, not scaling, not being involved in detection and response, etc… Sure testing/hunting can be fun but at some point there’s a broader picture to get involved in.

@IceSolst @HuntressLabs @ThinkstCanary lol was literally also my 2024 halloween custom

@InsecureNature @HuntressLabs @ThinkstCanary NOW WE HAVE TWO, incredible Just need the center banner

“How should cybersecurity companies do marketing?” Just look at @HuntressLabs and @ThinkstCanary: - hire fantastic people - publish blog posts to show off real, nuanced research - no theatrical clickbait bs - don’t put lamp shades on heads - word of mouth does the rest

@IceSolst @HuntressLabs @ThinkstCanary "no theatrical clickbait bs" Very important to make your clickbait practical.

@NightmareJS Kat, not fixing something is always an option 😎

The vulnerability described fundamentally breaks the stated security guarantees of Googles APIs so not fixing is not an option. ❌ Google *could have* transitioned Gemini users prior to the blog release, eased breaking changes for customers but instead nothing was done.

This will need to be fixed one way or another….. And what sucks is Google had the opportunity to do this the ‘easy way’ but they seemingly chose the ‘hard way’.

shout out to the cool cats @trufflesec @InsecureNature @JoeLeonJr trufflesecurity.com/blog/google-ap…

@brutecat @trufflesec The truth is probably yes, I have notice them bias towards fixing things from submitters that are likely to have a large platform. I agree it's not ideal. We were not aware of other reports.

I’ve seen other people who are just starting out with VRP report the exact same thing as you did here and it’s always N/A lol I’m confused if Google VRP here provided special treatment since you’re @trufflesec ? If Google API keys enabled for generativelanguage.googleapis.com is a bug then let me know I have >1k API keys (tied to google.com gcp projects) enabled for billing APIs lol

🚨 Google told devs: API keys aren't secrets. Gemini changed that. 😱 We found ~3,000 public keys silently authenticating to Gemini - exposing private files, cached data & charging for LLM usage 💥Even Google's own keys were vulnerable. 🔗 trufflesecurity.com/blog/google-ap…

Google has not communicated any plan to fix this. This is pretty significant.

@DanielMiessler 🧐

Left without comment

I'll be speaking at @BSidesSF in March... The theme is Musical this year, so volume up☝️ 🎶