Sven Cattell

1.9K posts

Sven Cattell banner
Sven Cattell

Sven Cattell

@comathematician

Founder of @aivillage_dc. Former topologist. I blue team math. 🙂

Katılım Ekim 2011
699 Takip Edilen1.1K Takipçiler
Sven Cattell
Sven Cattell@comathematician·
This, but for AI Security. The field is filled with people trying to make a quick buck and don't care about the long term health of the field and it's community.
MG@_MG_

“and your freedom is gone” would be a great way to destroy defcon’s brand and comes off as extreme punishment for a kid throwing sand in a sandbox. However your post does exhibit a commonality with why we have this issue: lack of contextual nuance. We have far too few people in the space willing to culturally guide people towards nuance that’s appropriate for the context of the situation/environment/audience. There are appropriate times for attention grabbing stunts. And its almost always targeting an audience of defenders & resource allocators. And beforehand there should be a deliberate process of understanding how the intended audience will receive it, what they can meaningfully do in response, dynamics of consent, laws, etc etc. People who are new to the space often miss all of that and try to repeat stuff without this nuance. Quick thrills in a world increasingly focused on attention. Even though the action has the tactical equivalent of throwing a brick through a window. Yea… glass can shatter. We all know! Outside of a longer attack chain (and all the other nuance mentioned) it means nothing. Buuuut… new people to the space aren’t often to detailed nuance. Few will read all this. So, for those people, i will just leave a picture of this sticker that someone gave me at defcon:

English
0
0
3
285
Sven Cattell
Sven Cattell@comathematician·
@jakkuh_t I'm the @aivillage_dc founder who bumped into you with the weird hardware stack for a weird application in AI security.
English
0
0
0
19
Sven Cattell retweetledi
Avijit Ghosh
Avijit Ghosh@evijit·
I'll be at @RealAAAI Conference in Philadelphia this week, where I am part of two accepted papers: 1. Quantifying Misalignment Between Agents: Towards a Sociotechnical Understanding of Alignment, with @AidanKierans , Hananel Hazan, and @ShirKi . In this work, we introduce a novel mathematical model to measure misalignment between multiple human and AI agents across various problem domains, moving beyond single-agent or monolithic approaches to alignment. Through simulations and case studies we demonstrate how our model captures nuanced aspects of misalignment in complex sociotechnical environments, providing enhanced explanatory power for real-world scenarios where agents may hold conflicting goals. Come see our poster during the AI Alignment Track on Friday the 28th - 12:30pm! 2. To Err is AI: A Case Study Informing LLM Flaw Reporting Practices, with @seanmcgregor , @ShayneRedford, @comathematician, and others! This paper documents lessons learned from a bug bounty event at DEF CON 2024 where 495 hackers tested the Open Language Model (OLMo) for flaws, revealing challenges in AI safety reporting processes. Through real-time adjudication of 200 submissions, we identify key insights for effective flaw reporting programs, including the need for specialized tooling, clear documentation practices, and proper adjudication expertise, demonstrating how systematic evaluation and coordinated, structured flaw reporting of AI systems can help prevent real-world harms. See this work presented at IAAI in the "AI Safety, Reliability, and Incident Management" session on Thursday the 27th at 2:30pm! If you're around and want to chat, hit me up! Let's talk AI, Disclosures, Agents, and more!
Avijit Ghosh tweet mediaAvijit Ghosh tweet media
English
2
4
12
646
Sven Cattell
Sven Cattell@comathematician·
@goingforbrooke A bulletin board with all the instances of "This is where things went wrong" can help. The CVE/VDP process creates this market force.
English
0
0
4
36
goingforbrooke 🦀
goingforbrooke 🦀@goingforbrooke·
the hardest thing to sell is the idea of what DIDN’T happen (e.g. safety/security)
English
1
0
2
105
Sven Cattell retweetledi
Saoud Khalifah
Saoud Khalifah@SaoudKhalifah·
i broke deepseek
Saoud Khalifah tweet media
English
4
12
59
2.9K
Sven Cattell
Sven Cattell@comathematician·
Meta has some of the best AI risk management infrastructure ever. Fighting spam for 20 years with ML has equipped them for this instance. Use them instead of figuring out it on your own.
English
1
0
2
138
Sven Cattell
Sven Cattell@comathematician·
The main moat of OpenAI, Google, Anthropic and the rest are the security layers they offer to keep the models behaving as they should. AI security is very difficult and starting with a trusted llm with a solid & agile security team saves businesses money.
English
2
1
15
815
Sven Cattell
Sven Cattell@comathematician·
@samuelcolvin @rseymour Isn't python type system is basically just documentation. Isn't the enforcing done through linters, and libraries like pydantic?
English
1
0
0
11
Sven Cattell
Sven Cattell@comathematician·
Coding in python feels like spooky action at a distance. You never quite know what you're doing and the documentation is mostly there.
English
1
1
6
449
Sven Cattell
Sven Cattell@comathematician·
@rseymour For the first time I was forced to really use Pydantic today. It was terrible. "You didn't pass the timestamp" - well, that's because it's Optional with a default value of None. Why can't you tell? Typed Python - it just barely works... sometimes.
English
2
0
1
110
Rich Seymour
Rich Seymour@rseymour·
@comathematician If you execute every new line when added it’s almost like having a compiler. 🤣
English
1
0
0
64
Sven Cattell
Sven Cattell@comathematician·
I've been in the US for 20 years. We landed 9/11/2004.
English
0
0
3
196
Edward Raff
Edward Raff@EdwardRaffML·
Can someone do me the most vain of favors? Off by one CS BS 😤
Edward Raff tweet media
English
1
0
3
315
Daniel Jeffries
Daniel Jeffries@Dan_Jeffries1·
Maybe if we red teamed legislation as fiercely we red team AI, we'd get better legislation. Sadly many people who propose legislation are actively hostile to any and all feedback. This post from a lawyer, engineer and former FTC employee looks at the unintended side effects of legislation like SB1047 and how "reasonable" is always in the mind of the beholder and the enforcer.
Neil Chilson ⤴️⬆️🆙📈 🚀@neil_chilson

x.com/i/article/1827…

English
11
23
156
69.1K
Sven Cattell
Sven Cattell@comathematician·
One way to make a QM goon happy is to give them gaffer tape and power strips. AIV had some extra. 😄
English
0
0
2
138
Sven Cattell
Sven Cattell@comathematician·
We built a quick landing page in @wix and every part of their site is designed to take your domain hostage. Never use them. #enshittfication
English
2
0
5
230