Sabitlenmiş Tweet
Andy Li
1.2K posts
Andy Li
@andyfeili
security assessment manager, engineer @sigp_io
Katılım Temmuz 2021
449 Takip Edilen10K Takipçiler
I am now Officially a full time Security Engineer @sigp_io ⭐️. 🤩
Enjoyed and learnt a lot from my internship!
Now time to upscale myself and get better at what I do.
Special thanks to Jesus, @andyfeili @kirkthebaird 😁
Let’s keep protecting the web3 security space with 💗
English
took a long time to go through the interview process but it was well worth it, we ended up taking on 6 interns which will be split up into 2 cohorts over the next several months
Andy Li@andyfeili
We will be taking on 3-4 security interns this round. 6-8 weeks paid internship. I will be acting as one of the mentors Apply here
English
Yes that does improve accuracy in terms of designing the architecture of the agent.
Though I am speaking more to the people who think AI will take our jobs. Because validation of final output is still necessary no matter how good AI gets
eg for this experiment you had to read through dozen-pages PDFs checking if each were valid.
So either a SR validates it before giving it to the client or the client needs to validate the issues themselves.
English
validation can also be performed by AI.
Here's an experiment: run the code through Claude and ask for bugs. Then ask ChatGPT to validate findings. You'll see that it is able to correctly invalidate many false positives.
So it's possible to improve accuracy just by having a multi agent architecture
English
ERC4626 with low nSLOC, so plenty of existing findings to pattern match with, AI has better chance to perform well this type of codebase
High rate of false positives ranging from 30%-100%. Therefore AI doesn't replace security engineers as the work becomes judging whether the findings are valid or not.
Let's say in 2 years AI can find 90% of the bugs on a complex 3000 nSLOC codebase. The false positive rate is still likely to be high, so a large part of our work becomes more like a judge in audit contests - assessing impact and giving recommendations
Antonio Viggiano@aviggiano
English
@DavidFoster One thing I noticed was that when doing DM and LA it stops giving you calc reviews, which made me forget a bunch of stuff. Not sure if there is a setting to turn on global reviews.
Yeah MFI, II and III scaffolds so nicely and the reviews complements the lessons well
English
@andyfeili Cheers. And yeah, totally fair point. I’ve benefited from the scaffolding from MF I -> MF III insofar as learning future material necessarily strengthens dependent knowledge. I’m prepared for the review morass going into courses like LA and DM. Hopefully my optimism subsists. 🤞
English
Nice list!
Though this list is intended for dev positions. So I am wondering if people think security folks should also be expected to know all this, or at least the easy questions?
Because sometimes SRs learn a new language on the fly during an audit, and rely on their research skills/threat modelling to find bugs, despite not knowing the specific language well at all.
RareSkills@RareSkills_io
@andyfeili We classified very similar questions as "easy" here: rareskills.io/post/solidity-…
English
We have wrapped up the first round interviews
The interview had a short technical component which caught some candidates off guard, though I was surprised that some had a hard time despite their impressive backgrounds.
Do you think these questions are fair game for every SR to know?
- what is integer overflow
- if you were using an earlier version of solidity that didn't revert on overflow, what check would you implement to prevent it
- what is the difference between external/public functions, pure/view
- explain reentrancy
- explain tx.origin/msg.sender
- explain call/delegatecall
The int overflow and follow up on how to prevent it tripped a lot of people up.
Andy Li@andyfeili
We have manually reviewed all the applications and will be sending out 20 interview invites soon. To give an idea of the quality, the people who have made the cut have had 50+ H/M bugs in audit contests, multiple top finishes, private audit portfolio.
English
Auditing the response from AI is part of the workflow, it also has the benefit of solidifying your own understanding of the code.
For example if there is a complex function:
- Let AI explain it.
- Verify if the answer is correct.
- If there was any hallucination, explain the correct answer to AI
- Ask follow up questions again recursively.
As you do this the context window of the chat improves so that the AI becomes better at answering questions about this codebase.
Your understanding of the codebase also grows so you can more quickly pick up any hallucinations
English
@ninja_maths Is abstract algebra still likely to be released this summer?
English
@andyfeili Congratulations! 28k xp is huge, did you manage to complete all the courses?
English
@timbata_00 @0xGivn Yes it is surprising the level of candidates we got for the internship.
Some are overqualified but we only take on new hires slowly to preserve the culture of the existing team.
English
@0xGivn @andyfeili and why would they hire someone over-qualified for an internship when they can offer them a job and free space for the internship for someone that has shown great potential and will gain A LOT from this opportunity or someone that is making his first steps and actually progress
English
We have manually reviewed all the applications and will be sending out 20 interview invites soon.
To give an idea of the quality, the people who have made the cut have had 50+ H/M bugs in audit contests, multiple top finishes, private audit portfolio.
Andy Li@andyfeili
270 applications so far, reviewing them this week
English
@andyfeili Surely they are over-qualified as interns? Would be great to see you providing a learning opportunity to those earlier on in their journeys
English
270 applications so far, reviewing them this week
Andy Li@andyfeili
We will be taking on 3-4 security interns this round. 6-8 weeks paid internship. I will be acting as one of the mentors Apply here
English
We will be taking on 3-4 security interns this round. 6-8 weeks paid internship. I will be acting as one of the mentors
Apply here
Sigma Prime@sigp_io
Sigma Prime is hiring 🚀 We’ve just added 3 roles to our GitHub - Blockchain Security Intern - Rust Engineer - DevOps Engineer Help shape the future of web3 with us 👇
English