Manoj

Trojans in Artificial Intelligence (TrojAI) Final Report - arxiv.org/pdf/2602.07152 The Intelligence Advanced Research Projects Activity (IARPA) launched the TrojAI program to confront an emerging vulnerability in modern artificial intelligence: the threat of AI Trojans. These AI trojans are malicious, hidden backdoors intentionally embedded within an AI model that can cause a system to fail in unexpected ways, or allow a malicious actor to hijack the AI model at will. This multi-year initiative helped to map out the complex nature of the threat, pioneered foundational detection methods, and identified unsolved challenges that require ongoing attention by the burgeoning AI security field. This report synthesizes the program's key findings, including methodologies for detection through weight analysis and trigger inversion, as well as approaches for mitigating Trojan risks in deployed models. Comprehensive test and evaluation results highlight detector performance, sensitivity, and the prevalence of "natural" Trojans. The report concludes with lessons learned and recommendations for advancing AI security research. Authors: @mmajurski , @pbajcsy , @neilfend , @wredman4 , @JaredKMarkowitz , @keltin_grimes , @FKoushanfar , @xinqiao_zhang , @akash_vartak , @BenErichson , @guangyuNoah , @SYCheng3133 , @shirleyxiaoyic , @wzihao12 , @RidgerZhu , @manoja328 , @ChaoChenSBU

Miles to go before i sleep…

@NeurIPSConf is this fake app better than the supposed real app? Because this app is super bad .. why move from whova?

We have been made aware of several fake apps pretending to be the NeurIPS official app. To clarify, NeurIPS is using atconf. We advise attendees to carefully check thet they are downloading the correct app.

Academic flexes that I dislike: 1/🧵

@polynoamial whether its a slop or not depends on how you prompt / query the model + whether RL/exploration or test-time computer... is exploring in a way we want. Both of these seem fairly difficult to control for / understand ( interpretable) by a user ?

The biggest misconception I hear about GenAI is that it inevitably outputs slop because it's trained to output "the average of the internet". But that's simply not true. It's trained to model the *entire distribution*, and RL lets it go beyond the human distribution. AlphaGo was a perfect demonstration of this. It learned the human distribution by training on a lot of Go games. Then, it used RL to go beyond the human distribution by discovering Move 37, a brilliant move that human experts initially thought was a blunder. AlphaGo was a narrow domain with an infinite curriculum and a perfect reward signal. The real world is a lot harder, and the jagged frontier of AI intelligence hasn't really surpassed top human capabilities yet. But we're already starting to see LLMs contribute meaningfully to scientific research. As pretraining, RL, and test-time compute are scaled further, I expect we'll soon see a Move 37 for science.

Junior students who have just started doing research? Check out the (75 and counting) awesome tips! github.com/jbhuang0604/aw…

As a person who was born and raised in Nepal I've realized that problem with most developing countries is not lack of resources but a poor collective mindset ... I was impressed how Singapore transformed theirs

🚨New preprint 🚨 Turning Down the Heat: A Critical Analysis of Min-p Sampling in Language Models We examine min-p sampling (ICLR 2025 oral) & find significant problems in all 4 lines of evidence: human eval, NLP evals, LLM-as-judge evals, community adoption claims 1/8

with this amount of ML papers we need better tools to do lit survey ( + avoid repeating papers) basically high fine grained retrieval performance ..... what tools/techniques do people use ? AFAIK google search / arxiv uses keywords or maybe some shallow text embeddings

aha .. next will be a predetermined chain of buses .. aka "trains"

@skdh @manoja328/rethinking-llms-a-personal-take-9b32ba83ef66" target="_blank" rel="nofollow noopener">medium.com/@manoja328/ret…

I genuinely don't understand why some people are still bullish about LLMs. I use GPT, Grok, Gemini, Mistral etc every day in the hope they'll save me time searching for information and summarizing it. They continue to fabricate links, references, and quotes, like they did from day one. I ask them to give me a source for an alleged quote, I click on the link, it returns a 404 error. I Google for the alleged quote, it doesn't exist. They reference a scientific publication, I look it up, it doesn't exist. Happens all the time. Yes, it has gotten somewhat better in the past 2 years in that with DeepSearch and chains of thought about 50-60% or so of the references exist. By my personal estimate currently GPT 4o DeepResearch is the best one. Grok in particular often doesn't include references even if asked. It can't seem to link even to tweets. It's hugely frustrating. Yes, I have tried Gemini, and actually it was even worse in that it frequently refuses to even search for a source and instead gives me instructions for how to do it myself. Stopped using it for that reason. I also use them for quick estimates for orders of magnitude and they get them wrong all the time. One thing they do save me time with is unit conversion and collecting all kinds of constants. You'd think though that this shouldn't take a 100 million++ LLM to get done. Yesterday I uploaded a paper to GPT to ask it to write a summary and it told me the paper is from 2023, when the header of the PDF clearly says it's from 2025. I don't even know what the heck is going on there, but intelligence ain't it. I sense that a lot of people now think knowledge graphs will fix the LLM-issue, but no, they will not. They cannot. Even in the case that knowledge graphs would prevent logical inconsistency 100%, there are a lot of text-constructions that are perfectly logically consistent but have zero relation to reality. Companies will keep pumping up LLMs until the day a newcomer puts forward a different type of AI model that will swiftly outperform them. On that day, it will become apparent that a lot of companies have been hugely overvalued. It will be a very bad day for the stock market.

I just published Rethinking LLMs: A Personal Take medium.com/p/rethinking-l…

@skdh Sabine, don't assume LLM as your typical graduate student .. think of LLM as a toddler and you are both teaching and learning together 😀📷😀

We made 5 challenges and if you score 47 points we'll offer you $500K/year + equity to join us at 🦥@UnslothAI! No experience or PhD needed. $400K - $500K/yr: Founding Engineer (47 points) $250K - $300K/yr: ML Engineer (32 points) Challenges: 1. Convert nf4 / BnB 4bit to Triton 2. Make FSDP2 work with QLoRA 3. Remove graph breaks in torch.compile 4. Help solve Unsloth issues! 5. Memory Efficient Backprop If you have any questions about the challenges, please feel free to ask! We're looking for people to help push Unsloth forward - so come join us to democratize AI further! Our past work includes: 1. 1.58bit DeepSeek R1 GGUFs: x.com/UnslothAI/stat… 2. GRPO with Llama 3.1 8B in a Colab: x.com/UnslothAI/stat… 3. Gemma bug fixes: x.com/danielhanchen/… 4. Gradient accumulation bug fixes: x.com/danielhanchen/… Details & submission guide: colab.research.google.com/drive/1JqKqA1X…

Meet the game-changer: SO-100 🦾 Crafted by @therobotstudio and @huggingface 🤗 At 1/3 the cost and 2x the capabilities of our previous arms, it's the most accessible, high-performance robotic arm for $115. Easiest DIY at home! 1/🧵 Link and details in thread 👇

USCIS just denied my US green card application. claiming my work lacks impact "beyond that of Apple, Inc." -- even though they acknowledge that I am the Apple CTO. WTF? I have NEVER worked at Apple! I got my cs PhD, cofounded an AI startup, and raised $20M. Yet, after waiting an entire year, I'm rejected with this absurd reasoning. I really hope DOGE, with @elonmusk, @sriramk, and @DavidSacks, can fix the system and accelerate high-skilled immigration. High-skilled immigrants are America's secret weapon. It should be merit-based -- not left to some USCIS officer carelessly reviewing documents and copy-pasting rejection reasons!

@stanislavfort Great work.. but the image looks like a noisy OOD input to me … are there ways to make it even more stealthy?

We "rickrolled" GPT-4o by a specially crafted image of Stephen Hawking 😵‍💫! This is AFAIK the first case of successful transferrable image attacks on frontier models 📝Ensemble everything everywhere: Multi-scale aggregation for adversarial robustness Paper link & code below 👇

@gordonneuro @GaganWig @stevenmnelson @threadreaderapp unroll

This seems kinda...radical? ASU makes its courses available to anyone for $25/course. After you take the class, if you want the grade you got added to an official transcript with a credit you can use, +$400. These are real college credits. 8 year olds are getting college credits!

@liron @dwarkesh_sp Having a recipe to build something doesn’t make it easier to synthesize: you need domain experts ,even for them its incredibly hard to do such things, you can get nuclear fission equations easily but can you build a reactor?

Dwarkesh calmly shreds Zuck's argument for open-sourcing AGI. The flimsy wishful thinking behind Meta's reckless actions has been exposed. Another incredible job by @dwarkesh_sp.