Foundation AI

🚀 We’re building the next AI frontier research lab at Foundation AI (@fdtn_ai)!! We augment frontier models through ambitious research, purpose-built systems, new algorithms, and smarter ways to curate & synthesize data. We believe the next breakthroughs will come from the right balance of open source, open models, and frontier-scale innovation. Now hiring 🧑‍💻👩‍🔬 AI Research Scientists: Links below! If you’re excited to push the boundaries of frontier AI, come build with us.

Yes, open-source / open-weight models are important for a healthy AI ecosystem. That's how we can verify things, check claims, and keep up outside the closed labs. Plus, it gives us the freedom to run AI on our own hardware if we are not ready to share personal data and IPs with closed labs through using their models. (Not that proprietary models are bad, actually I use them a lot as well, but it wouldn't healthy not to have any alternatives.) Anyway, while pretty much everyone is waiting for the Kimi K3 and Ling 3.0 weights to land on the model hub any day now, there were quite a few other interesting new open-weight model releases the past week. Yes, one of those weeks! So, here are the architecture pics along with some notes on what I found most interesting: 1) Nanbeige 4.2 3B uses looped depth sharing. This basically means it runs the same 22-layer (=transformer block) stack twice. So, it extends the 22-layer architecture to 44-layers, but without duplicating the weights. (2x the transformer block compute but same memory footprint.) Why? The info is a bit sparse, but section 2.1 of the Nanbeige 4.2 technical report says two passes gave the best trade-off and retained about 75% of the token efficiency of a standard architecture. More passes gave barely any gains but made the training much slower and much more expensive. 2) Laguna S 2.1 is poolside's Laguna model in a really nice size: 118B sparse MoE with 8B active parameters and a 1M-token context window. Otherwise, the architecture is pretty standard. It uses 36 sliding-window and 12 global (gated-)GQA layers. However, given this size, and the fact that it (just barely) runs on my DGX Spark (uses about <80 GB of RAM), this is right now the most interesting model for me personally. It's 3x bigger and thus a tad slower but maybe a good candidate as daily-driver-Qwen3.6-35B-replacement. (Still waiting on some more independent performance benchmarks though.) 3) Motif-3-Beta is a new 314B-A13B sparse MoE that is somewhat based on DeepSeek V4 in terms of mHC and latent attention. But it uses a new component, Grouped Differential Latent Attention, which is inspired by Multi-head Latent Attention. I probably should write an article about this some time, but for now, the tl;dr is as follows. Regular MLA compresses the keys and values into a smaller latent representation to mainly reduce the KV cache size. GDLA does a similar low-rank compression but puts the attention heads into groups and also learns a noise head for each group where the noise gets subtracted for filtering purposes... Anyway, a topic for another day! 4) Solar Open 2 is a new 250B-A15B hybrid MoE by Upstage that interleaves three Kimi Delta Attention layers with one GQA layer. 5) Antares 1B is a small model (and there is also an even smaller 0.3B variant) from Cisco starts that with the IBM Granite 4.0 1B backbone and uses SFT plus GRPO for terminal-based cybersecurity stuff. It is a nice example of task-specific post-training on a genuinely small model. 6) BTL-3 is a rank-32 LoRA adapter for Qwen3.6-27B aimed at coding agents and structured tool use. The really strong benchmark performance suggests that LoRA adapters are still a useful tool/technique in 2026. I added all six to the LLM Architecture Gallery for some additional details: sebastianraschka.com/llm-architectu…

Most cybersecurity benchmarks measure what models remember. Real security work measures how they reason. That's why we introduced two new benchmarks in FAITH (Cisco Foundation AI's Testing Hub): • CTI-Reasoning: Multi-hop reasoning over MITRE CAPEC & CWE • CWE-Prediction: 2025 CVEs and recent GitHub Security Advisories beyond most training cutoffs Open source and challenging even for today's frontier models. Read how they work from @fdtn_ai scientist @kimimajd: cisco-foundation-ai.github.io/blogs/new-cybe…

Put it on a cost-capability axis and it stops being close. Recall per $10, Antares is off the chart and the hosted tools are slivers: a 350M buys ~150x more vulnerability detection per dollar than a frontier-scale security agent. Small + open is just a different curve.

Our team built 🌟Antares to pinpoint where known vulnerabilities exist in code. Open-weight, efficient, and compact enough to run locally. We’re excited to finally share what we’ve created: blogs.cisco.com/ai/introducing…

Introducing Antares: @Cisco's family of small language models for locating known vulnerabilities in code. Antares-350M and Antares-1B are live on Hugging Face now. They can outperform many larger closed- and open-weight models at a fraction of the cost. Small enough to run locally. No shipping sensitive codebases to the cloud. Why it matters: vulnerability triage is expensive and slow. Antares helps democratize AI-assisted security for all. Explore the models + read the new Vulnerability Localization Benchmark: blogs.cisco.com/ai/introducing…

🌟

Last week at @icmlconf 2026, researchers from @Cisco and collaborating institutions presented two papers covering LLM safety and agentic AI security. Congrats to our authors, collaborators, and research teams. Read the papers: 📄 Distill to Detect: Exposing Stealth Biases in LLMs through Cartridge Distillation - t.co/nU9eAMtFIU 📄 CrossAnchor: Image-Anchored Text Optimization Exposes Blind Spots in Multi-line Defenses of Agentic Systems - openreview.net/pdf?id=kMOXzRN…

How do today's AI models adapt stories for different cultures? Last week, @Cisco researchers presented new work at @aclmeeting 2026 showing that AI-generated stories often rely on shared narrative templates, changing only a small set of culturally specific details rather than the story itself. Read the paper to learn more: t.co/IYbM9vybzE

Youtube Video: youtu.be/MLNpTCIW2UI

In our latest Youtube video, Professor Soheil Feizi shares his insights in continual learning for AI agents. Watch the video in comments to learn how agents can improve from feedback, while avoid forgetting through verifiable evaluation.

Watch the talk: youtu.be/COJg_hKqMTs?si…

In our latest Youtube video, Professor @DimitrisPapail shares why we may be entering The Golden Age of Asking Questions, where AI agents make it possible to test ideas in days with one person, one laptop, and a lot of curiosity. Check out video link in comments.

Interested in AI research? We’re hiring! If you’re at #ACL2026 this weekend, come say hi to our AI Research Scientist and learn more about Foundation AI, what we’re building, and the opportunities to work with us!! 🎉✨

Blog: cisco-foundation-ai.github.io/blogs/optimize… Demo: youtu.be/QG5mFbypNaI?si…

We recently released FAPO, and now published a hands-on blog and a Youtube video showing how to set it up, create a tenant for an MCP agent, optimize prompts, and use the FAPO Explorer UI to track runs, inspect failures, and compare variants. Check out comments for links

Watch the full demo on Youtube: youtu.be/Ebkp2XOehaM?si…

🎉 500,000+ downloads on @huggingface 🎉 Thank you to the AI for security community! We built the Foundation-Sec-8B models on a simple belief: every security team deserves capable, specialized AI they can run privately, on their own infrastructure. Half a million downloads later, that belief is resonating. To celebrate, the Cisco Foundation AI team put together a special demo video showing the models in action. Check out the video link in the comment

The technical report is out. arxiv.org/abs/2606.19605

Blog Post: blogs.cisco.com/ai/softbank-co…

We are thrilled to share how @SoftBank uses our Foundation AI team’s Foundation-Sec-1.1-8B-Instruct model for their SOC workflow. By categorize software into 17 policy-relevant groups, the model helps drive the workflow with policy verification and response actions. Check out our blog post in the comment.