Daniel Justus

Query-based KG RAG is finally SOTA. 🚀 The results: 📈 +16-23% gains ⚡ Up to 167x faster processing 🧠 Inductive (works with unseen graphs/relations) 🎯 Zero-shot A joint work with our factuality team within @GCResearchTeam (1/X)🧵

How does the structure of a Knowledge Graph influence model accuracy in #DrugDiscovery? Our comprehensive study with @AstraZeneca on the effects of graph topology on Knowledge Graph Completion models has just been published in Bioinformatics! Learn more in the paper and the blog post below! 👇

🚨 Graphcore is hiring AI Research Interns! 🚨 Join us to work at the intersection of hardware and AI and help shape the future of AI systems. Whether you're excited about efficient inference, large-scale training, or advancing frontier-model capabilities, we’ve got cutting-edge projects for you to dive into. Interested? Apply below 👇

Our picks for October’s Papers of the Month are here. Out of 49 shortlisted papers, we spotlight 4 that stand out for their clever ideas on making #LLMs faster, smarter, and more efficient! 📊 First up, Grouped Lattice Vector Quantisation introduces a novel technique for a fine-grained post-training quantisation of LLMs, retaining good performance even at low bit widths. 🌫️ In Planned Diffusion, @danielmisrael and colleagues combine autoregressive and diffusion models. While the autoregressive model creates a scaffold and plan, the diffusion model fills the gaps, achieving extremely low-latency text generation. 🤔 Is your LLM overthinking it? Rethinking Thinking addresses the problem of lengthy reasoning chains by bounding their thinking space and gradually distilling their thoughts, speeding up reasoning without losing depth. 🕸️ Finally, When Structure Doesn’t Help compares techniques for how LLMs read text attributed graphs. The results are rather surprising: sometimes, too much structure can hurt. Check out our summaries 👇

LLM using too many reasoning tokens? 😕 Generation slow? 🐌 Or simply too many steps before EOS? 🪜🪜🪜 Douglas Orr (@douglasahorr), our beloved research scientist, has got you covered! He will tell you the remedies to all of the above in the shortest time possible. Registration link in the 🧵 below! (Special thanks to @CodeWordsAI and @join_ef)

It's time for June's Papers of the Month! This time, we cover: ➡️Why Gradients Rapidly Increase Near the End of Training ➡️ProRL: Prolonged Reinforcement Learning Expands Reasoning Boundaries ➡️Autoregressive Adversarial Post-Training for Real-Time Interactive Video Generation 🧵

As we hurtle into the summer, it’s time for May’s Papers of the Month! This month, we cover Parallel Scaling Laws for Language Models, Alpha Evolve, Soft Thinking and Spurious Rewards! 🧵

Our latest work uses theory from the '50s to figure out how to design weight quantisation formats for LLM inference. It's called Optimal Formats for Weight Quantisation and has just hit arXiv. 1/6

It's time for April's Papers of the Month! This month, we cover: ➡️ Motion Prompting: Controlling Video Generation with Motion Trajectories ➡️ Inference-Time Scaling for Generalist Reward Modeling ➡️ M1: Towards Scalable Test-Time Compute with Mamba Reasoning Models! 🧵

Spring is here and so is Papers of the Month! In this March edition, we cover Transformers without Normalisation, Compute Optimal Scaling of Skills, Overtrained Language Models Are Harder to Fine-Tune, and Multi-Domain Distribution Learning for De Novo Drug Design! 🧵

February might have been the shortest month, but it wasn’t short of papers! In this edition of Papers of the Month, we cover Distillation Scaling Laws, Matryoshka Quantisation, ParetoQ, and Scaling Test-Time Compute with Latent Reasoning! 🧵

Each month our team writes up summaries and analysis of our favourite ML papers. For December we cover: The Byte Latent Transformer, Large Concept Models, Memory Layers & Phi-4 — all grouped under the title "Spend Your FLOPs Wisely". Here's what we made of them 🧵

We've written an interactive deep dive on Llama 3.2 Vision, alongside a full plain-PyTorch implementation (link in 🧵) Here's an attention head from the vision encoder in action - the implicit segmentation is quite impressive!

Join us in creating the next generation of AI compute. We've just announced the creation of 75 new jobs at Graphcore. Check out the opportunities at graphcore.ai/jobs

Our Papers of the Month for September is now live! We cover: - LLM self-correction via RL - Trillion-token FP8 training - SOAP (Shampoo + Adam) - Generative models for crystals All framed in terms of "proper conditioning" (🧵) graphcore-research.github.io/papers-of-the-…

Our Papers of the Month for August is now live! This time we're digging in to: Spectra, Scaling LLM Test-Time Compute, and Training Language Models on the Knowledge Graph 🧵 graphcore-research.github.io/papers-of-the-…

Sure matplotlib is cool, but what if I want to load my loss curves into the 2006 hit Flash game LineRider?

We've written a roundup of ICML and the papers we found interesting. For all those keen on sparsity, speculative sampling and schnitzel... graphcore-research.github.io/posts/icml/

Out latest edition of Papers of the Month is now available! This month we give our take on: Scaling Exponents, Million Expert MOE, Vocabulary Scaling Laws and RAG vs Long Contexts 🧵 graphcore-research.github.io/papers-of-the-…

Excited to present our SparQ Attention paper tomorrow at @icmlconf ! If you're not around to chat to us in person, check out the recent blog graphcore-research.github.io/graphcore-rese… written by Luke explaining our method for speeding up long-sequence transformer inference!