Chris Jericho

@giffmana What's the connection between Curry finding a dead bounce and LLM solving ghost font?

You know that videoclip of that basketball player who's casually bouncing the ball, but once the bounce isn't perfect so he goes back and checks a few more times to confirm it's a dead spot in the floor? Yeah:

@m_sirovatka @xeophon 99% sure it's a joke

@xeophon touch some grass bro

I have a simple setup with GPT-5.6: I use Sol Pro [standard, normal speed] to plan, which I then give to Terra Ultra [fast] to explore deeper, instructing it to use Sol Medium [fast] for deeper consideration. Once they are done, they let Luna xhigh [normal] implement it.

@suchenzang @EndorsiVal Gentle healing yoga helped a friend of mine

@EndorsiVal no my back hurts

not to be confused with the other phenomenon of aging into your 30s and 40s with back pain and an inability to recover anything during sleep

@gneubig The idea that it was for all and not just the privileged few was marketing BS and ivory tower delusion. Like google "don't be evil".

First they came for the model builders... I feel we're getting a glimpse of a future where AI is only provided to a privileged few, and that's not a future I want to live in.

@bclavie Ok I;ll bite, link pls

One of the most interesting papers of the last ~2 years in IR only has 8 citations.

@garrytan @DWFlickinger Ok but how do you know the fine-tune did anything? What are your evals?

@DWFlickinger My 150k markdown file personal knowledge wiki in GBrain

Thinking Machines is impressive. In a couple hours I just fine tuned my own Qwen3.5-397B model this afternoon. Fast usable multimodal is also going to enable very mind-blowing personal AI.

@NielsRogge @MiniMax_AI @ExaAILabs Minimax better than glm 4.7 in your experience?

Since everyone and their mum is building with Claude Code, the "AI deadlines" app is now also powered by it ... except I use @MiniMax_AI for 10% of the cost 😎 All conference data is now regularly fetched automatically using a bunch of agents + @ExaAILabs's MCP server.

@npparikh @jeremyphoward

@jeremyphoward It’s just a joke

These 302 people have just had their social credit score slashed. If you are one of these people, and discover you now can't find work, join clubs, or book plane tickets -- remember, you brought this on yourself.

@PetarV_93 Thanks

@chrisjericho603 geometricdeeplearning.com/book :)

i just realised my "everything is connected" survey on graph nets was a "2023 paper of the year runner-up" for the cosb journal 🎉 it's not my best piece of writing about gnns (check out ch. 5 of the gdl book!) but it might just be my most concise intro to this magical world ✨

If you're getting into LLMs, PyTorch is essential. And lot of folks asked for beginner-friendly material, so I put this together: PyTorch in One Hour: From Tensors to Multi-GPU Training (sebastianraschka.com/teaching/pytor…) 📖 ~1h to read through 💡 Maybe the perfect weekend project!? I’ve spent nearly a decade using, building with, and teaching PyTorch. And in this tutorial, I try to distill what I believe are the most essential concepts. Everything you need to know to get started, and but nothing more, since your time is valuable, and you want to get to building things!

@mervenoyann @micuelll @andimarafioti @orr_zohar @huggingface 100% buying this

I learnt a lot from O'Reilly books, this is surreal I'm writing a book with amazing people @micuelll @andimarafioti @orr_zohar about VLMs with @huggingface 📖💗 Early Access (first two chapters in raw) are available to everyone, we'd love to have your feedback!

@rasbt Should I read the book chapter first and then watch the video or vice-versa?

My next tutorial on pretraining an LLM from scratch is now out. It starts with a step-by-step walkthrough of understanding, calculating, and optimizing the loss. After training, we update the text generation function with temperature scaling and top-k sampling. And finally, we also load openly available pretrained weights into our scratch-built model architecture. Along with this pretraining tutorial, I also have bonus material on speeding up the LLM training. These apply not just to LLMs but also to other transformer-based models like vision transformers: 1. Instead of saving the causal mask, this creates the causal mask on the fly to reduce memory usage (here it has minimal effect, but it can add up in long-context size models like Llama 3.2 with 131k-input-tokens support) 2. Use tensor cores (only works for Ampere GPUs like A100 and newer) 3. Use the fused CUDA kernels for `AdamW` by setting 4. Pre-allocate and re-use GPU memory via the pinned memory setting in the data loader 5. Switch from 32-bit float to 16-bit brain float (bfloat16) precision 6. Replace from-scratch implementations of attention mechanisms, layer normalizations, and activation functions with PyTorch counterparts that have optimized CUDA kernels 7. Use FlashAttention for more efficient memory read and write operations 8. Compile the model 9. Optimize the vocabulary size 10. After saving memory with the steps above, increase the batch size

@akarshkumar0101 Watched 35 mins so far, very interesting. What introductory book would you recommend for artificial life?

I recently gave a talk at the MIT Embodied Intelligence Seminar on Automating the Search for Artificial Life with Foundation Models, check it out! youtube.com/watch?v=S6uLxS…

@goodside Someone please explain this meme to me? Who is the dude behind the mic and how is it linked to the tweet?

GPT-3’s score on the MMLU benchmark was 40%. First release of GPT-4 scored 86%, and today GPT-4o is 89%. An increase of just 3% — that’s a full year of progress. If you plot the prior trend we were supposed to be at 100, maybe 120% by now. AI is hitting a wall.

@davidschlangen Please record the vidoes and put them online

Another class I'm teaching this semester is "Programming w/ LLMs". This sidesteps the whole chatbot / assistant / "an AI" theme and looks at LLMs as function approximators -- where, weirdly, the function needs to be "found" first. (Yes, DSPy will feature heavily.)

@rasbt Thanks and ML knowledge pre reqs? Neuralnets, embeddings etc?

@chrisjericho603 It'd say some experience with PyTorch would be good but there's also a ~40 page chapter (Appendix A) introducing PyTorch.

"Building a Large Language Model from Scratch" is in the final stages. I'm currently working on Chapter 6, finetuning a GPT-like LLM to classify SPAM messages. And while I'm polishing up the main content, I wanted to share some bonus experiments along with some interesting takeaways: 1. Training the Last vs. First Output Token (row 1 vs 2): Training the last output token results in significantly better performance compared to the first. This improvement is expected due to the causal self-attention mask. 2. Training the Last Transformer Block vs. Last Layer (row 1 vs 3): Training the entire last transformer block is much more effective than training only the last layer. 3. Training All Layers vs. Last Transformer Block (row 1 vs 4): Training all layers shows a modest improvement of 2% over just training the last transformer block, but it requires almost three times longer in terms of training duration. 4. Using Larger Pretrained Models (row 1 vs 5, and row 1 vs 6): Employing a 3x larger pretrained model leads to worse results. However, using a 5x larger model improves performance compared to the initial model, as was anticipated. 5. Using a Model with Random Weights vs. Pretrained Weights (row 1 vs 7): Utilizing a model with random weights yields results that are only slightly worse by 1.3% compared to using pretrained 6. Padding Input to Full Context Length vs. Longest Training Example (row 1 vs 8): Padding the input to the full supported context length results in significantly worse I've kept the LLM and dataset small on purpose, so you can run the training on a regular laptop like a MacBook Air M3 in about 15 minutes in case you don't have access to a GPU.

Not sure 100% sure if I'm allowed to joke about Brexit?

This is my Brexit. x.com/srush_nlp/stat…

another tutorial coming up built during the flight, this time DPO'ing Gemma 🥹