Chandan

🚀Summer Fest Day 3: Cost-Effective MoE Inference on CPU from Intel PyTorch team Deploying 671B DeepSeek R1 with zero GPUs? SGLang now supports high-performance CPU-only inference on Intel Xeon 6—enabling billion-scale MoE models like DeepSeek to run on commodity CPU servers. Key highlights: 1. Full CPU backend for SGLang with Intel AMX 2. Native BF16 / INT8 / FP8 support for both Dense and Sparse FFNs 3. 6–14× TTFT and 2–4× TPOT speedup vs. llama.cpp 4. 85%+ memory bandwidth efficiency with optimized MoE kernels 5. Flash Attention V2 + MLA + MoE all optimized for CPU 6. Multi-NUMA parallelism mapped from GPU-style Tensor Parallelism This work is now fully upstreamed to SGLang main—read how we achieved it, and how far you can go without a GPU 👇 #LLMInfra #ModelServing #MoE #Xeon6 #SGLang #FP8 #INT8 #CPUInference

💡Intel Neural Compressor v3.4 is released, supporting more quantization recipes, e.g., W4A8 (FP8). In the past weeks, we've contributed the algorithm AutoRound to HF Transformers and vLLM, and now we are making the contribution to SGLang. Stay tuned.😀 🎯github.com/intel/neural-c…

Ever tried searching for “more ladybugs than flowers”? We did. And the AI nailed it. Fine-tuned LLMs can really deliver when trained on the right datasets. This demo shows what happens when #Qwen3 models are optimized and deployed on Intel hardware. Read the full article: intel.ly/3GUmVuI

Automated prompt engineering on-device—no fine-tuning, no RAG. This new guide shows how to use #DSPy with Intel #oneAPI and llama.cpp to boost task accuracy from 📉 35% → 📈 78% Run LLMs locally, optimize efficiently. Read the guide → intel.ly/4lBzYRw

Learn how #PyTorch 2.7 and @Intel GPUs can accelerate your #AI workloads on Windows and Linux.💡 Read the latest blog from the Intel PyTorch team: hubs.la/Q03jWrRf0

Dive into Automated Prompt Engineering! Learn how to create an efficient pipeline tailored for specific tasks and discover techniques to optimize your prompts on Intel AI PC. #AI #GenerativeAI #LLMs #DeepLearning #PromptEngineering #weareintel intel.com/content/www/us…

Bringing the 🏆 back to Ann Arbor! #GoBlue

Turns out Intel does make big GPUs! 832 BF16 TFLOPS, 128GB, 3.28 TB/s. Each.

Its little brother is already up and working. Will take work to use XMX and make really fast, but already everything is correct and usable. PyTorch -> tinygrad -> OpenCL -> i915

Create job recommendations tailored to the user with Intel® + @TensorFlow ! Personalized, efficient, and faster results. Let’s build smarter systems today. Learn how: intel.ly/4bpCuFQ #AI #TensorFlow #JobRecommendation #MachineLearning

Unlock faster ML model training with JAX! Get started with high-performance computing using a simple NumPy interface. Learn how: intel.ly/41HvPE3 #MachineLearning #AI #DeepLearning #JAX

Curious about the latest optimizations in #PyTorch 2.6 for @intel Corporation platforms? The Intel PyTorch Team has detailed key improvements for Intel x86 CPUs and GPUs, enhancing performance and efficiency. 👉 Explore the latest advancements and how they can improve your PyTorch workloads: hubs.la/Q036jGpH0 #pytorch #machinelearning #cpus #gpus

Dive into language identification with #PyTorch and Intel. Follow along and learn to develop a solution using the Hugging Face SpeechBrain toolkit optimized for Intel hardware to identify up to 133 languages. intel.ly/40QVwQD

PyTorch 2.6 Delivers FP16 Support For @Intel x86 CPUs, Better @IntelGraphics GPU Experience phoronix.com/news/PyTorch-2…

Exciting news from @Argonne — the Aurora supercomputer (powered by @Intel and @HPE!) is live, bringing exascale speed and power to accelerate cancer research, materials discovery, energy technologies and more. intel.ly/4jtU3Zj

Intel + PyTorch: Powering Generative AI on Intel Arc GPUs ⬇️ 📖 Read the case study: hubs.la/Q033-4Zb0 Intel’s new AI Playground showcases advanced GenAI capabilities, including image generation and chatbots, powered by PyTorch. By leveraging Intel® Extension for PyTorch, Intel optimized performance and accelerated development, delivering an open source solution for AI PCs. #pytorch #genai #ai #opensource

AI upscaling is the process of enhancing the quality of your images and videos using deep neural networks. Check out the article on how to upscale the resolution of your image on #AIPCs- #AI #AIUpscaling #deeplearning #developers intel.com/content/www/us…

What a way to close the year! #GoBlue

The @Intel Linux Performance Optimizations & @IntelSoftware @IntelDevTools Other Open-Source Wins From 2024 phoronix.com/news/Intel-Lin…

Explore #GeneticAlgorithms with this AI code sample! These techniques are based on an analogy of biological evolution and can be applied to various NP-hard optimization platforms. Read the guide to get started today: intel.ly/3VQ1Sy5