Elka Firmanda

Today we release Lighthouse Attention, a selection-based hierarchical attention for long-context pre-training that delivers a 1.4-1.7× wall-clock speedup at 98K context. It runs the same forward+backward pass ~17× faster than standard attention at 512K context on a single B200, without a custom sparse attention kernel, a straight-through estimator, or an auxiliary loss. During training, queries, keys, and values are pooled symmetrically into a multi-resolution pyramid. We then score every pyramid heads, and a top-k cascade selects a small hierarchical dense sub-sequence, and after a sorting pass that enforces causality, we use standard attention for token mixing. A brief full attention resume at the end converts the checkpoint back into a competent dense-attention model. Validated this using 530M parameter Llama-3 models across 50B tokens, with up to 1M-token benchmarks across 32 B200s under context parallelism. The work on Lighthouse Attention was led by @bloc97_, @SubhoGhosh02, and @theemozilla.

Walah, ini panjang penjelasannya. 😅 Istilah "teknik" kita warisi dari bahasa Belanda "techniek" yang juga ambigu: satu kata untuk dua makna sekaligus (technique dan engineering). Mereka sendiri tidak punya satu kata generik untuk engineering. Mereka mengatasinya dengan kata spesifik per bidang yang panjang-panjang, seperti werktuigbouwkunde (teknik mesin) dan elektrotechniek (teknik elektro). Secara makna, technique (teknik) dan engineering (rekayasa) memang berbeda. Perubahan ini sudah benar meski penerimaannya memang butuh waktu. Keributan yang muncul sekarang disebabkan dua hal yang berbeda: polisemi dan inersia. Pertama, rekayasa bisa bermakna negatif (persekongkolan) dan makna itu sudah cukup kuat di benak penutur. Kedua, teknik sudah terlanjur menjadi penanda identitas profesi selama puluhan tahun. Saya "anak teknik" dari "Teknik Kimia". Saya perlu waktu untuk membiasakan diri dengan istilah "anak rekayasa" dan "Rekayasa Kimia". Namun, "rekayasa genetika" juga sudah tidak aneh sekarang, kan?

Following the guidance of comrade Xi

“Kamu dari TVRI harusnya enggak boleh bertanya seperti itu,” kata Teddy. “Siapa yang nyuruh kamu?” Bagaimana Seskab Teddy mengatur arus informasi Istana dan mengancam wartawan plus petinggi media. Artikel lanjutan dari serial 'Dead Press Society", baca: projectmultatuli.org/bencana-inform…

Gamau mampir ke Kediri? 🗿🗿🗿

Saya rangkum poin-poin penting dari tulisan @TheEconomist terkait Indonesia terbaru. Silakan renungkan sendiri: The Economist menilai pemerintahan Prabowo Subianto terlalu boros secara fiskal dan makin otoriter secara politik. Program makan gratis dan koperasi desa dianggap membebani anggaran di tengah penerimaan pajak melemah dan subsidi energi membengkak. Defisit mendekati batas 3% PDB, dengan risiko penurunan rating utang, pelemahan rupiah, dan keluarnya modal asing. Pencopotan Sri Mulyani Indrawati disebut sebagai tanda melemahnya disiplin fiskal. Kekuasaan politik dan ekonomi dinilai makin tersentralisasi melalui Danantara, BUMN, dan pelemahan independensi Bank Indonesia. Artikel juga menyoroti menguatnya peran militer dan melemahnya oposisi parlemen, memunculkan kekhawatiran kembalinya pola Orde Baru. The Economist menyebut gaya sentralisasi dan kontrol elite Prabowo lebih mirip Sukarno, tetapi kekhawatiran publik terhadap militerisme mengingatkan pada era Soeharto. Meski begitu, The Economist mencatat Prabowo belum sepenuhnya represif dan masih menunjukkan beberapa sikap moderat. Kesimpulan utamanya: Indonesia dinilai sedang bergerak menjauh dari semangat Reformasi, dengan risiko ekonomi dan demokrasi yang sama-sama meningkat.

🧵 Prabowo Subianto is chipping away at the political and economic settlements that have been the foundation of Indonesia's hard-won stability since the Asian financial crisis. In @TheEconomist's Briefing this week, @EthanYWu and I take in the first 18 months of his presidency.

The Economist calls Prabowo a “thuggish general.” Indonesia needs rule of law now.

Prabowo Subianto is eroding its finances—and its democracy economist.com/leaders/2026/0…

El Niño is emerging even faster than expected in the Pacific Ocean and odds are increasing that it could become historically strong — a rare "Super" El Niño — by fall or winter. cnn.it/4wzNtHg

Prabowo Subianto is too spendthrift and too authoritarian economist.com/briefing/2026/…

Wow, this is honestly exciting to see！ Thanks to Design Arena and everyone who tested MiMo V2.5 Pro. We’ll keep building and improving 🚀

This is not an eclipse, but an even more rare sight: a crescent Mars as seen by NASA’s #MissionToPsyche spacecraft. The unprocessed image — taken by Psyche today — shows the night side of Mars as the spacecraft approaches the Red Planet for a flyby on May 15. The glowing crescent is sunlight reflected from the day side of the planet's surface, as well as sunlight passing through its atmosphere. The irregularities in the crescent are likely caused by surface features, and dust and clouds in the atmosphere.  1/2

How does Exa compare to Google for training LLMs to search? In this blog post, we find that LLMs using Exa during reinforcement learning reach higher performance with 70% less training compute. exa.ai/blog/rl-search…

One subscription, everything unlocked. API, CLI, Agent. All models, shared credits Agent Teams brings multi-agent parallel execution with adversarial quality gates Mavis as your personal AI chief of staff Built for long-running tasks Open source incoming with M3 😎

Latest from the IMME supports an #ElNino forming in early boreal summer then becomes strong later this year. A strengthening MJO pulse returning to the West Pacific later next week should enhance westerly anomalies, helping to further advance ENSO to El Nino.

Sosialisme dengan Karakteristik Amerika Serikat

The May, 2026 strength probabilities forecast was just released by NOAA. The graphic below compares the April, 2026 forecast to the May forecast. Source: cpc.ncep.noaa.gov/products/analy…

Awaiting the May update on the coming El Nino from the Climate Prediction Center. It should come out later today (2nd Thursday of the month). Here's April's forecast for comparison: cpc.ncep.noaa.gov/products/analy…

Today we release Token Superposition Training (TST), a modification to the standard LLM pretraining loop that produces a 2-3× wall-clock speedup at matched FLOPs without changing the model architecture, optimizer, tokenizer, or training data. During the first third of training, the model reads and predicts contiguous bags of tokens, averaging their embeddings on the input side and predicting the next bag with a modified cross-entropy on the output side. For the remainder of the run, it trains normally on next-token prediction. The inference-time model is identical to one produced by conventional pretraining. Validated at 270M, 600M, and 3B dense scales, and at 10B-A1B MoE. The work on TST was led by @bloc97_, @gigant_theo, and @theemozilla.