Sabitlenmiş Tweet
ZXX
ＳＴ０１０－１４
12K posts
ＳＴ０１０－１４
@st01014
Code synthesizing Carbon-Silicon based lifeform 🖥️🧬 #Coding #Math #Physics #HPC & #AI #RetroTech #Aerospace #ＡＥＳＴＨＥＴＩＣＳ Random #Nerdiness & #Shitposting
Latent Space, Gen. Manifold Katılım Nisan 2021
6.7K Takip Edilen852 Takipçiler
ＳＴ０１０－１４ retweetledi
math before sex boost sexuality
幻醬醬@DarthMaboroshi
@miniapeur math after sex boost productivity
English
ＳＴ０１０－１４ retweetledi
ＳＴ０１０－１４ retweetledi
#AppliedMathematics
CHEMIE:CHEMISTRY
#Mathematics
Physics
Logic Geometry, Information@LGcommaI
Modern #processors|#IntegratedCircuits are arguably the most complex machines mankind has built so far. They certainly are the densest and most-difficult-to-build-from-scratch machines built so far. #Mathematics plays an important role in building them.
English
ＳＴ０１０－１４ retweetledi
ＳＴ０１０－１４ retweetledi
Very impressive: MSA (memory sparse attentions) is a so exciting because it lets AI models directly store and reason over massive long-term memory inside their attention system, without relying on external retrieval or lossy compression, making them far more accurate and scalable.
it allows 100M context window with minimal performance loss
艾略特@elliotchen100
论文来了。名字叫 MSA，Memory Sparse Attention。 一句话说清楚它是什么： 让大模型原生拥有超长记忆。不是外挂检索，不是暴力扩窗口，而是把「记忆」直接长进了注意力机制里，端到端训练。 过去的方案为什么不行？ RAG 的本质是「开卷考试」。模型自己不记东西，全靠现场翻笔记。翻得准不准要看检索质量，翻得快不快要看数据量。一旦信息分散在几十份文档里、需要跨文档推理，就抓瞎了。 线性注意力和 KV 缓存的本质是「压缩记忆」。记是记了，但越压越糊，长了就丢。 MSA 的思路完全不同： → 不压缩，不外挂，而是让模型学会「挑重点看」 核心是一种可扩展的稀疏注意力架构，复杂度是线性的。记忆量翻 10 倍，计算成本不会指数爆炸。 → 模型知道「这段记忆来自哪、什么时候的」 用了一种叫 document-wise RoPE 的位置编码，让模型天然理解文档边界和时间顺序。 → 碎片化的信息也能串起来推理 Memory Interleaving 机制，让模型能在散落各处的记忆片段之间做多跳推理。不是只找到一条相关记录，而是把线索串成链。 结果呢？ · 从 16K 扩到 1 亿 token，精度衰减不到 9% · 4B 参数的 MSA 模型，在长上下文 benchmark 上打赢 235B 级别的顶级 RAG 系统 · 2 张 A800 就能跑 1 亿 token 推理。这不是实验室专属，这是创业公司买得起的成本。 说白了，以前的大模型是一个极度聪明但只有金鱼记忆的天才。MSA 想做的事情是，让它真正「记住」。 我们放 github 上了，算法的同学不容易，可以点颗星星支持一下。🌟👀🙏 github.com/EverMind-AI/MSA
English
ＳＴ０１０－１４ retweetledi
My biggest open-source release!
NumKong — 2'000+ SIMD kernels for mixed-precision numerics, from Float6 to Float118.
Started in 2023. Opened the PR in 2024. Finally, merged this week!
RISC-V, Intel AMX & AVX-512, Apple SME & SVE, WASM Relaxed SIMD. 200'000 lines of code in a 5 MB binary. Same scale as OpenBLAS. Available for C 99, C++ 23, Python 3, Rust, Swift, GoLang, & JavaScript.
Int4 dot products via nibble algebra. Ozaki Float64 GEMMs on Float32 tile hardware. 6-bit and 8-bit floats back-ported to 10-year-old CPUs. 5'300x faster Geospatial metrics than GeoPy. 200x faster Kabsch than BioPython. 0 ULP where OpenBLAS hits 56... and a lot more!
pip install numkong
Or pull it from NPM, Crates, GitHub... and let me know what breaks 🤗
Links & highlights ⬇️
English
ＳＴ０１０－１４ retweetledi
ＳＴ０１０－１４ retweetledi
NVIDIA Feynman は、下記の2点の特徴から
- 3D Die-Stacking
- メモリ: カスタムHBM
2021年の論文
「GPU Domain Specialization via Composable On-Package Architecture」
の中の
Domain-Specialized COPA-GPU for DL の実現か？
長かったね。。。
日本語
"anyone could build a RISC core even the most retarded hw engineer could get you that in a day" - I don't know if I feel represented or offended! 🤣
bubble boi@bubbleboi
Anyone could build a RISC core even the most retarded hw engineer could get you that in a day. The base ISA is so simple it doesn’t even support floating point numbers lmaooo. You could literally write one in less than >1000 lines of Verilog. So I would like to ask the author to provide more context and some source files to analyze did it pass the RISC-V verification suite? I personally use AI tools everyday to write RTL and it’s god awful.
English
ＳＴ０１０－１４ retweetledi
ＳＴ０１０－１４ retweetledi
ＳＴ０１０－１４ retweetledi
ＳＴ０１０－１４ retweetledi
ＳＴ０１０－１４ retweetledi
It ain't rocket science... It's way f*cking harder...
🌿 lithos@lithos_graphein
Play Mode: Easy / Nightmare #terafab
English
ＳＴ０１０－１４ retweetledi
ＳＴ０１０－１４ retweetledi
