@she_llac probably not “curable” if it underlies continual/effective learning
English
lineardiff
1.2K posts
lineardiff
@lineardiff
isolation, perception, and communication
Katılım Haziran 2023
409 Takip Edilen164 Takipçiler
@kalomaze the only thing stopping you getting pwned these days is being unimportant enough or not worth the time
English
Feross@feross
🚨 CRITICAL: Active supply chain attack on axios -- one of npm's most depended-on packages. The latest axios@1.14.1 now pulls in plain-crypto-js@4.2.1, a package that did not exist before today. This is a live compromise. This is textbook supply chain installer malware. axios has 100M+ weekly downloads. Every npm install pulling the latest version is potentially compromised right now. Socket AI analysis confirms this is malware. plain-crypto-js is an obfuscated dropper/loader that: • Deobfuscates embedded payloads and operational strings at runtime • Dynamically loads fs, os, and execSync to evade static analysis • Executes decoded shell commands • Stages and copies payload files into OS temp and Windows ProgramData directories • Deletes and renames artifacts post-execution to destroy forensic evidence If you use axios, pin your version immediately and audit your lockfiles. Do not upgrade.
ZXX
@leothecurious imo it’s likely to be one of the most critical aspects of human learning, although i think the augmentation probably happens internally in a semi-abstract way. i think this mechanism might be one of evolution’s largest contributions.
English
@beffjezos obv they’ll just fork it and old vulnerable coins will become worthless
English
Gotta give it to Anthropic, the whole AI danger fearmongering narrative is quite the marketing trick.
Disclose.tv@disclosetv
JUST IN - Leaked documents from Anthropic show that a new generation of super-strong models, "Claude Mythos," is already in testing with Anthropic believing it "poses unprecedented cybersecurity risks." — Fortune
English
@RyanPGreenblatt been a fan of ARC for many years now, since Icecuber. think the guys behind it are great, but worried it’s starting to push a bit into bad faith territory now.
English
I wish they published the performance for each human baseliner rather than just the performance of the second best human run on each task.
My current guess is that the median human baseliner would score around ~15% on the metric but we can't check because the data isn't public!
ARC Prize@arcprize
Announcing ARC-AGI-3 The only unsaturated agentic intelligence benchmark in the world Humans score 100%, AI <1% This human-AI gap demonstrates we do not yet have AGI Most benchmarks test what models already know, ARC-AGI-3 tests how they learn
English
@AcerFur they plausibly have the compute, but probably not. they do not have the data, but they could buy it.
English
lol Apple have the compute and data necessary to train decent models of their own
Why aren’t they
Mark Gurman@markgurman
Apple will let any AI platform - big apps include Gemini, Claude, Alexa, Meta AI etc. - to be queried in Siri if they enable an Extensions service inside of their iOS, macOS or iPadOS app. Apple will have a new section in the App Store. Unclear if there’s an approval process.
English
@lineardiff Not anthropics thing. They got caught out. Unless intentionally sloppy.
English
“ A draft blog post that was available in an unsecured and publicly-searchable data store prior to Thursday evening said the new model is called “Claude Mythos” and that the company believes it poses unprecedented cybersecurity risks. “
English
Real-time Moondream inference using our new inference engine
moondream@moondreamai
VLMs too slow for production? Not anymore: 46ms end-to-end inference, 60+ fps on a single H100. Introducing Photon, Moondream's inference engine. Runs on everything from edge to server. moondream.ai/blog/photon-re…
English
here's @JeffDean talking about how labs will do multi-epoch pretraining with heavy regularization to keep scaling even with limited data.
no wonder slowrun gets so much attention from pretraining teams at big labs. pretraining is about to look very very different.
English
this will result in the greatest text compression tool ever made
-98% to file size
shellac@she_llac
what did i just wake up to when i went to sleep it was at 0.90
English
noticing an obvious LLM inflection while reading a passage is like accidentally biting your tongue while eating
English
@lineardiff i am saying you could
- pretend a sequentially deep linear network is parallel during fwd (get true loss fast)
- do backward over the sequential structure (do sequential chain rule matrix products)
- get better inductive bias from backprop sequentiality, maintain fwd parallelism
English
most practitioners in ML know that you can't express deeper functions with pure linear projections that lack nonlinearities, but don't let this distract you from the fact that chain rule optimization over deeper linear networks imposes a factorized prior over optimization itself
English
@lineardiff in principle you could... optimize for a chain that the forward pass structure never computes, and get cleaner/better gradients, no inference cost, just bwd cost... (ofc you would need to do activation checkpointing but that's fairly typical anyways)
English
@lineardiff by my intuition, nonlinearities are doing selection/implicit branching, while the decomposition of linear projections in sequence is the actual primary thing causing backprop to build representations hierarchially (in a typical fixed depth fwd pass)
English