@EastlondonDev Any practical benefit over giving the model a wasm interpreter tool? Cool experiment regardless
English
michael ruddy
97 posts
It’s fucking working
This LLM brain has been fused with a mini computer and it can switch between generating text and generating and executing machine code - all running in a single GPU & torch graph
Andrew Jefferson@EastlondonDev
It turns out that teaching an existing language model new tokens takes a bit of work. To use wasm directly in the neural network I need the language model to output specific wasm tokens and byte tokens (one token for every byte value 0-255) that match the hard coded wasm interpreter subgraph. There are two problems. 1) the language model has never seen wasm tokens before and 2) when wasm tokens are used they flow into the wasm interpreter which will compute them and will hard fail if given invalid instructions. So the llm has to learn to use tokens it has never seen before in perfectly correct sequences. Thats enough of a challenge that my AI agent couldn’t get SFT on pretrained nanochat language model to work with about a week of trying different approaches. We either got mode collapse where the only wasm token predicted was the most common one (CONST_I32) or it learned to use the wasm operations but completely lobotomised the language model in the process and it could not produce correct byte values for inputs.
English
The next step for autoresearch is that it has to be asynchronously massively collaborative for agents (think: SETI@home style). The goal is not to emulate a single PhD student, it's to emulate a research community of them.
Current code synchronously grows a single thread of commits in a particular research direction. But the original repo is more of a seed, from which could sprout commits contributed by agents on all kinds of different research directions or for different compute platforms. Git(Hub) is *almost* but not really suited for this. It has a softly built in assumption of one "master" branch, which temporarily forks off into PRs just to merge back a bit later.
I tried to prototype something super lightweight that could have a flavor of this, e.g. just a Discussion, written by my agent as a summary of its overnight run:
github.com/karpathy/autor…
Alternatively, a PR has the benefit of exact commits:
github.com/karpathy/autor…
but you'd never want to actually merge it... You'd just want to "adopt" and accumulate branches of commits. But even in this lightweight way, you could ask your agent to first read the Discussions/PRs using GitHub CLI for inspiration, and after its research is done, contribute a little "paper" of findings back.
I'm not actually exactly sure what this should look like, but it's a big idea that is more general than just the autoresearch repo specifically. Agents can in principle easily juggle and collaborate on thousands of commits across arbitrary branch structures. Existing abstractions will accumulate stress as intelligence, attention and tenacity cease to be bottlenecks.
English
@jonaaronbray Looked at the audio analysis. You reference videos by file name.. where are those specific files?
English
Download the forensics package with all the data... followtheepicenter.com
English
The FBI claims a rifle was fired from 142 yards away at a UVU event.
So we did something no one else has done — we used the physics of light vs. sound across 6 independent cell phone recordings to measure the actual distance to the source.
Light arrives instantly. Sound at 343 m/s. The delay = the distance.
Expected delay at 142 yards: 379ms
Measured delay: 35ms
11x too short.
English
@PinkDraconian Generate a bunch of keys w IP restricted ranges, set quotas, rotate often.
Subdivide ranges that hit quotas to narrow down your attacker.
Find and neutralize said attacker.
English
I still don't understand Google Maps API keys.
If you're showing a map on your website, the API key is in your client-side code.
An attacker can use this API key to send millions of requests and you're paying for it.
There's no way to secure it?
English
@realtimsharp “At least we still have a country”. I knew the bar was low but holy shit lol
English
If Donald Trump involves America in Iran I will disagree with him.
I will still support him just as much however. Without Trump’s 2024 victory, we most likely wouldn’t have a country left to even be discussing this.
English
@matt_vanswol @nicksortor You want them to be bombed.. for protesting?
English
@nicksortor It really seems like these people love Palestine!
We should give them a one-way ticket to the place they seem to love the most.
English
🚨 #BREAKING: Pro-Palestine protestors have illegally TAKEN OVER Grand Central Terminal in New York City
DEPORT THEM ALL NOW! They have NO BUSINESS in our country!
Why is this being allowed??!
English
@matt_vanswol @EricLDaugh Yes bc those are crimes and how civil disobedience works. But we should all fight for the first amendment. Also not controversial.
English
@EricLDaugh They should be arrested for trespassing and anyone who is here illegally and is trespassing should be deported.
This is not controversial.
English
BREAKING: Left-wing protestors just took over Trump Tower, NYC by force.
They've overtaken the first floor. They won't leave.
They are chanting "free Mahmoud, free them all" - reference to the pro-Palestine man from Columbia University.
English
@BobMurphyEcon The Great Wall is a barrier to an inclusive society.
English
that would be amazing if the laid off feds went to a new job and imposed regulations and taxes on the chinese
unusual_whales@unusual_whales
BREAKING: US intelligence indicates that Russia and China are attempting to recruit federal employees fired by the Trump administration, per CNN
English
@Conservative1AZ Can’t bluff when your cards are on the table. What does winning mean here? American boots in Ukraine? No thanks.
English
You may like Trump and Vance dressing down Zelinsky, but you seem to be ignoring they are negotiating a win for Putin, who invaded Ukraine.
Russia won today, not America...
English
@Villgecrazylady I mean.. they have a point.
Operation Iraqi Freedom? Good Americans took part in that. Propaganda is a helluva drug. The tribal brain is easily gamed..
English
“…I hope decent people will understand that Israel is not characterized by the dirty games that are played by its intelligence services…”
LOL
LMAO even
So now collective punishment is wrong? Aren’t yall the ones always bragging about the “brilliance” of the 📟 attack?
English
@BuenoForMiami Legalizing hard drugs has its own problems. Mafias can always find a new business in extortion, kidnapping etc.
If cartels are government (they are) and they operate in the US - then isn’t that an invasion by a hostile state actor? That needs to end.
English
You don’t “end cartels” - the cartels become the government.
Just like Gustavo Petro became the President of Colombia after literally being the head of a Narco organization.
The Cartels are winning elections these days - while Americans think they’re just going to drone strike and be done with it.
English
@alexisxrivas Could software automate this? GitHub actions for CAD to run compliance tests..
English
In the USA, you’re innocent until proven guilty — except when it comes to building permits.
This is a big reason housing is so expensive.
The default is that you’re guilty of building “illegally” even if you do everything to code and by the book - unless you first navigate a maze of paperwork and clearances and bureaucratic back and forth to prove you’re innocence.
Why not flip this? Innocent until proven guilty. Let professionals build homes and structures under a certain size. The city inspects at the end. If there’s a code or zoning violation, they flag it for correction. Simple.
San Diego County (and many others) already do this - it’s called self-certification.
On Jan 13, 2025 Los Angeles Mayor Karen Bass directed the Department of Building and Safety to explore this option in 15 days. Those days have passed. No public update.
Self certification should be the norm for ADUs and Single Family Homes, especially on straightforward lots - not just for disaster recovery - but permanently.
This would lower housing costs and free up city staff to focus on larger projects with real public safety impact (like perhaps brush clearance).
It’s time.
Alexis Rivas@alexisxrivas
Day 4 update for our Palisades permit. We got "corrections" and a request for clearances from 4 separate departments, which we'll have to reach out to individually. At LA City this is normal. Some cities have consolidated permit reviews so that people don't have to coordinate with multiple departments. The corrections are in a 14 page doc with 16 highlighted items that the city says need our attention. Many are not applicable to this project. Some are. We'll be reaching out to the plan check engineer to discuss this. Unfortunately, this is pretty typical. This stage of the permit application has been faster than normal, Day 4 instead of what's normally around Day 10-14. Good to see that the expediting is working and fire builds are being prioritized.
English
@DrRupaMarya And who trains the cartels? I like your rhetoric but this onion has layers
English
@russell_m @naval More specifically, throw it away. The first amendment is good enough.
English
michael ruddy retweetledi
CAN SOMEONE ANYWHERE IN THE NORTH CAROLINA GOVERNMENT THAT REPRESENTS THE PEOPLE PLEASE FOR THE LOVE OF GOD GET FEMA ON THE FUCKING PHONE AND EXTEND THESE HOTEL VOUCHERS?!!!!
WE ARE TALKING TO ELDERLY PEOPLE AND WOMEN WITH KIDS BEING KICKED OUT OF THEIR HOTELS, WTF IS HAPPENING
English
@bpcostello What if they got FISA authorization? 🤭
We’ve abandoned the constitution. Xi is the least of our worries.
English
Let's call this what it is, an act of war through the cyber realm. Making a deal with Xi will not happen.
Michael Ron Bowling@mrbcyber
Chinese hackers had access to court ordered surveillance system in US telecoms for over a year. Who knows how much damage was done. aol.com/chinese-hacker…
English
@AFactCheck2U @RepThomasMassie Women using war analogies lol. The dishwashers that way sweetie
English
@RepThomasMassie For all the commenters of hate: I agree with 95% of Massie’s policy. But if you can’t accept an agreement with 90% of what you need, you lose the war and never even get a chance to battle. Please stop being so weak.
English
At the beginning of each Congress, we are given congressional pins to wear and parking plates for DC. Each has our seniority rank in Congress. This year I’m #100 out of 435. I call my pin “my precious” (LOTR reference) and I never wear it because I don’t become Gollum. 😱
English
@thejamesonellis We can import Chinese workers just not their cars. MeRiTocRacy 🤡
English
The new Trump administration needs to let the auto manufacturers fail.
Let it crash - no bail outs!
They need to learn a lesson.
English
@bpcostello So could wiretapping our calls and texts. I don’t care do you?
English
Resolving TikTok’s issues will provide substantial capital to China-controlled ByteDance, directly fueling its AI programs...
Chubby♨️@kimmonismus
ByteDance Research Introduces 1.58-bit FLUX: A New AI Approach that Gets 99.5% of the Transformer Parameters Quantized to 1.58 bits
English
@ID_AA_Carmack XML/HTML makes more sense. Structured, typed, accessible..
English
LLM assistants are going to be a good forcing function to make sure all app features are accessible from a textual interface as well as a gui. Yes, a strong enough AI can drive a gui, but it makes so much more sense to just make the gui a wrapper around a command line interface that an LLM can talk to directly.
English