Sabitlenmiş Tweet
AI server and homelab setup is rapidly becoming a solved problem. It should feel that way for everyone. You don’t need to start from scratch, and the solutions are free and open source.
github.com/Light-Heart-La…
English
Mike Bradley
3.6K posts
Mike Bradley
@MikeBradleyAI
Helping to make Opensource AI the default @OsmanticAI
Philadelphia, PA Katılım Aralık 2022
304 Takip Edilen10.6K Takipçiler
@trikcode It’s true, I’ve genuinely seen people who were using a lot of AI ROI hardware in less than 5 months. And to do that with all the other benefits? It becomes a no brainer.
English
local AI is winning because:
- no $200/month subscription
- actually private
- runs when internet dies
- nobody can revoke your access
Ahmad@TheAhmadOsman
MASSIVE NEWS Teamed up with NVIDIA to make Local AI The Default
English
Hey that’s totally fair! If one CLI command to set up an entire local AI server still feels like a jump, you could also just point your favorite agent at the repo and it’ll run the command for you. Or! Come hang out on our Discord and a small army of people would jump to show you how to run the command 💪❤️
discord.gg/DqNatZXDv
English
i see it constantly now, everyone "making local AI the default," launching platforms to bring it to the masses. and i've yet to see a single one that actually solved the infra for real users.
here's the irony nobody says out loud:
almost every "local AI for everyone" tool still needs you to be a computer expert just to stand it up. you have to already know the thing it claims to remove the need for. that's not solving infra, that's a dev kit with better marketing.
the actual problem is making local AI work for someone who isn't an engineer, is still wide open. i see it clearly.
Sudo su@sudoingX
my name is sudo and i'm 26. i am going to build the biggest data centers in southeast asia. not to go chasing users. because the demand from what i'm building will get so big i'll have no choice but to own the metal myself. datacenters in southeast asia. then in space. remember the name. sudo.
English
@keennay I can't believe I get to meet the amazing YANNICK!!! See you soon my friend 💪♥️
English
I'll see yall on Thursday!
Ahmad@TheAhmadOsman
MASSIVE NEWS Teamed up with NVIDIA to make Local AI The Default
English
@Baxate Great meeting you today my friend, LFG (host Local AI) 💪♥️
English
Local AI Summit
AI Engineer World Fair
Thursday
Be there
Ahmad@TheAhmadOsman
MASSIVE NEWS Teamed up with NVIDIA to make Local AI The Default
English
Opensource AI will win.
Excited to see what comes ahead with @TheAhmadOsman's new adventure
Ahmad@TheAhmadOsman
MASSIVE NEWS Teamed up with NVIDIA to make Local AI The Default
English
@benitoz @TheAhmadOsman @NaderLikeLadder @Baxate @alexocheema @aiDotEngineer That random other guy seems nice too 🤣
English
Excited to see my brothers @TheAhmadOsman @NaderLikeLadder @Baxate and @alexocheema Local Mogging Dario with frontier models on your own device
Tune in Thursday at @aiDotEngineer
LFG 🚀🇺🇸🙏🏽
$NVDA
Ahmad@TheAhmadOsman
MASSIVE NEWS Teamed up with NVIDIA to make Local AI The Default
English
@jasonkneen There actually are snacks in the fridge if you want 😁
English
Welcome to the local-first AI club -- not had many new members for a couple of years but we've been busy.
Snacks in the kitchen, cold ones in the fridge -- there's a lot to bring you all up to speed with.
Ahmad@TheAhmadOsman
MASSIVE NEWS Teamed up with NVIDIA to make Local AI The Default
English
Mike Bradley retweetledi
We actually kickstart things early tomorrow with 2 workshops on Local AI
See you guys tomorrow
Ahmad@TheAhmadOsman
MASSIVE NEWS Teamed up with NVIDIA to make Local AI The Default
English
@Hikari_07_jp I really want to see you swap the order on those GPUs so you don't have 600w blowing into a 300w rated card. Please do it for me I beg of you 🤣♥️
English
My dual RTX PRO 6000 setup is currently training a Draft model for Qwen 3.6 27B! 🔥
I'm taking the paper DeepSeek dropped on 6/26 and going for a super ambitious application to the 27B scale. Thanks to my homelab, I was able to dive straight in — I read the paper and immediately started experimenting. The amount I've learned has been insane:
- How memory bandwidth bottlenecks speed and clever ways to hack around it
- Methods to train the draft model and boost its accuracy
- Mechanisms to reference tokens all the way back to the previous one to skyrocket draft acceptance rates
- The impact of Attention vs. GateDeltaNet on speculative decoding performance and how to handle those differences
- The unique approaches and trade-offs of MTP, Dflash, JetSpec, and DSpark
I could go on forever, but just from speculative decoding alone I've learned so much. The 27B architecture feels way more DSpark-native than JetSpec, so once draft training finishes, I'm going all-in with DSpark! My goal is to beat existing speculative decoding speeds outright — no task-specific shortcuts or cheating, pure general improvement.
If you're into this kind of research, I'd love to hear your thoughts, impressions, and any suggestions — please reply! 🚀
English
Qwen3-Coder-Next has rocked for me on OC and shipped a lot work I'm very happy with, but OC isn't really designed to be production local friendly or at least wasn't when I was using it. I had to do a ton of mods to get it cooking with vLLM and build my own setup. But I've had a lot of fun and done some awesome stuff with local agents. What models and hardware are you running that you aren't happy with and what are you trying to do? Happy to share setups and tips
English
@petergyang I don’t buy the local model story. We get bug reports that are essentially “this cutting edge model can’t reliably reference these 15.000 facts I dumped into the session”. I want local to work, but even with insane hardware you aren’t getting performance folks expect.
English
I'm trying to wrap my head around these two ideas:
1. Cloud agents are coming we should use VPS vs. our laptops
2. Everyone should buy hardware to run local models
Aren't they conflicting a bit?
English
@MatthewBerman @iAmHenryMascot @exolabs Definitely not my friend. Though you can network them and coordinate loads.
English
@iAmHenryMascot @exolabs Do they behave all as one machine?
English
256gb dgx spark x2 (just acq)
128gb m5
16gb m3
20gb 2 iphones + ipad + android
8gb RasPi
428gb total
400gb usable for inference using @exolabs
Still looking forward to the rumored 700gb studio
Sudo su@sudoingX
okay nerds, how much memory do you actually own right now? not rented, owned, sitting on your desk. i'll start: > dgx spark, 128gb unified > strix halo, 128gb unified > 5090 laptop, 24gb vram + 64gb ram > 3090 node, 24gb vram + 32gb ram > 3060 node, 12gb vram + 16gb ram > old acer laptop, 8gb (yes it counts) > phone, 12gb ram 448gb of memory i own outright. all mine. flex yours.
English
Mike Bradley retweetledi
Watch us make Local AI the default
Ahmad@TheAhmadOsman
MASSIVE NEWS Teamed up with NVIDIA to make Local AI The Default
English
@MikeBradleyAI Will you be around SF this week? Would love a chat!
English
@MikeBradleyAI Hell yeah, they should be watching us make Local AI The Default 🔥
x.com/MikeBradleyAI/…
Mike Bradley@MikeBradleyAI
Watch us make Local AI the default
English
WERE HOSTING LOCAL AI SUMMIT
THIS THURSDAY
AT AIE IN SF
STATE OF THE UNION
EPIC PANELS
DGX STATION RUNNING GLM5.2
LET IT RIP 🤙
Ahmad@TheAhmadOsman
MASSIVE NEWS Teamed up with NVIDIA to make Local AI The Default
English
