antirez

My second short story release in English is ready: Tales of Illustrious Computer Scientists: Iola Varga, nun and computer scientist. invece.org/iola.html

@lastknight I wish it was that. Apparently people in order to have more popular X accounts, paste conversations to LLMs saying: "tell me something insightful I could say here". And when they interact a lot like that, X algorithms give them some boost. It's nuts.

@antirez For you: arxiv.org/abs/2409.01754

Recently here on X there is this thing of replying to tweets with a vague remark of what the tweet expressed. I see this for months now. I thought most were bots, but as I investigate, I see that many are legitimate humans that are starting acting as bots. Worrying.

@chrisvbuskirk We invented Trumpismp with Berlusconi before there was ever Internet. The idea that Trump is elected because of social network manipulation is something I don't believe even for a second for this reason other than many others.

@antirez Boy this is embarrassing, but its daily for us now....but hey, Italy done some characters is past as well. 😉

AI translated Italy president reply to Trump. Thanks Matteo.

@VerrielloMic It is a family of optimizations applied for many years, nothing new.

@antirez Interesting, just read your issue and at first look the idea (on high level) looks similar to a paper i was reading few days ago. eecs.harvard.edu/~michaelm/post…

I opened an issue in the Redis repository about the 50% memory savings for sorted sets I talked about a few days ago: github.com/redis/redis/is…

@sergey11g Check the DwarfStar documentation about distributed inference please. Two 64GB in a fast thunderbolt link can be fine.

@antirez If this is fundmentally possible with the existing models that don't fit one machine, the scheduling and coordination is doable. Wdyt? Is this the future of local inference?

I'm using the Unsloth 4-bit weights to integrate interence into DwarfStar. If the experiment works well I'll specialize the quants with an optimizer for the best setup. My target is however the 512GB M3 ultra and in distributed inference 4 128GB machines with 3-bit variants if quality holds. Inference quality must be the local inference mantra.

@eddy_mane Those are basically bots on my account, they are just acting as a proxy, an adapter, to let AI write on a humans web site.

@antirez Many of them are just humans who use AI to generate replies to copy and paste here, like they are brainless so they delegate everything to AI.

@twid I perfectly know but they have surely many for these kind of stuff and more.

@antirez The M3 Ultra 512GB is out-of-stock, and doesn't even exist as an orderable SKU now, so really no units to send you.

The feeling that Apple is selling 92304294024 m5 max 128GB macbooks thanks to DwarfStar but doesn't give a fuck about sending me an M3 Max with 512GB (that I would buy if possible but it is borderline note possible) is growing on me :D

@ivanfioravanti @Apple I know. Btw, would happily buy one myself at the normal price, but not on the corners of eBay at 30k...

@antirez Don't tell me... I spent a fortune on these two beasts. I bet Apple will help us with M5 Ultra (🙏 @Apple )

Appreciate, already have access to two (of friends) but would simplify developments if Apple could ship one and I could have it on my desk. So this needed to be told ;D In AI there are money everywhere but the local AI scene is left alone (of course... it's the contrary of making money).

@antirez Happy to share one of mine with you super @antirez 🙌🏻

@sergey11g Among two, latency is not terrible. Among four, it starts to be an issue, but it's worse than that: 4 128GB machines, you may 4 times the operating system memory tax, so you have actually ~95GB each machine to stay safe. It's not ideal... and a huge waste.

@antirez Noob question: is inter-machine latency and throughput a huge speed factor?

@Wauplin That's a very good news. Thanks.

`hf upload` got a full rewrite! Single-pass hashing, multi commits, resumable uploads. Same CLI, way faster, way cleaner. Available in hf latest release (1.20.0).

@__anshukla__ Appreciate the help about remote access but I already got two machines online thanks to friends. But for the CI testing I need a dedicated locale one and for them it's I guess not hard.

@antirez Can you DM me? I may be able to help.

@Youssofal_ Typo but impossibile to edit courtesy of X but anyway...

@antirez M3 Ultra

Jesus X has the most ridiculous ever "edit" implementation.

@experquisite @NVIDIAAIDev The DGX Station would be useless for me since I want to support hardware people can actually buy. At 90k this is an elite thing, the Mac Studios are a much better target.

@antirez Yeah there’s a big gap in the market for 512GB+ unified memory machines, between the now-cancelled studios and the $90k DGX Stations. You could probably get @NVIDIAAIDev to give you a DGX Station though?

@Vamzzz93 Appreciate but it is even hard to find one, and I believe that Apple could benefit a ton from local inference scene, so would be a good idea for them, that have the ability and spending power, to sustain a bit the devs.

@antirez Just out up a crowdfund and we will buy you one