Ivan Kuleshov

Where's the Money?

Do you remember?

Lovely

The new model from JetBrains. Works great on GPU

The perfect use case for nvidia.com/en-us/products…

Introducing the Lamark agent, The agent that actually trains the model for you. github.com/merocle/lamark… It's designed with a focus on the “Local First” philosophy. I tested and developed it primarily for the Nvidia DGX Spark to leverage its strengths - specifically, its ability to fine-tune models. The agent learns every night through conversations with you. The focus on tools and the ability to call cloud AI models is implemented as a tool, too. The agent is completely open-source The next step will be to add support for more devices, and of course, the new @NVIDIAAI RTX Spark. It is based on Hermes and supports all of its features. Although personally, I've mostly used Telegram and the CLI I'm Inviting Contributors!

When I write “Hello” to Claude

@stvchu Via TB5

@Merocle how did you connect these books?

It’s finally up and running! And the result is pretty decent. Now I want to put this 1U platform... 😅

@seslly Not for a long time 😂

@Merocle nice thing is you have a built in battery backup

@lozhnikov_ Will check

@Merocle Did you check if a single m3 ultra 512gb is faster?

@lozhnikov_ I would say around 25K

@Merocle Is this setup around €18K?

We're benchmarking every model, every quant, on every different hardware setup for every price point. All developers, companies, and people will have access to local, open source intelligence. Releasing soon.

There are some fairly detailed tests here reddit.com/r/LocalLLaMA/c…; overall, I got roughly the same results, but I only tried one model from this list. However, the heat, noise and cursor lags (probably because of high GPU utilization) when running the LLM ruin the whole experience

@Merocle @exolabs Hows prefill speed? On coding agents that’s what kills me on my m3. Can generate tokens fast enough but hits the wall when its time to read a bunch of files

Some real-world test results. 1. 2-node model – simple coding 2. Single-node model – similar task 3. Single-node model – simple chat (Hi, what can you do?) 4. 2-node model – simple chat As for the software @exolabs – it delivered the best results of everything I tested (but the choice for Mac OS isn’t great) A 4-node cluster is next. What’s really cool is that loading the model is significantly faster than on Spark (takes 15–20 seconds, rather than 5–10 minutes). But even with simple chats, the laptop starts making an obscene amount of noise.

I see. It’s just clever marketing. The original photos are brilliant, and now millions of people know this phone exists 🤣

But why?

There isn't a single positive comment under this post

@EgorPochekutov 128GB решают и проц круче

@Merocle Разве не дешевле было бы использовать 4 Mac mini вместо четырех Mac book?

The "S" in "Local AI" stands for Savings

Cluster testing is currently in progress