@billhfoos My wife just found out, and now I'm having a serious conversation 😅
English
Max Zanoga
1.1K posts
Max Zanoga
@zanoga
🚀 AI Enthusiast | 🔍 Exploring the frontiers of LLMs & ML | 📚 Lifelong learner & tech optimist
LAB32 Bergabung Ekim 2009
1.8K Mengikuti5.8K Pengikut
This is probably the worlds best working example of what local ai really means
Max Zanoga@zanoga
Finally finished building my AI datacenter! 🚀 32x3090s across 4 servers (8 GPUs each), all connected over InfiniBand. The whole setup is solar-powered with a massive battery bank and generator backup. More technical details and benchmarks coming soon.
English
@Deriglasow @AndJakobsson Privacy and just try processing 10M documents through a paid API - the bill will give you your answer.
English
@zanoga @AndJakobsson what for? I have all the money to build something like that, but whats the purpose?
English
Finally finished building my AI datacenter! 🚀
32x3090s across 4 servers (8 GPUs each), all connected over InfiniBand.
The whole setup is solar-powered with a massive battery bank and generator backup.
More technical details and benchmarks coming soon.
English
@ktusing34 Around $30k, and that's just GPUs + CPU + RAM + Motherboard + NVMe + PSU.
Network: $4k
Solar station + generator + other stuff: up to $10k more. (All very rough estimates)
English
@ktusing34 Need to calculate, but I bought all before the memory crisis. I bought GPUs in the $550 - 650 range.
English
@TheZachMueller Ok, I promise to make an additional post about solar + batteries and how I chose and configured all this.
English
@zanoga Would love to poke your thoughts on solar power more. Choices? How did you calculate what you need? Etc
English
@rosmine For all kinds of stuff - from dataset creation to coding assistance on private projects, and a lot more.
I'll share all my experience and use cases in the future.
English
@micheltamanda It all starts innocently with "I just bought 2x3090s" 😅
English
@mattstaton Around $30k, and that's just GPUs + CPU + RAM + Motherboard + NVMe + PSU.
Network: $4k
Solar station + generator + other stuff: up to $10k more.
(All very rough estimates.)
English
Thank you all for the interest in my setup! 🙏
I'll read and reply to every message.
As promised:
Tech specs of my AI cluster 👇
- Board: ROMED8-2T
- CPU: EPYC 7302
- RAM: 256GB DDR4-3200
- Storage: 2TB NVMe
- Risers: ADT-Link MCIO
- PSU: 2xSuper Flower Leadex 2000W
Network:
- ConnectX-6
- SB7800 InfiniBand
Cooling:
- 8x12mm fans exhausting GPU heat
- 9733 for the ConnectX-6
Max Zanoga@zanoga
Finally finished building my AI datacenter! 🚀 32x3090s across 4 servers (8 GPUs each), all connected over InfiniBand. The whole setup is solar-powered with a massive battery bank and generator backup. More technical details and benchmarks coming soon.
English
You can blame this guy for 3090's being expensive.
🤣
Max Zanoga@zanoga
Finally finished building my AI datacenter! 🚀 32x3090s across 4 servers (8 GPUs each), all connected over InfiniBand. The whole setup is solar-powered with a massive battery bank and generator backup. More technical details and benchmarks coming soon.
English
@gospaceport Thank you for your work! You really make interesting content that helps people!
English
@jenzhuscott I am very interested! I think @__tinygrad__ has already done something similar.
English
If we built 4 server unit comes with immersion cooling (also eliminating the noise & fire risk), batteries (also immersion cooled), solar sets, all electronics needed in one residential local micro data center - would anyone be interested?
Max Zanoga@zanoga
Finally finished building my AI datacenter! 🚀 32x3090s across 4 servers (8 GPUs each), all connected over InfiniBand. The whole setup is solar-powered with a massive battery bank and generator backup. More technical details and benchmarks coming soon.
English
@0xSero @royjossfolk Yes, I’ve got a garage for it 😄 and the AC runs 24/7
English
@zanoga @royjossfolk Holy moly what the hedge
Do you have a garage for this? The heat!
You beat me to 784gb vram
English
Needs some greek statues and it will be perfect
Max Zanoga@zanoga
Finally finished building my AI datacenter! 🚀 32x3090s across 4 servers (8 GPUs each), all connected over InfiniBand. The whole setup is solar-powered with a massive battery bank and generator backup. More technical details and benchmarks coming soon.
English