C. Taewan

My new employeeee

DIY server rack. My plan is. 3. Edge devices -> AI Research 2. RTX 5090 2x -> AI Research 1. RTX PRO 6000 8x -> LLM SERVE

@MiaAI_lab I'm happy guy

Who said money can't buy happiness?

@wicksplay 2020 profile. DIY

@taewan_02 What frame

My new employeeee

@spaceodili 3000w x2 but cable is shipping now..

@taewan_02 Let’s see the power supplies

8 RTX PRO 6000 GPUs = the productivity of 10+ people. 🚀

@andrew_Tas Up to 8

@taewan_02 6! nice, very nice

@Launch1Labs My sub agent in DGX Spark.

@taewan_02 Whats better... one super employee or a half dozen narrowly focused less powerful process executors?

@MikeBradleyAI TRX 50 AI TOP, Phanteks ENTHOO PRO 2

@taewan_02 What motherboard and chassis did you go with?

Super Insane!! at 4x RTX PRO 6000 madeby561/GLM-5.2-MXFP8-NVFP4-NF3-Hybrid

Nvidia/GLM-5.2-NVFP4 running on 4x rtx pro 6000 with 400k context @ 82 tok/s and is doing amazing I have to water cool these GPUs, the temperatures and sounds are limiting how far I can push this thing This new moet compression is going straight in local studio for onboarding

@BizDevKevv Yes :)

@taewan_02 Did they stuff 4 6000s on a trx50?

@giles_home It seems fine so far since I installed a lot of case fans.

@taewan_02 Do the middle two cards not cook?

GLM-5.2 Nvidia NVFP4 No pruning, no quantising, exact same model card. 110 tok/s single stream. My friend is a genius, holy fudge. Here's the repo, he got full DS4-Flash on a 5090 + DDR5 github.com/kacper-daftcod… at 38.5 tok/s This is a breakthrough.

GLM-5.2 running on 4x RTX Pro 6000s Full model, unpruned. - Top 32 experts for coding and agentic at nvfp4 - Remaining 196 experts at FP3 autoround tuned - Gates MXFP8 - FP16 for dense layers - 400K context FP8 Took this from the RTXPRO6000 discord and applied REAP scoring

Hy3-NVFP4 running on 2 RTX PRO 6000 Blackwells Unfortunately, could only get 90K context right now. Details: > LibertAIDAI/Hy3-NVFP4 > MoE model: 295B total params, 21B active params > Native context: 262K > Served context: 90,112 (memory constrained) > Precision: ModelOpt NVFP4 > KV: FP8 > Runtime: vLLM Hy3 CUDA 13 image w/ TP=2 25 tok/s single-stream decode 96.3 tok/s C8 decode ~2.3K tok/s near-limit prefill at 78K tokens HF in comments

@Hikari_07_jp @tare_azure Wow

🔥🙏 Thanks to @tare_azure for access to a server with 8x RTX PRO 6000 Server Edition GPUs! Daily X research posts have been great. Now maxing this to supercharge model dev! 🚀💪 Open source & open weights as always. これからもよろしくおねがいします！

@Hikari_07_jp I'm really looking foward to your research

While training a draft to apply JetSpec to 27B, we found that a chain approach is more suitable for this model than a tree approach. Decoding speed = number of accepted tokens ÷ time taken per check. Applying a tree approach to GatedDeltaNet did not contribute to speed improvement because the verification cost increases with each additional branch. Although our hypothesis was incorrect, this is a significant step forward. For 27B, we will apply DSpark, published by DeepSeek. DSpark has a mechanism to prevent the accuracy of predicted tokens from deteriorating in the later stages. This seems advantageous for chain-based speculative decoding. We will continue our research.

DwarfStar(ds4)로 DeepSeek V4 Flash/Pro를 DGX Spark, Mac Studio, Mac mini에서 돌려봤습니다. Flash q2는 Mac Studio에서 빠르고, 64GB Mac mini에서도 SSD streaming으로 실행은 됩니다. 하지만 RSS/커뮤니티 제목 번역 품질은 Gemma4 31B Dense MTP 쪽이 더 안정적이었습니다. seapy.com/ds4-deepseek-v…

@Hikari_07_jp He is very kind

I often receive DMs asking about model selection and configuration—please feel free to reach out with your questions! I am particularly confident in my expertise regarding the Blackwell architecture and inference engines👍