Vivek

Breakdown of my February AWS bill to run my side projects: EC2: $44.22 RDS (reserved Aurora MySQL): $41.31 ELB: $16.96 Data Transfer: $15.12 VPC: $11.61 CodeBuild: $1.29 S3: $1.10 ECS: $0.67 ECR: $0.29 ------------ Total: $132.57 For completeness, here's August to February- August: $203.95 September: $210.77 October: $245.98 November: $261.70 December: $221.30 January: $146.65 February: $132.57 In November, I moved all my instances from Fargate to EC2. <--- cheaper and much more performant. In December, I fixed the binpack strategy for one of my projects so I didn't pointlessly run an extra EC2 instance. I also moved my RDS to a reserved instance. In January, I moved the most resource intensive scheduled jobs to Fargate and I was able to drop the base container size, which dropped the EC2 instance sizes. Specifically I am able to see that my scheduled Fargate jobs ran for 13 hours and cost a total of $0.67. No changes in February that I remember, but it's 3 days shorter than January so 🤷‍♂️

With the coming tsunami of demand for tokens, there are significant opportunities to orchestrate the underlying memory+compute *just right* for LLMs. The fundamental and non-obvious constraint is that due to the chip fabrication process, you get two completely distinct pools of memory (of different physical implementations too): 1) on-chip SRAM that is immediately next to the compute units that is incredibly fast but of very of low capacity, and 2) off-chip DRAM which has extremely high capacity, but the contents of which you can only suck through a long straw. On top of this, there are many details of the architecture (e.g. systolic arrays), numerics, etc. The design of the optimal physical substrate and then the orchestration of memory+compute across the top volume workflows of LLMs (inference prefill/decode, training/finetuning, etc.) with the best throughput/latency/$ is probably today's most interesting intellectual puzzle with the highest rewards (\cite 4.6T of NVDA). All of it to get many tokens, fast and cheap. Arguably, the workflow that may matter the most (inference decode *and* over long token contexts in tight agentic loops) is the one hardest to achieve simultaneously by the ~both camps of what exists today (HBM-first NVIDIA adjacent and SRAM-first Cerebras adjacent). Anyway the MatX team is A++ grade so it's my pleasure to have a small involvement and congratulations on the raise!

We just hired a new senior engineer. Starts next month. His salary is $140K. That's $15K more than my current senior engineer who's been here for four years. My current guy doesn't know this yet. But he will eventually. And when he finds out, he's going to be pissed. Here's what I'm banking on: he'll come to me asking for a raise to match the new hire. I'll act sympathetic. I'll say the market has changed, and yes, it's frustrating, but my hands are tied by HR compensation bands. Then I'll offer him $10K more. He'll take it because it's better than nothing. Net result: I keep a good engineer happy with less money than he could've gotten, and I look like I fought for him. The new hire? He has no idea he could've probably gotten $150K if he'd negotiated harder. I lowballed the offer and he accepted immediately. So I'm paying two senior engineers a combined $25K less than market rate, and both of them think I'm on their side. Salary management is just information asymmetry with better branding.

Nikhil Kamath (and a lot of Indian podcasters/founders) love projecting this image that their audience is full of future founders and unicorn builders. But let’s be honest: 80%+ of the people watching these podcasts are mid-senior level corporate professionals in Bangalore, Gurgaon, Hyderabad, and Pune who: - Earn 20L - 1CR - Secretly want to “start something” but won’t leave the job till the next band kicks in They’re not building the next Zerodha. They’re just hopping between MNCs and startups for that extra hike so they can soon afford a 3BHK in North Bangalore without a 30-year EMI eating their sanity. The entrepreneur cosplay is strong, but deep down we’re just highly educated, mildly burnt-out employees refreshing LinkedIn every 18 months. And honestly, that’s fair. Most of us grew up middle class in a developing country where uncertainty wasn’t a playground - it was pressure. You can’t expect the same risk appetite as someone who grew up in SF with safety nets baked into their life story.