Mutsumi

Brief introduction: A Backend-focused Software Engineer in Tokyo🗼🇯🇵 Working on modernizing our legacy Ruby/RoR system to a newer Go system, using LLMs in product features with Python/Celery worker, improving CI/CD pipelines, and structuring AI development workflows. NTMU😊🤝🏻

How have the fundamentals of building large, distributed software systems changed the last decade? A conversation with Martin Kleppmann (author of Designing Data-Intensive Applications) - given that the second, updated edition of the book was just released. Timestamps: 00:00 Early career 05:46 Building Rapportive 10:47 Working at LinkedIn 14:09 Writing Designing Data-Intensive Applications 23:00 Reliability, scalability, and repeatability 26:24 DDIA: the second edition 30:50 Tradeoffs of using cloud services 39:02 How the cloud changed scaling 42:53 The trouble with distributed systems 49:02 Ethics for software engineers 52:45 Formal verification 1:00:12 Academia vs. industry 1:03:50 Local-first software 1:09:50 Computer science education 1:18:32 Martin’s current research and advice Brought to you by: • @statsig  – ⁠ The unified platform for flags, analytics, experiments, and more. statsig.com/pragmatic • @SonarSource – The makers of SonarQube, the industry standard for code verification and automated code review. Check out Sonar's new architecture management capabilities that ensure both humans and AI agents respect your system’s blueprint. sonarsource.com/solutions/arch… • @WorkOS – Ship enterprise features – SSO, directory sync, RBAC, audit logs – in days, not months. workos.com Three things worth considering, as discussed with Martin, in this episode: 1. Multi-region and multi-cloud are risk/cost trade-offs, not best practices. Martin does not believe that there is a “best practice” in deciding whether to go multi-region or multi-cloud. This decision is a tradeoff between risk and costs. It’s a business decision to be made. Designing Data-Intensive Applications gives engineers the vocabulary to articulate the tradeoffs, not to dictate answers. 2. Replication for fault tolerance is more relevant for most engineers these days than sharding. Though the book has a full chapter on sharding, Martin said that the cloud has reduced the need for manual sharding for the majority of teams. This is also because machines are increasingly bigger, and more workloads fit on a single machine. Sharding across machines is increasingly a specialist concern; replication for fault tolerance, however, is still relevant at every scale. 3. Knowing system internals as a superpower for application developers. Martin maintains that Designing Data-Intensive Applications is not a book for people who build databases or even infrastructure, but it’s helpful for application developers to develop an intuition for making good design decisions and debugging performance issues we will eventually encounter.

Can't Wait!

Tomorrow on The Pragmatic Engineer Podcast: Designing Data Intensive Applications with Martin Kleppmann

Excited to partner with the SpaceX team to scale up Composer. A meaningful step on our path to build the best place to code with AI.

Excited to see how Cursor accelerates their Growth with SpaceX

@Malix_Labs Which do you mean, Go itself or Go with agents?

@mutsumi_dev it's one of the weakest forms of type safety available though

Sounds interesting! I've written Go with agents for over two years and it's obvious Go is excellent with them. Go is type safe and the simple grammar and syntax works well with agents. But I didn't imagine how `go doc` and `gopls` are efficient. I'll try it on my environment.

@complex_maths Which model have you used and how was it? I'd also like to know which language works well with the model.

@mitchellh @mutsumi_dev Going to give this a try with some of the open weight models, they sometimes struggled a bit with Go in my experience but it’s been a while since I tried them.

@xm_build I've never heard the Goreplay before. This one? github.com/probelabs/gore…

@mutsumi_dev go doc and gopls are indeed powerful, but don't overlook goreplay for debugging agent workflows in real time

@mitchellh Thanks! I'll try it soon.

@mutsumi_dev Just put a sentence or two in your agents.md to use go doc and gopls for API and doc inspection and it’ll do amazing.

Wow! That's soooooo cool!

I Read Designing Data-Intensive Applications 2nd Edition and It’s Awesome by @soma_81597 medium.com/javarevisited/…

You can also set your autocompact threshold yourself and effectively lower your context window if you'd prefer. For example, 400k context is a good compromise: CLAUDE_CODE_AUTO_COMPACT_WINDOW=400000 claude see docs here: #environment-variables" target="_blank" rel="nofollow noopener">code.claude.com/docs/en/settin…

This guide was interesting to me. I always started a new session when I failed to run something. Regarding this blog, it seems like `/rewind` and `/compact` may help me to accomplish complex tasks. At the end of this blog, there is a table of situations, which is also useful.

@steipete Do you use any skills to explore an unfamiliar codebase? Just talking is not enough to make it understandable for AIs, in my case. If you have some tips to help AIs dive into a huge codebase and become experts in it.

I never use plan mode. The main reason this was added to codex is for claude-pilled people who struggle with changing their habits. just talk with your agent.

@mattpocockuk Did you create an introduction video on how to use `/grill-me`? It sounds good to me, but I'd like to know which use case is suitable for this skill.

I have also stopped using plan mode It creates a plan FAR too eagerly and usually asks you zero questions en route The whole point of planning is to get on the same wavelength with the LLM, not to generate an asset you don't read /grill-me all the way

@mitchellh How about using GitLab on your local machines? I've never tried it, but some people say it's good, and I'd like to know your opinion about it.

Every. single. day. It's increasingly becoming difficult to do real work with GitHub. Git isn't the issue, since I can work offline. Its issues, PRs, CI, etc. Imagine going to work and your workstation randomly restarts a few times a day. That's what it feels like using GH.

I created a skill that analyzes a UseCase and its dependencies, and generates a UseCase test. It works really well, actually. Before creating it, reviewing and understanding my own generating process had a huge impact on it. That was the key of creating a skill.

Claude Code tried to improve /init... Is it any better?