Nuno Bispo (aka Developer Service)

I'm Nuno Bispo, a Senior Software Engineer and Solutions Architect with 15+ years building Python systems, AI integrations, and production backends. I write for Real Python and Unstract. I build and ship hardware. I've seen what breaks in production and what doesn't. I take on a small number of advisory engagements with teams that are serious about getting their Python and AI stack right.

One of the things that separates a Python AI integration that holds up in production from one that doesn't is how the team thought about failure modes before they shipped rather than after. Not failure in the obvious sense, the API is down, the network is unreachable, the response times out, but the subtler kind of failure that doesn't raise an exception and doesn't trigger an alert and doesn't show up in your error logs because from the application's perspective everything worked fine. The model returned a response. The response passed validation. The data was stored and the user got an answer. But the answer was subtly wrong in a way that's hard to detect automatically, a sentiment misclassified, a summary that missed the key point, an extraction that got the right structure but the wrong values. These failures are invisible to your monitoring stack because they require judgement to detect, not just pattern matching. And they accumulate quietly in production until someone notices that something is off and you have to work backwards through weeks of outputs to understand what happened and when it started. The teams that handle this well build what I think of as a confidence layer around their LLM calls. Not just validation of structure and type, which Pydantic handles well, but periodic sampling of outputs for human review, consistency checks that compare outputs against known good examples, and anomaly detection that flags when the distribution of outputs shifts in ways that might indicate model drift or prompt degradation. None of these are complicated to implement individually. Together they give you visibility into the part of your AI system that your standard observability stack was never designed to cover. The goal isn't to catch every bad output, that's not realistic at scale. The goal is to know quickly when something has changed, so you can investigate before the problem has been running long enough to matter. Silent failures in AI output quality are one of the hardest things to retrofit observability around after the fact. If your integration is live and you don't have a confidence layer yet, that's worth addressing sooner rather than later. Work with Me - developer-service.blog/work-with-me/ #Python #AI #SoftwareEngineering #LLM #DevOps

5 signs your "AI agent" isn't actually production-ready (I learned these the hard way): - You can't tell me what a bad output costs: in dollars, in user trust, in support tickets. - Your only eval is "I tried a few prompts and it seemed fine." - The output schema is a suggestion, not a contract, so downstream code is full of defensive if isinstance() checks. - There's no canary, no rollback, no staged rollout; just prod, all at once, fingers crossed. - You're the runbook. If you're on a plane, the system has no memory of how to fix itself. None of this is exotic. It's the same discipline we've applied to every other kind of software for years, we just haven't caught up to applying it to agents yet. I wrote Production AI Agents with PydanticAI to close that gap: a build-along book where you take a real agent (a GitHub issue triage assistant) from prototype to something you'd actually trust with a pager rotation; typed contracts, CI-wired evals, observability, canary rollouts, and a runbook that survives without you. 📖 Free sample chapters: leanpub.com/production-ai-… #PydanticAI #LLMOps #AIEngineering #PythonEngineering

A scope bug in production is not a Python problem. It's a trust problem. When a system returns wrong data silently with no exception, no alert, no obvious failure, the people who depend on it stop trusting it. And once trust goes, it's expensive to rebuild. I've seen this play out in real systems. A reporting pipeline returning stale figures. An API endpoint serving the wrong user's data. A background job processing the right input with the wrong configuration. All traceable, in hindsight, to a variable resolving to the wrong scope. Python's scoping rules aren't complicated. But they're implicit, which means they only become visible when something breaks. The free Python Scope Guide makes them explicit, before something breaks: - The LEGB rule - the complete name resolution model - Closures, late binding, and why they produce hard-to-reproduce bugs - global and nonlocal - when they're the right tool and when they're a liability - The scope antipatterns most likely to reach production undetected Free. No signup. …variable-scope.developer-service.blog #Python #SoftwareEngineering #ProductionCode #PythonDevelopment

A pattern I've noticed across many of the AI projects I've reviewed is that the riskiest moment in the life of an AI feature isn't the launch. It's the three months after the launch when everyone has moved on. The launch gets attention. There are eyes on the system, people checking the outputs, engineers available to fix things quickly if something unexpected happens. The adrenaline of shipping creates a kind of informal monitoring that compensates for the absence of formal monitoring. Problems get caught not because the system caught them but because people were watching. Three months later the team is two sprints into the next project. The AI feature is running in the background, generating outputs that nobody is systematically reviewing, handling edge cases that weren't in the original test suite, and making decisions that affect real users in ways that have quietly drifted from what was originally intended. The informal monitoring is gone and in most cases nothing has replaced it. This is the window where the expensive surprises live. Not at launch, when everyone is watching. After launch, when everyone has stopped. The teams that avoid this aren't the ones that never move on to the next project. They're the ones that replace the informal monitoring of the launch period with something deliberate and systematic before they move on. Alerting, logging, scheduled output reviews, drift detection, the specific mechanisms matter less than the decision to have them at all. Most teams make that decision after the first surprise. The ones that make it before are the ones worth learning from. If your team is approaching that three month mark on an AI feature and the informal monitoring is starting to thin out, that's the right moment for a production readiness review. Work with Me - developer-service.blog/work-with-me/ #AI #CTO #TechLeadership #SoftwareEngineering #Python

Six months ago I shipped an AI agent to production that worked great in testing and then quietly started hallucinating issue labels the week I went on vacation. Nobody caught it for four days. The eval suite I "meant to write" would have caught it in four minutes. That agent is the reason this book exists. Production AI Agents with PydanticAI is everything I wish I'd known before that week, not the happy-path tutorial where the demo always works, but the boring, load-bearing stuff: typed output contracts so a bad response fails loudly instead of silently, eval suites that run in CI instead of living in your head, cost and latency SLOs, canary rollouts, and a runbook that doesn't require me personally to be awake. You build a real system across the book, a GitHub issue triage assistant, and by the end you'll have a repo you can actually point to when someone asks "is this thing production-ready"? If you've got an agent one bad prompt away from an incident, this is the book I needed before mine had one. 📖 leanpub.com/production-ai-… #PydanticAI #LLMOps #AIEngineering #PythonEngineering #AI

Practical Pydantic by Nuno Bispo is on sale on Leanpub! Its suggested price is $39.00; get it for $16.24 with this coupon: leanpub.com/practical-pyda… @DevAsService #fastapi #python #apis #computer_programming

There is a version of technical leadership that I have enormous respect for and almost never see written about because it doesn't make for a compelling conference talk or a viral LinkedIn post. It's the willingness to slow down and ask uncomfortable questions about something that is already working. Not something that is broken. Not something that has caused an incident or failed a test or generated a complaint. Something that is working fine, shipping value, making stakeholders happy and that a good technical leader looks at anyway and asks whether it is working for the right reasons or just working by luck and circumstance and the absence of edge cases that haven't been encountered yet. That kind of scrutiny is genuinely hard to practice because everything around you is incentivised against it. The roadmap wants you moving forward. The team wants to ship new things. The business wants to see velocity. Pausing to ask hard questions about something that isn't visibly broken requires a kind of confidence that doesn't come from metrics or dashboards, it comes from experience, and from having seen enough things break unexpectedly to know that working and safe are not the same thing. The technical leaders I've worked with who are genuinely exceptional all share this quality. They create space to ask the uncomfortable question before it becomes the urgent question. They treat that space as part of their job rather than a distraction from it. And the teams around them ship with a confidence that isn't just optimism, it's grounded in actually knowing what they have. That's a harder thing to build than any feature. But it compounds in ways that features don't. Creating that space is exactly what a production readiness audit gives you. A clear picture of what you have, before the urgent question arrives. Work with Me → developer-service.blog/work-with-me/ #SoftwareEngineering #AI #TechLeadership #CTO #Python

@reiver_ai I like the slot machine description 👍

'A slot machine' is the perfect description, if you can't version it, test it, or put it on call, it's not a system. The bit that makes it fixable is visibility: the reason teams ship the try/except-and-pray version is the failure is invisible until finance asks. Once a bad output is traceable to the exact call, it stops being a slot machine and starts being software.

Unpopular opinion: most "AI agents" in production right now are a try/except block wrapped around an LLM call and a prayer. I've seen teams ship agent features in a sprint, celebrate the demo, and then spend the next quarter firefighting: silent schema drift, no eval suite, no idea what a bad output even costs them until finance asks questions. The uncomfortable truth: an agent that returns unstructured text isn't a system, it's a slot machine. You can't version it, you can't test it, and you definitely can't put it on call rotation with a straight face. That's the problem I wrote Production AI Agents with PydanticAI to solve, not another "here's how LLMs work" book, but a build-along for treating agents like the software they actually are: typed contracts, eval suites in CI, cost observability, canary rollouts, and a runbook someone other than you can follow at 3am. If your agent's output schema is a suggestion rather than a guarantee, this one's for you. 📖 leanpub.com/production-ai-… #PydanticAI #LLMOps #AIEngineering #PythonEngineering

The if statement that appears in every function is a design smell. You've seen it. Maybe you've written it. A defensive check at the top of the function: make sure the field exists, make sure the value is what you expect, return early if it isn't. Then the same check in the next function. And the one after that. Each one reasonable in isolation. Collectively, a signal that the data entering your system has no guaranteed shape, so every function has to defend itself individually. This is what happens when validation is an afterthought. The logic that should live at the boundary gets copied across the codebase instead. Every new function inherits the same boilerplate. Every developer has to remember which fields might be missing. Every code review has to catch the ones they forgot. The fix isn't better discipline. It's better structure. When data is validated once, at the point it enters the system, every function downstream can trust what it receives. The defensive checks disappear. The functions shrink. The logic becomes easier to read, test, and change. This is one of the most consistent improvements teams report after adopting structured data models: not just fewer bugs, but simpler code. Functions that do one thing, because they no longer have to do everyone else's job too. The Practical Pydantic book shows how to apply this pattern across real Python systems: APIs, pipelines, and services where data quality determines code quality. Currently 50% off: 👉 leanpub.com/practical-pyda… Validate once. Trust everywhere. #Python #Pydantic #BackendDevelopment #SoftwareDevelopment #CleanCode

One of the architectural decisions that seems minor early in a project and becomes significant later is how you handle context management in your LLM integrations. Most implementations I see start with the simplest possible approach, build a messages list, append to it, send the whole thing with every call and that works fine until it doesn't. Until the conversation gets long enough that you're sending thousands of tokens of context that the model doesn't need, your latency climbs, your costs climb with it, and you're still not getting better results because most of that context is noise rather than signal. The problem is that context management feels like an optimisation problem when you're building fast, which means it gets deferred to later. In practice it's an architectural decision that touches how you store conversation history, how you retrieve relevant context, how you decide what to include and what to leave out, and how you handle the cases where the context window fills up before the conversation is finished. Retrofitting a good answer to all of those questions onto a system that was built assuming unlimited context is significantly harder than designing for it from the start. The pattern that holds up best in production is treating context as something you actively curate rather than passively accumulate. You decide what goes in based on relevance to the current turn, not based on what happened to be in the conversation history. For most applications that means some combination of a fixed system prompt, a sliding window of recent turns, and retrieved context from whatever the user is actually asking about right now. The exact balance depends on the application but the principle is consistent, context is a resource with a cost and a limit and it should be managed deliberately. Getting this right early is one of those decisions that makes everything else easier. Getting it wrong is one of those decisions that makes everything else slower, more expensive, and harder to change. Context management is one of the architectural gaps that's easiest to miss early and hardest to fix later. If your AI integration is growing and nobody has thought carefully about this yet, it's worth a conversation. Work with Me - developer-service.blog/work-with-me/ #Python #AI #SoftwareEngineering #LLM #Architecture

You shipped an LLM feature last quarter. The demo worked, the stakeholders were happy, and the model output was good enough to put in front of users. Three months in, your bill is climbing, and your board is asking what your gross margin looks like by feature. You open the provider dashboard. It shows you a monthly total. That's it. But here is the uncomfortable question: do you actually know which of your features is profitable? Not "are we under budget this month", but which features, which users, and which prompt shapes are eating your margin - and would you know if one of them was looping at 3am, burning through your runway one token at a time? If you can't answer that with a query, you don't have a monitoring system. You have a credit card statement and a hope. This article is for the ones building an LLM product and want to know what it actually costs you - per feature, per user, per request - without paying for Datadog or wiring up OpenTelemetry. We'll walk through a Python wrapper (Anthropic in the example, the same pattern for any provider), a SQLite cost store, and a small watcher script that pages you via ntfy.sh before a bad night becomes a bad month. developer-service.blog/the-3am-pager-… #Python #LLM #Cost #ntfy #Claude #AI

Your proof-of-concept works on your laptop. The demo got a nod. Then someone called it "production", and now you're the one on call when it mislabels issues, burns through the API budget, or quietly degrades after a "small prompt tweak." That's the exact gap I'm writing about in my new book, Production AI Agents with PydanticAI. This isn't a prompt-engineering book. It's not an LLM intro. It treats agents as software, because that's what they are, and uses Pydantic AI as the framework where typed outputs and dependency injection actually give you the seams to test, version, and operate what other stacks leave loose. You build one system end to end: an open-source maintainer assistant that triages GitHub issues. Along the way you'll work through: → Versioned output contracts and contract tests → Eval suites wired into CI → Observability and cost SLOs → Canary rollouts and human-in-the-loop controls → A runbook that actually survives a 3am page If you've got an agent running in a notebook and a deadline to put it in production this quarter, this is the checklist. 📖 Free sample chapters + full book: leanpub.com/production-ai-… #AgentsAreSoftware #PydanticAI #PythonEngineering #Python #AI

Python looks for variables the same way you'd look for your keys. First you check your pockets. Then the bowl by the door. Then the kitchen counter. Then anywhere in the house. If you find them early, you stop looking. If you don't find them at all, you panic. This is exactly how Python resolves variable names, Local first, then Enclosing, then Global, then Built-in. It stops at the first match. It never skips ahead. Simple concept. But the moment you add nested functions, decorators, or closures to the mix, "simple" gets complicated fast. And when Python finds the wrong set of keys? No error. No warning. Just the wrong value, running your code as if nothing happened. I wrote a free guide that makes the lookup rules impossible to forget: - The LEGB chain explained with clear, practical examples - Why closures "remember" the wrong value and how to fix it - global and nonlocal, what they actually do under the hood - The scope patterns most likely to catch you off guard in production Free: …variable-scope.developer-service.blog #Python #PythonTips #SoftwareEngineering #LearnPython

Practical Pydantic by Nuno Bispo is on sale on Leanpub! Its suggested price is $39.00; get it for $16.24 with this coupon: leanpub.com/practical-pyda… @DevAsService #fastapi #python #apis #computer_programming

Something that doesn't show up in any hiring metric, velocity dashboard, or sprint review is the moment a senior engineer quietly decides that a part of the codebase is too risky to touch without more time than anyone is going to give them. It doesn't get flagged. It doesn't get escalated. It just gets avoided. Features that would require touching that part of the system get scoped around it. Bugs that live near it get fixed at the surface rather than at the root. The system grows around the problem the way a tree grows around a nail — slowly, organically, and in ways that make the original problem progressively harder to address without affecting everything that has grown up around it. In most codebases this happens with legacy components that have been there long enough to accumulate scar tissue. In AI codebases it's happening faster than anyone expected because the original build was fast enough that the scar tissue started accumulating before the feature even launched. Teams are inheriting technical risk on systems that are weeks old rather than years old and the usual signals that something needs attention — age, instability, staff turnover — haven't had time to surface yet. The risk is real even when it's invisible. Especially when it's invisible. The question worth asking your team this week is not whether your AI feature is working. It's whether there are parts of it that nobody wants to be responsible for explaining if something goes wrong. If the answer is yes, that's the conversation to have before it becomes the post-incident review. Work with Me → developer-service.blog/work-with-me/ #AI #CTO #TechLeadership #SoftwareEngineering #Python

Something I've been thinking about lately is the difference between a team that is good at building software and a team that is good at owning software. They're not the same thing and the distinction matters more than it used to. Building is a sprint. There's a clear goal, a deadline, a moment of completion. The feedback loop is tight and the progress is visible. You can feel yourself moving forward. Owning is indefinite. There's no finish line, the goal keeps shifting, and the progress is measured in things that didn't go wrong rather than things that did. It's a fundamentally different relationship with the work and it requires a fundamentally different mindset. The reason this matters now more than it did five years ago is that AI features blur the line between the two in ways that are genuinely new. The model you deployed six months ago is not the same model you're running today even if you haven't touched the code - the underlying behaviour shifts, the edge cases evolve, the things users try to do with it change in ways you didn't anticipate. Owning an AI feature means staying in a relationship with something that is quietly changing underneath you even when you're not looking at it. Most teams are built and incentivised for the sprint. The ownership part gets assigned to whoever is left when the sprint ends, with whatever attention they have left after the next sprint starts. That gap between how teams are structured and what AI systems actually require is where most of the expensive surprises come from. The teams that figure out how to own what they build will be the ones still standing when the sprint mentality runs out of road. Owning an AI system properly starts with understanding exactly what you have. Work with Me - developer-service.blog/work-with-me/ #SoftwareEngineering #AI #TechLeadership #CTO #Python

Your tests pass. Your data assumptions don't. Most Python test suites do a reasonable job of checking behaviour. They verify that functions return the right values, that edge cases are handled, that the happy path works. What they rarely check is whether the data flowing through the system is actually what the system thinks it is. A function that receives a dictionary and accesses data["user"]["email"] can pass every test you write for it - and still fail in production when user is None, or email is missing, or the key is spelled differently than expected. The test passed because the test data was clean. Production data isn't. This is a structural gap, not a coverage gap. Writing more tests won't close it. Changing what you're testing might. When your data layer is built on explicit models with enforced types and constraints, your tests get sharper automatically. Invalid data can't quietly pass through - it fails at the boundary, with a clear error, before it reaches the logic you're trying to test. You spend less time debugging data-related failures and more time testing the behaviour that actually matters. The teams that catch this early stop treating data validation as a runtime concern and start treating it as a design constraint. The result is a test suite that reflects reality - not just the happy path you imagined when you wrote it. Practical Pydantic covers how to build Python systems where data integrity is structural, not assumed. leanpub.com/practical-pyda… Clean tests start with clean data contracts. #EngineeringLeadership #TechStrategy #SoftwareArchitecture #ProductEngineering #SoftwareQuality #Python #Pydantic

Something that took me longer to appreciate than it should have is how much the structure of your prompt affects the maintainability of your AI integration. Not just the quality of the output, but how easy the whole system is to work with six months after it was written. Most prompts I see in production codebases are built one of two ways. Either they're a single large string assembled inline at the point of use, with variables interpolated directly into the text using f-strings, or they're scattered across the codebase in ways that make it genuinely difficult to understand what the model is actually being asked to do at any given point. Both approaches work well enough when you're the person who wrote them and the context is fresh. They become a maintenance problem the moment someone else needs to understand, modify, or debug them, which in a production system is a matter of when, not if. The pattern I've settled on is treating prompts as first class artifacts in the codebase. They live in their own module, they're versioned explicitly, they have clear names that describe their purpose, and they're constructed through functions that make the variable parts obvious and the static parts stable. When a prompt needs to change, because the model behaviour shifted, or the requirements evolved, or a bug was found, there's one place to go and one thing to change. The rest of the system doesn't need to know it happened. It sounds like a small thing. In a codebase with three prompts it probably is. In a codebase with thirty prompts running across multiple features in production it's the difference between a system you can confidently modify and one you're afraid to touch. Prompt management is one of the structural gaps I look for in every AI codebase review. If yours has grown beyond a few prompts and nobody has thought about this yet, it's worth addressing before it becomes a refactor. Work with Me - developer-service.blog/work-with-me/ #Python #AI #SoftwareEngineering #LLM #PydanticAI

"I don't need to study Python scope. I just write code and fix errors when they show up". I hear a version of this a lot. And I understand the instinct - scope feels like theory until it isn't. Here's the problem: scope bugs don't always throw errors. They just quietly return the wrong value. No traceback. No red flag. Just output that's subtly, confidently incorrect - the kind of bug that survives code review and ships to production. The "fix errors as they show up" approach doesn't work here, because there's often no error to catch. This is exactly why I wrote a free Python Scope Guide - to give developers the mental model before the bug happens, not after. - The LEGB rule, explained with precision - Closures and the late binding trap - global and nonlocal - mechanics and when they signal a problem - Real scope bugs and how to spot them before they ship Understanding this once saves you from relearning it the hard way, repeatedly. …variable-scope.developer-service.blog #Python #SoftwareEngineering #PythonDevelopment #CodeQuality