I recently spent a couple of weeks building-out a LLM-driven product concept (which I’ll be writing more about soon over on Mozilla’s blog). The experience triggered a lot of thoughts and feelings. Among them is the realization that the way we build software products is about to change irrevocably.
For as long as people have been writing software, it’s been a largely predictable undertaking. Someone like a product owner figures out what the product should do. Someone like a designer develops that definition into a user experience. And someone like an engineer turns it all into functioning software.
The resulting product is usually deterministic. It does what it is supposed to do and behaves as specified. When it doesn’t, we call that a bug.
Products built around AI are different because their behaviors aren’t nearly as predictable. Each level of the solution stack requires you to make decisions — e.g. your choice of model, your approach to prompt engineering, the settings and characteristics of your runtime environment, the way your interface manifests both the user’s inputs and the model’s outputs — and those decisions then interact with each other. You’re building a complex system that is more than the sum of its parts.
The resulting product isn’t deterministic, it’s emergent. It doesn’t behave the way you expected, only now that’s not a bug, it’s a feature.
In this new world, the traditional tools and mindsets of product development fail us. Writing a detailed spec or designing a fully-realized experience doesn’t make sense in a world ruled by emergence because you simply don’t have the same degree of prediction and control. Mapping out a shipping schedule becomes an exercise in futility when every build your team puts out can exhibit wild new behaviors that impact both your thinking and your planning.
We’re no longer professionals working from a manual. We’re kids playing with home chemistry kits, mixing random vials of liquid and waiting for a puff of smoke. We’re in a new universe and we don’t know the physics yet.
It’s all rather exciting.
(Image credit: Airwolfhound, CC BY-SA 2.0 https://creativecommons.org/licenses/by-sa/2.0, via Wikimedia Commons)