AI features inside products
AI built into ordinary apps as buttons and features, where the app's own server calls a model API behind the scenes.
Read first: Retrieval-augmented generation (RAG).
D1 · StoryFor young children and total beginners
Lots of apps now have an AI button: summarise this, tidy my writing, find similar photos.
There may be no chat box at all. When you press the button, the app quietly asks a model and shows you the answer.
What happens when you press the AI button
- You: press the button
- The app: on your phone
- The app's computer: far away
- The model: writes the answer
Like: A café that buys its bread from a bakery: you only see the plate.
D2 · PictureFor curious beginners
The usual pattern: the app's server builds a prompt from a template plus your data (often found with retrieval), calls a model over an HTTP API, and gets text or structured JSON back to show in the interface.
The API key stays on the server, never in the web page or phone app. Many products route calls through a gateway that can switch models and providers, using a small, cheap model for simple jobs and a larger one for hard ones.
Model output is treated as untrusted input: it is checked before it is shown, stored or used to trigger actions, and usage is logged and limited to control cost.
Like: A café that buys its bread from a bakery.
Where it breaks: A bakery makes the same loaf every day. A model can return a different or wrong answer, or one steered by text hidden in your document, so the app must check what comes back.
D5 · FrontierFor experts
Open problems: features that demo well but are rarely used, prompt injection through user content that flows into prompts (OWASP lists it first among LLM app risks), checking output quality at scale, and costs that rise with every use.