← All work
Generative AIMulti-agentVoice

AI ordering assistant

An AI that takes orders by voice and text, rebuilt when the first version fell short.

Role
Product Owner & Business Analyst
Timeline
May 2026 – Sep 2026
Context
Capgemini · global QSR brand
Status
Client work

Problem

Customers should be able to order by talking or typing, with the assistant replying in text and voice. The first version, an agent built natively on each platform, was slow, misread prompts and sometimes suggested products unrelated to the request.

Outcome

The assistant was redesigned as a single REST API where a main agent hands work to cart, recommendation and order agents. The model fills a fixed query schema that runs against the menu database, so it can only return items that exist.

  • 1REST API replacing per-platform agents
  • 3sub-agents: cart, recommendation, order
  • < 2 mingoal to order and pay
Lifecycle6/10
  1. Discover
  2. Understand
  3. Define
  4. Ideate
  5. Prototype
  6. Build
  7. Launch
  8. Adopt
  9. Measure
  10. Improve

Discover · Understand · Define · Ideate · Prototype · Build

Client work: described without screenshots, client names or confidential figures.

How it works

  1. 01

    Customer speaks or types

    "Two burgers, no onions, and something cold."

  2. 02

    Main agent

    Understands the request and hands each part to the right sub-agent.

  3. 03

    Schema, not guesswork

    The model fills a fixed query schema; the database returns real menu items.

  4. 04

    Reply in text and voice

    The answer streams back, with a fallback if the item isn't available.

The goal

Let a customer order by voice or text and get answers in both, from choosing items to paying, in under two minutes.

Version one, and why it fell short

The first approach was an agent written in Kotlin with native screens on Android and iOS. It worked, but not well enough: responses were slow, the model often misread what the customer asked for, and it sometimes suggested products that had nothing to do with the request.

The redesign

  • One conversational REST API instead of separate per-platform logic, with a self-describing response format so the app can show new kinds of responses without a new release.
  • Multi-agent orchestration: a main agent reads each prompt and delegates to cart, recommendation and order sub-agents.
  • Grounded in the live menu: I defined the menu taxonomy, business rules and a constrained query schema. The model fills the schema from the customer’s words, and the schema runs against the vector database to fetch exact products. It cannot invent an item.
  • Edge cases designed up front: what the assistant says when an item is unavailable, out of stock or ambiguous.
  • Voice in three tiers: streamed AI speech over WebSocket, pre-recorded clips, and a local fallback, with automatic failover.
  • Bias checks, fairness guardrails and a model evaluation and monitoring approach were part of the backlog, not an afterthought.

How I worked

I prioritised and refined the backlog with engineering, data and UX, wrote the user stories and acceptance criteria, accepted the AI APIs in Postman and tracked capacity and coverage in Power BI.

Stack and tools

Gemini LiveRAGVector databaseRESTWebSocketPostmanPower BI