The key idea
Describe a task and its expected output before choosing technology. Test the most uncertain decision first.
“We need an AI app” can start a conversation, but it is not yet a scope. First identify a task, the person completing it and the point where the current process gets stuck.
Imagine a fictional company receiving inquiries by email and copying them into a spreadsheet. A form and rules might solve the problem. A web application might be needed to let people check status. AI might help interpret messages written in many different ways. These are separate decisions that can be tested separately.
Map the task before drawing screens
Describe the journey with verbs: receive an inquiry, check information, assign an owner, respond and check status. Add who handles each step, the information required and what happens when something is missing.
Instead of starting with “dashboard, chatbot and login,” ask what decision each element enables. An unused dashboard adds maintenance. Mandatory accounts can prevent someone from finishing a simple inquiry.
Before: “An AI app to handle everything.” After: “A form collects five fields; a rule proposes an inquiry category; a person confirms the owner before responding.” The revised brief defines a testable first prototype.
Distinguish four starting points
- Explain an offer and receive inquiries: start with a website or landing page and form. Expand when people need to return to their own information.
- Complete a recurring task with data and status: assess a web application. Check whether a specific device capability proves necessary.
- Use device features in a particular working context: assess a mobile app and test critical functions on target devices and browsers.
- Move information between tools: start with integration and rules. Assess AI when variable language becomes too complex for maintainable rules.
This is a scoping framework, not a rigid taxonomy. Websites can contain application functions. They can also offer PWA installation in supporting browsers. Support varies by environment, so test critical functions on the actual devices before deciding. MDN PWA installation documentation.
Decide whether rules are enough
If a form asks someone to select “campaign, website or support,” a rule can assign the category. If people write freely about “a page for a trade fair with bookings,” classification may require context. That creates a bounded reason to assess AI against the manual process.
Begin with a limited decision: suggest a category, extract fields or prepare a draft. Keep the original message and a correction route. The system does not need permission to send responses or change business records for you to find out whether it helps prepare work.
“Insufficient information” is a useful outcome. A message combining a complaint with a sales inquiry should be able to enter human review. Missing deadlines and budgets should remain missing.
Build a prototype that answers a question
For the inquiry example, the first journey could include fictional inputs, a suggested category, missing information, human review and an editable draft. Final confirmation should be explicit.
Before building it, calculate the time you could recover from monthly volume, current task time, the eligible share and the review minutes remaining per eligible case. The result compares work capacity; it does not classify messages or establish actual savings. An integration needs observed cases to validate those inputs.
A later integration needs agreed sources, permissions, provider data flows, error records and an operator. Development and operation are different costs. Hosting, maintenance and service usage depend on the actual scope and workload.
Define what working means
- Task completion: completed tasks / observed attempts, using one completion criterion.
- Time per task: start to confirmation, including review. Report median and sample size.
- Category correctness: matches with a human-reviewed reference label / evaluated cases.
- Human review rate: cases routed to review / processed cases.
- Cost per accepted task: period operating cost / accepted outputs in that period.
Routing a case to review may be the correct result. Keep a separate count of errors that reach confirmation. Evaluate classification and draft quality separately: a correct category does not establish that the response text is appropriate.
Include normal, incomplete, ambiguous and out-of-scope inputs. Add messages containing instructions that attempt to override the workflow. Defining a task, examples and evaluation criteria before comparing versions follows established evaluation practice. OpenAI evaluation guidance.
Compare the full processes
Observe the manual task using the same types of cases the prototype will handle. Record time, errors and uncertainties. Then observe the assisted journey, including corrections. Comparing complete manual work with an unreviewed automatic output understates the work still required.
Early sessions expose confusing steps. They do not establish monthly savings or a company-wide improvement percentage. Keep the examples and record what changed between versions so the next comparison is interpretable.
Finish with a decision
The right next step might be a clearer landing page, a small dashboard, a rules-based integration or an AI evaluation. If the difficult part is still deciding who responds, a larger app will not settle that operational question.
Bring a task, a sample input, the expected output, one case that should stop and the person responsible for review. Then we can scope a web product or app or assess an AI automation.