Northwind Helpline: the right model type per decision

Northwind Helpline: the right model type per decision A workflow diagram generated by Archify. 01 / Customer 02 / Decision model: Jev 03 / Language model Sort Hand to a person Draft Check and route Customer message · free text from a customer · Customer › Sort Customer message free text from a customer Reply sent · grounded in a source · Customer › Check and route Reply sent grounded in a source Intake · Jev decision: topic + urgency · Decision model: Jev › Sort Intake Jev decision: topic + urgency Escalation Router · Jev decision: which queue · Decision model: Jev › Hand to a person Escalation Router Jev decision: which queue Grounding Checker · Jev decision: grounded? · Decision model: Jev › Check and route Grounding Checker Jev decision: grounded? Answer · Language model: drafts reply · Language model › Draft Answer Language model: drafts reply draft not grounded grounded topic, urgency if not clearly decided Legend User UI Agent logic Policy External system

Why a decision model for three of them

  • • Intake picks 1 of 5 topics and scores urgency
  • • Grounding Checker answers one yes or no question
  • • Escalation Router picks 1 of 3 queues

Why the fourth stays a language model

  • • Answer writes prose for a customer to read
  • • It is also the only agent that can drift from its source
  • • That drift is what the Grounding Checker exists to catch

What the live calls showed

  • • Jev returns a confidence for choice and score, and none for boolean
  • • Below 60% confident, a decision goes to a person instead of onward
  • • The grounding bar is the same 60%, set from 6 labeled drafts, not from one