Classification stops too early
A classifier answers “what kind of request is this?” An agent router must answer a harder question: “which combination of model, tool, policy, and context can complete it?” That shift changes both the output schema and the evaluation strategy.
Routing should remain an explicit system boundary. Even when an LLM proposes the route, application code should validate permissions, required entities, and supported tools before execution.
A five-stage pipeline
- Understand the task and detect whether it contains multiple goals.
- Extract entities needed to execute each goal.
- Resolve missing information or apply declared defaults.
- Select an agent or tool based on capability and policy.
- Record confidence and the reason for the decision.
query -> task analysis -> entities -> policy gate -> agent/toolThe route should be inspectable. “Weather Agent” is useful; “Weather Agent because the query requests a forecast for Beijing tomorrow” is much easier to debug.
Routing is a product surface
Bad routing feels like a model quality problem to users. In reality, it often comes from vague capability definitions, overlapping tools, or missing fallback behavior. Each route needs a clear contract: accepted inputs, expected outputs, cost, latency, and failure semantics.
Evaluation beyond accuracy
Top-one intent accuracy is not enough. Measure route completion rate, unnecessary tool calls, clarification quality, latency, and the cost of false positives. For multi-intent queries, evaluate whether the system preserved dependencies between tasks rather than merely detecting both labels.
The best router is not the cleverest classifier. It is the smallest decision system that consistently sends work to the right capability.