Define the role as a contract
The input is not just a job title. It includes the desired business outcome, systems, policies, authority, examples, independent success criteria and the company’s quality, cost and speed priorities. An existing agent can enter as the baseline.
Construct complete specialists
Each candidate binds a model, instructions, context, tools, memory policy, allowed authority, escalation behaviour, independent verifier, cost limits and latency limits. The configuration being tested is the whole worker, not only its prompt.
Reject invalid designs early
Wrong tools, missing context, mismatched verification, excessive authority, unsafe memory scope, impossible budgets and invalid escalation rules are hard failures. Safety and authority cannot be averaged away by a high performance score.
Use meaningful baselines
DAS does not merely rank candidates it generated itself. The bounded campaigns compare them with strong general agents, ordinary manually configured agents and expert configurations where those alternatives can be run fairly.
Freeze the exam
Candidate and baseline identities are fixed before prospective unseen cases are released. Development, validation, adversarial, unseen and repeatability evidence stay separate so a good-looking result cannot be created through post-hoc score chasing.
Check the world, not the claim
A separate external-state verifier checks whether the intended order, ticket, CRM record or other business result exists and whether forbidden side effects occurred. The candidate is not allowed to grade itself.
Preserve failures
Missing outcomes, incorrect effects, handoffs, tool use, latency and cost remain in the evidence. A narrow repair may address a missing outcome. Incorrect or unsafe effects stop instead of disappearing behind an automatic retry.
Select or retain
A generated specialist activates only when it wins under the decision rule fixed in advance. Otherwise DAS recommends keeping the existing setup. Retaining the current agent is a valid and sometimes correct output.
Connect bounded real tools
Current local adapter work can import an explicitly selected safe subset of OpenAPI operations or a pinned customer-local Model Context Protocol tool list. Exact tool allowlists, authority, uncertain-write reconciliation and external-state visibility remain attached.