DEMO / Automation

Agent Workbench · Reviewed automation

A concept study of evidence, constrained tools and human approval for agent tasks. It proposes evaluation boundaries rather than claiming a working integration.

Concept demo: no live model, enterprise knowledge base or client system is connected. Accuracy, savings and autonomous execution are not established.

Concept illustration showing agent task steps, evidence and an approval panel
Concept illustration

Background

A concept proposal for evidence, human approval and constrained tools, rather than a deployed agent.

Challenges

  • Keep evidence, tool permissions and human decisions visible instead of treating a fluent response as proof of successful work.

Solution

Separate retrieval, recommendation, approval and execution, with constrained permissions, retry limits, cancellation and human takeover.

Capabilities and scope

  • Proposed: separate source evidence from answers
  • Proposed: approval checks before tool execution
  • Proposed: bounded retries, cancellation and human takeover

Architecture

Concept layers: a TypeScript workspace, FastAPI task endpoints, Python retrieval and tool adapters, and Redis coordination. No live model, knowledge base or business tool is connected.

Results and evidence

  • Task permission and human review boundaries are documented
  • No domain evaluation, accuracy measurement or efficiency gain is established

Separate the answer from the task

A fluent response does not establish that work was completed. This concept study proposes separate records for preparation, retrieval, recommendation, human approval and execution. The useful questions are when a person must intervene and what evidence they need to continue safely.

The cover illustrates an interface direction. There is no connected model, client knowledge base or external tool, and there are no measured accuracy or efficiency results.

Before data enters the workflow

Identify which sources may be read, which may be sent to a provider and which must stay in a specified environment. A prompt is not an access-control mechanism: retrieval services and tool endpoints need independent permission checks.

Updates and removals also need ownership. Define when indexes refresh and what happens to old citations. An answer should not present withdrawn material as current evidence. Insufficient evidence must be allowed to produce uncertainty rather than a fabricated completion.

A recommendation is not authorization

Retrieval can prepare a summary and citations. A recommendation can outline possible next steps. Changing records, sending messages or spending money requires approval conditions matched to the action's consequences.

The approval object should be an inspectable action list with targets, bounded parameters and expected effects. Approval of one proposal must not silently authorize every later call. Changed parameters may require another decision.

Failure, retries and cancellation

The proposed state model distinguishes network failures, provider limits, business rejection and missing permissions. Only recoverable conditions enter bounded retries. If the business record changed, the task should read current facts before repeating an action.

Cancellation spans queued, running and result-persistence stages. A late success must not overwrite an accepted cancellation. Human takeover should show completed actions and remaining work rather than a single unhelpful failure label.

What evaluation needs to inspect

  • Retrieval stays within sources the user may access.
  • Citations support the specific factual statements made.
  • Missing, contradictory and outdated evidence can be recognized.
  • Unapproved tool calls are denied by the receiving service.
  • Repetition, timeouts and cancellation preserve consistent task state.

Implementation would require authorized, redacted domain examples. Retrieval quality, answer quality and action correctness should be evaluated separately. Model selection, costs and provider retention remain independent decisions. Commerce Lab provides a related discussion when tools affect order states or amounts.