F2-CUSTOMER-RAG-AGENT2026 · Applied AI · GenAI

Customer-interaction RAG agent

A confidence-gated customer-operations agent that combines policy citations, scoped synthetic ERP lookups, prompt-injection handling, and inspectable human escalation.

Fictional customer-operations queue showing a grounded answer and human-review boundary

Decision and workflow

LinenFlow is a fictional commercial linen provider. The system must decide whether a customer-operations answer is authorised, evidence-backed, and complete enough to send. It can explain approved policy and read scoped synthetic records, but it cannot change a quote, order, route, credit, or entitlement.

System approach

The MVP compares lexical, dense, and hybrid retrieval before wrapping the selected retriever in a constrained single-agent workflow. Retrieval is filtered by persona scope, tool arguments are validated, citations point back to versioned policy evidence, and a confidence gate routes uncertainty, conflicts, access violations, and authority requests to a human.

Baseline and evaluation

The offline evaluation contains 60 attributed synthetic cases across sales, fulfilment, service, access, clarification, conflict, and adversarial behavior. It records retrieval hit rate, workflow-status accuracy, citation correctness, tool selection, escalation accuracy, coverage, latency, and safe refusal.

Operational boundary

The agent is useful because it knows when not to guess. The project separates policy evidence from operational facts, labels the fictional business clearly, and treats prompt injection, stale policy, unavailable tools, and missing identifiers as first-class failure cases.

Reproducibility

The repository includes versioned policy fixtures, a scoped SQLite tool layer, an evaluation set, offline traces, monitoring reports, and a Streamlit presentation surface. All results shown here are portfolio evidence rather than observed production outcomes.

Selected evidence

Each visual answers a specific question about system behavior. Read the caption and limitation together.

Summary of retrieval and workflow outcomes across tested methods
Workflow outcomeThe agent is evaluated on whether it chooses the right workflow status, not only whether it generates fluent text. Limitation: The evaluation set and routing are deterministic and synthetic.
Retrieval hit at five comparison across lexical, dense, and hybrid methods
Retrieval comparisonRetrieval quality is treated as a measurable part of the system contract. Limitation: The corpus is deliberately small.
Tool and escalation accuracy view for the customer operations agent
Tools and escalationAuthority boundaries turn uncertain or consequential requests into inspectable handoffs. Limitation: Tool records and escalations are simulated.
Simulated monitoring dashboard showing stale-policy and unavailable-tool incidents
MonitoringThe monitoring story includes stale evidence and dependency failure, not only happy-path quality. Limitation: Both incidents are injected simulations.
Architecture diagram showing retrieval, synthetic records, tools, confidence gate, and human handoff
ArchitectureEvidence, operational records, permissions, and the confidence gate remain separate concerns. Limitation: This is a portfolio design, not a deployed production system.