Stage 11 — Feedback Loop

File: agents/sdlc/11-feedback-loop.md · Model: Sonnet · Tools: Bash, Read, Write

Purpose

The Feedback Loop agent is the TRUE final agent in the pipeline — it runs after Agent 10 (Summary) and reviews every contract the other ten stages produced. It does not add new functional work; it cross-references FR-XXX / US-XXX / TC-XXX identifiers end-to-end (requirements → stories → code → tests → deployment) and reports where the chain breaks. A CRITICAL finding here means something real would ship broken; a WARNING is a documented, accepted risk.

This stage also authors goal_evaluation — the structured, evidence-only object the bounded goal loop (rstack-agents pipeline loop, BLE-4) reads to decide PASS / RETRY / ASK_USER / BLOCK. The harness evaluator (src/core/harness/goal-check.js) never calls a model — it only consumes a criterion when every listed evidence path actually exists on disk.

Core principle

A consistent score of 90+ means the pipeline can be trusted. Below 70 means significant rework is required before delivery. The agent reports the real number — it does not normalize gaps.

It also applies a maintenance discipline: it never recommends a remediation that touches existing (baseline) code without naming what to read first, and it never proposes patching a module issue-by-issue once three or more corrective findings cluster on the same file, service, or stage artifact — that’s a bug swarm, and the remediation is one preventive rewrite of the module, not N patches.

How it works

Inputs it reads

The agent reads every canonical stage artifact it can find, with legacy paths as fallback:

Stage	Canonical path
01 Transcript	`artifacts/stages/01-transcript/transcript.json`
02 Requirements	`artifacts/stages/02-requirements/requirement_spec.json`
03 Documentation	`artifacts/stages/03-documentation/documentation.json`
04 Planning	`artifacts/stages/04-planning/plan.json`
05 Jira	`artifacts/stages/05-jira/jira_tickets.json`
06 Architecture	`artifacts/stages/06-architecture/system_design.json`
07 Code	`artifacts/stages/07-code/code_report.json`
08 Testing	`artifacts/stages/08-testing/test_report.json`
09 Deployment	`artifacts/stages/09-deployment/deployment_report.json`
10 Summary	`artifacts/stages/10-summary/summary.json`

Optional stage outputs (12-security-threat-model, 13-compliance-checker, and a cost-estimation artifact if that agent ran) are checked too, but their absence is never treated as an error. Graceful partial read is a hard rule, not a convenience: a missing contract is logged as a WARNING (“agent may not have run”), a malformed one as a CRITICAL (“MALFORMED JSON”), and the agent must never crash on either — it analyzes whatever it can read and states its analysis scope explicitly.

Adopted-run behavior (brownfield)

If the run manifest has "mode": "adopt" and stage artifacts carry "source": "brownfield-adoption", the agent treats harvested stages as DONE-with-evidence: it reviews new work against those baselines rather than recommending they be regenerated, doesn’t log NOT-FOUND warnings for stages adopt deliberately skipped, and scores traceability for the new work separately from thinner baseline coverage so a legitimately thin adopted spec doesn’t crater the score for good feature work in the same run.

The eight analysis tasks

Requirements traceability — every FR must reach a story, code, and a test; a story with zero tests is CRITICAL, an FR with zero stories is CRITICAL.
Architecture-to-code consistency — every API endpoint / DB table / service in system_design.json must have a corresponding artifact in the code output.
Sprint capacity validation — flags developers over 15 (CRITICAL) or 13 (WARNING) story points per sprint, blocking stories scheduled after what they block, unassigned stories.
Security & compliance gap analysis — every security/compliance NFR must trace to an architecture control and a test case; HIGH/CRITICAL threats from the optional threat model must have mitigations in code.
Documentation completeness — BRD/FRD/SOW/HLD/test plan coverage against their source contracts.
Cross-contract version consistency — contract_version alignment, valid ISO 8601 timestamps, no dangling FR-XXX/US-XXX/TC-XXX references.
Remediation plan generation — every CRITICAL/WARNING gets an owning agent, an estimated effort, and a required maintenance_category (perfective | adaptive | corrective | preventive) — an unclassified finding is treated as an incomplete finding.
Structured goal evaluation — the goal_evaluation object described below.

Consistency score

Starts at 100; each CRITICAL is −10, each WARNING −3, each INFO −0.5, floor 0:

Score	Health
90–100	`HEALTHY`
70–89	`NEEDS_ATTENTION`
0–69	`CRITICAL_GAPS`

The `goal_evaluation` contract

If $RUN_BASE/goal.json declares an active goal, the agent reads the run’s most recent loop_iteration_started event from events.jsonl and stamps every criterion with that exact iteration value — an evaluation with a missing or stale iteration is ignored by the harness, and one stamped ahead of the current iteration is rejected outright and routed to ASK_USER. For each criterion in goal.json the agent emits a result (met | not_met | unknown) that the harness will only trust if every path in that criterion’s evidence[] actually exists in the run directory — an unverifiable claim always falls back to unknown, never a guess. On not_met, the criterion’s recommended_rerun_stages[] must include 11-feedback-loop itself, or the next loop iteration has nothing to re-evaluate and the loop stalls. A human- or host-authored goal-verdict.json for the same criterion outranks the agent’s own evaluation. If no goal.json exists, the agent still emits a goal_evaluation block (evaluated against the harness’s default goal) with an empty criteria[].

Outputs

Canonical stage output, then a legacy compatibility copy:

$RUN_BASE/artifacts/stages/11-feedback-loop/feedback.json      ← canonical
$RUN_BASE/artifacts/feedback/consistency_report.json           ← legacy copy
$RUN_BASE/artifacts/stages/11-feedback-loop/REMEDIATION_PLAN.md ← canonical
$RUN_BASE/artifacts/feedback/REMEDIATION_PLAN.md                ← legacy copy

feedback.json shape (abridged):

{
  "contract_version": "1.1",
  "produced_by": "feedback_loop_agent",
  "pipeline_complete": true,
  "analysis_scope": {
    "contracts_analyzed": ["..."],
    "contracts_missing": ["..."],
    "contracts_malformed": ["..."]
  },
  "traceability_matrix": {
    "total_requirements": 0,
    "requirements_fully_traced": 0,
    "coverage_percentage": 0.0
  },
  "issues": [
    {
      "id": "FBK-001",
      "severity": "CRITICAL",
      "category": "traceability",
      "remediation": {
        "action": "...",
        "agent_to_rerun": "07-code",
        "maintenance_category": "corrective"
      }
    }
  ],
  "summary": {
    "overall_consistency_score": 0.0,
    "pipeline_health": "HEALTHY|NEEDS_ATTENTION|CRITICAL_GAPS"
  },
  "goal_evaluation": {
    "goal_id": "pipeline-complete",
    "iteration": 1,
    "status": "PASS|RETRY|ASK_USER|BLOCK",
    "criteria": [
      { "criterion_id": "...", "result": "met|not_met|unknown", "evidence": ["..."] }
    ]
  },
  "pipeline_status": "REVIEWED_AND_COMPLETE"
}

The agent also writes the standard Task Contract pair — builder.json and, because it acts as the pipeline reviewer, a full validation.json (status, checks[] — one per cross-reference rule, issues[], retry_recommendation) — into $RUN_BASE/tasks/<task_id>/.

pipeline_complete: true in this contract is the pipeline’s own termination signal. The agent never triggers a further stage — it is documented as the absolute end of the pipeline unless a goal loop reopens it via recommended_rerun_stages.

Where it sits in the pipeline

Stage 11 runs after Stage 10 — Summary and is the last stage in the base 15-stage sequence before the optional stages (12-security-threat-model, 13-compliance-checker, 14-cost-estimation). Its goal_evaluation output is what closes the loop: a RETRY verdict with recommended_rerun_stages is exactly what rstack-agents pipeline loop uses to reset those stages and re-run the pipeline within its bounded iteration budget.

Try it

Inspect the latest run’s consistency verdict directly:

RUN_BASE="$(ls -td .rstack/runs/*/ | head -1)"
cat "$RUN_BASE/artifacts/stages/11-feedback-loop/feedback.json" | python3 -m json.tool \
  | grep -E '"overall_consistency_score|"pipeline_health|"status"'

Or drive the goal loop that consumes it end-to-end:

rstack-agents pipeline loop --run-id <run_id>

Overview

Getting Started

Core Concepts

The Pipeline

Business Hub

Adapters

Security & Compliance

Roadmap & Research

Stage 11 — Feedback Loop

Purpose

Core principle

How it works

Inputs it reads

Adopted-run behavior (brownfield)

The eight analysis tasks

Consistency score

The `goal_evaluation` contract

Outputs

Where it sits in the pipeline

Try it

​Purpose

​Core principle

​How it works

​Inputs it reads

​Adopted-run behavior (brownfield)

​The eight analysis tasks

​Consistency score

​The goal_evaluation contract

​Outputs

​Where it sits in the pipeline

​Try it

​Related

Purpose

Core principle

How it works

Inputs it reads

Adopted-run behavior (brownfield)

The eight analysis tasks

Consistency score

The `goal_evaluation` contract

Outputs

Where it sits in the pipeline

Try it

Related