school-core Architecture Review

2026-06-14 · Agent School — school-core

module seam leakage deep module

1. Replace the static toy curriculum with a real codebase task adapter

Strong local-substitutable
autonomous_loop.py:58-197 — CURRICULUM dict
autonomous_loop.py:215-281 — _pick_task_for_agent()
issue_bridge.py:174-267 — bridge_issues()
github_fetcher.py:24-80 — fetch_issues()

BEFORE

flowchart TD
  S[AutonomousScheduler] -->|"random.choice(CURRICULUM)"| T["Toy prompt: 'Write is_palindrome()'"]
  T --> D[run_task]
  D --> R["Agent produces trivial code"]
  R -->|"score=70 (hardcoded)"| SC[ScoreStore]
  SC -->|"EMA: 70 → 70 → 70..."| G["Gate: easy → easy → easy"]
  classDef leak stroke:#dc2626,stroke-width:2px;
  class T,R leak
              

Static toy dict. Same 30 prompts forever. Agents plateau at "easy" because the curriculum never scales.

AFTER

flowchart TD
  GF[github_fetcher] -->|"real issues"| T[triage_classifier]
  T -->|"ready-for-agent"| IB[issue_bridge]
  IB -->|"issue body as prompt"| D[run_task]
  D --> R["Agent reads codebase, writes real fix"]
  R --> V[verify_task_output]
  V -->|"score=0-100"| SC[ScoreStore]
  SC -->|"EMA converges on true skill"| G["Gate: easy → medium → hard"]
  classDef deep background:#1e293b,color:#fff;
  class GF,T,IB,V deep
              

Real GitHub issues drive task selection. Verification scores actual output quality. Gates reflect real capability.

Problem: The CURRICULUM dict (autonomous_loop.py:58-197) is 30 static toy problems. Agents have run 50+ rounds producing is_palindrome, factorial, and add(a, b). The EMA score converges to ~70 because every task is trivial and the hardcoded score was 70. The gate system is decorative — no agent ever crosses a gate from real capability growth.

Solution: The issue_bridge already provides real-task drive (GitHub issues). The curriculum should be a fallback, not the primary path. Wire the bridge as the default task source; keep curriculum only for warm-up or when no issues are available. The real fix: agents need to read the codebase before writing code — the issue prompt should include repo context (file paths, existing code), not just the issue title.

  • Agents produce real fixes, not toy functions
  • Verification scores reflect actual output quality
  • Gate crossings become meaningful capability signals
  • Delete 140 lines of static CURRICULUM dict

2. Delete routing.py — routing logic is duplicated in director.run_task

Strong in-process
routing.py:18-76 — route_task() — never imported
director.py:228-247 — run_task() candidate selection — duplicates routing.py

BEFORE

flowchart LR
  subgraph "routing.py (DEAD)"
    RT[route_task] -->|"qualifying_agents"| R1[agent]
  end
  subgraph "director.py (LIVE)"
    D[run_task] -->|"qualifying_agents"| R2[agent]
    R2 -->|_try_agent| EX[executor.call_model]
  end
  classDef leak stroke:#dc2626,stroke-width:2px;
  class RT leak
              

Two sources of truth. routing.py is dead code — zero imports anywhere.

AFTER

flowchart LR
  D[run_task] -->|"qualifying_agents"| C["sorted candidates"]
  C -->|"try in score order"| EX[executor.call_model]
  EX -->|"success → response"| V[verify_task_output]
  classDef deep background:#1e293b,color:#fff;
  class D,V deep
              

Single source of truth in director.run_task. Delete routing.py.

Problem: routing.py defines route_task() (lines 18-76) with candidate selection and gate logic. director.run_task (lines 228-247) implements the same logic inline. routing.py is never imported by any module. This is dead code that creates confusion about where routing decisions actually live.

Solution: Delete routing.py. The routing logic in director.run_task is the real implementation — it's already wired into both the autonomous loop and the issue bridge. Keeping two copies guarantees future divergence.

  • Delete routing.py (76 lines of dead code)
  • Single source of truth for agent selection
  • Zero risk of routing logic diverging

3. Give the verifier codebase context — right now it scores blind

Worth exploring ports & adapters
issue_bridge.py:103-172 — verify_task_output()
director.py:203-206 — enrich_prompt() (vault context)
context_orchestrator.py — enrich_prompt()

BEFORE

sequenceDiagram
  participant IB as issue_bridge
  participant D as director
  participant V as verifier
  participant E as executor

  D->>E: run_task(prompt + vault_context)
  E-->>D: response
  IB->>V: verify(prompt, response)
  Note over V: NO vault context
  V-->>IB: score=90 (everything looks "comprehensive")
              

Verifier sees only the issue text + agent response. No repo structure, no existing code, no architecture. Gives 90 to "Step 1: Identify the Error Source" because it looks structured.

AFTER

sequenceDiagram
  participant IB as issue_bridge
  participant CO as context_orchestrator
  participant V as verifier
  participant SC as scoring

  IB->>CO: enrich_prompt(domain, issue_body)
  CO-->>IB: vault_context + repo_tree
  IB->>V: verify(prompt, response, context)
  Note over V: Sees repo structure + existing code
  V-->>SC: score=30 (response ignores codebase)
              

Verifier gets same vault context as the agent. Can check: "Does the response reference actual files? Does it match the codebase's patterns?"

Problem: director.run_task enriches prompts with vault context via context_orchestrator.enrich_prompt() (line 204). But verify_task_output in issue_bridge sends the raw prompt to the verifier with no context. The verifier has no idea what the codebase looks like, so it scores based on surface structure — bullet points and confident language get high scores regardless of substance.

Solution: Pass the same vault context to the verifier. Add a context parameter to verify_task_output and include it in the verification prompt template. The verifier should see: repo tree, relevant file paths, the agent's system prompt, and the original issue — then judge whether the response actually addresses the real codebase.

  • Verifier scores substance over structure
  • Hallucinated "Step 1: Identify..." gets low scores
  • Responses referencing real files get high scores

4. Wire triage_classifier into issue_bridge — currently it's a dead-end module

Worth exploring in-process
triage_classifier.py:19-72 — classify_issue()
director.py:139-142 — triage_issue() — never called by issue_bridge
issue_bridge.py:188-210 — bridge_issues() — assumes ready-for-agent

Problem: triage_classifier.py implements a rule-based classifier (label matching, keyword heuristics, body-length analysis) that categorizes issues as bug/enhancement and assigns a workflow state. But issue_bridge.py never calls it — line 188 fetches issues and line 196 iterates them, but the triage step is skipped entirely. The bridge assumes every fetched issue is already ready-for-agent. The classifier only exists as a helper in director.triage_issue() (line 139) which nothing calls.

Solution: Add a triage step in bridge_issues() before dispatch. Call classify_issue(issue["title"], issue["labels"], issue.get("body", "")) and skip issues that return needs-info or needs-triage. This makes the pipeline self-healing — new issues get classified automatically instead of requiring manual labeling.

  • New issues auto-classified on fetch
  • needs-info issues skipped (no wasted agent runs)
  • Classifier's state machine actually exercised

5. Clean dead imports — StaffSandbox, StaffLoader, StaffContext are imported but never instantiated

Strong in-process
director.py:13-15 — from staff.sandbox import StaffSandbox (never instantiated)
director.py:13-15 — from staff.loader import StaffLoader (only used inside run_staff)
director.py:13-15 — from staff.plugin import StaffContext (never used)
scoring.py:134-135 — _queue_for_confirmation() stub (pass, never called)

Problem: director.py imports StaffSandbox, StaffLoader, and StaffContext at the top level (lines 13-15) but only uses them inside run_staff() (line 431+). These imports create hard dependencies on the staff package for every director call, even though staff plugins are never invoked in the main flow. Similarly, _queue_for_confirmation in scoring.py is a stub that does nothing.

Solution: Move staff imports inside run_staff() (lazy imports already used at line 439-440). Remove the top-level stub imports. Delete _queue_for_confirmation or implement it. This eliminates 3 unnecessary module-level dependencies.

  • 3 unused top-level imports removed
  • Staff package only loaded when staff tasks run
  • Cleaner dependency graph

6. Consolidate autonomous_loop.py and issue_bridge.py into one entry point

Speculative ports & adapters
autonomous_loop.py:562 — if __name__ (curriculum + issue mode)
issue_bridge.py:269-280 — bridge_poll() (polling loop)
issue_bridge.py:282-293 — if __name__ (single-run + polling)

Problem: autonomous_loop.py (562 lines) and issue_bridge.py (318 lines) both orchestrate the same pipeline: fetch tasks → dispatch to agents → score results. The autonomous loop has two modes: curriculum (synthetic) and issue-driven (delegates to bridge_issues). The issue bridge has its own polling loop (bridge_poll). Two entry points, two polling implementations, two CLI argument parsers — for what is fundamentally one pipeline.

Solution: Merge into one entry point with mode selection: python -m school --mode issues --repo user/repo (issue-driven, the primary path) or python -m school --mode curriculum (synthetic fallback). One polling loop, one CLI, one place to add features. The issue_bridge flow becomes the default; curriculum becomes a legacy fallback.

  • One entry point, one polling loop
  • Issue-driven mode as default
  • Curriculum mode as fallback (or delete entirely)

Top Recommendation

Candidate 1: Replace the static toy curriculum with a real codebase task adapter.

This is the root cause of the "students produce nothing" problem. The CURRICULUM dict is 30 toy problems that agents have memorized. The EMA converges to 70 because every task is trivial. The gate system is decorative because no task is hard enough to fail.

The issue_bridge already provides the real-task path — it fetches GitHub issues and dispatches them. But the agents don't read the codebase before responding, so they produce generic advice instead of real fixes. The fix is: enrich the issue prompt with repo context (file tree, relevant source files) so agents can produce targeted fixes. Then the verifier (Candidate 3) checks whether the response actually references real files.

Candidates 2 and 5 (delete routing.py, clean dead imports) are quick wins — do them in the same pass. Candidate 4 (wire triage) is needed for the bridge to handle unlabeled issues. Candidate 6 (consolidate entry points) is a follow-up once the pipeline is stable.