Agent thinks: "I need to run experiments and write a paper."
Agent writes: Code that runs quantization experiments
Agent runs: The code (it executes)
Agent observes: "Experiment completed. F1 score improved by 15%."
Agent concludes: "Quantization is effective."
Agent writes: "Our method improves F1 by 15%. This demonstrates efficacy."

# Agent writes this code
def load_and_preprocess(df):
    # Fill missing values with a constant
    df['age'].fillna(-1, inplace=True)  # -1 is a sentinel

    # Normalize features
    df_normalized = (df - df.mean()) / df.std()

    return df_normalized

Agent writes: "Our method achieves 95% accuracy, a 5% improvement over prior work."

Reality:
- Prior work: Not clearly defined
- Accuracy on what dataset? Test set? Validation set?
- Was statistical significance tested? (With only 100 samples, 5% difference might be noise)
- Did you run multiple seeds? What's the variance?

The claim is plausible. It's grammatically correct. But it's unsupported.

Agent reads a paper: "We use Adam optimizer with learning rate 0.001"
Agent writes code: "optimizer = Adam(lr=0.001)"
Agent runs on modern PyTorch: It fails

Problem: The paper was from 2018. PyTorch changed the API in 2020.
The agent produced plausibly correct code that doesn't actually work.

Single Agent:
  "I wrote this code."
  "Does it look right? ...yes, it does."
  → Proceeds confidently (no external reality check)

Swarm:
  Executor Agent: "I wrote this code."
  Reviewer Agent (different model, different priors): "Let me check..."
  → Debate until aligned or escalate

Claude 3.5 (Executor):
  Strength: Writing fluent, coherent code
  Blind spot: Might miss edge cases

GPT-4o (Reviewer):
  Strength: Catching logical flaws, testing assumptions
  Blind spot: Sometimes over-critical, rejects valid approaches

Together:
  Claude writes, GPT-4o questions
  → Stronger outputs than either alone

Executor: "The code runs and produces results."

Reviewer: "Running without errors ≠ correctness.
  Show me: (1) that the output matches the hypothesis,
  (2) the edge cases you tested, (3) statistical significance."

Executor: *Must now verify these things or fix the code*

Goal: "Write and publish an ML paper on quantization"

DAG:
  ├─ Literature Review (parallel with ↓)
  ├─ Experimental Design (depends on ↑)
  │  ├─ Baseline experiments (parallel with ↓)
  │  ├─ Proposed method experiments
  │  └─ Ablation studies (depends on ↑)
  ├─ Results Analysis (depends on ↑)
  ├─ Manuscript Writing (depends on ↑)
  ├─ Self-Review (depends on ↑)
  ├─ Revision (depends on ↑)
  └─ Publication

Critical: Identify dependencies.
- Literature Review can start immediately
- Experimental Design waits for Literature Review
- Ablations wait for baselines
- Writing can start before all experiments finish

Traditional sequential:
  Literature (2h) → Design (1h) → Baseline (3h) →
  Proposed (2h) → Analysis (1h) → Writing (3h)
  = 12 hours total

With parallelization:
  Literature (2h) ────────────────────┐
  Design (1h) ─────────────────────┬──┤
  Baseline (3h) ─┐                 │  │
  Proposed (2h) ─┼─ Parallel ──────┤  │
  Analysis (1h) ─┘                 │  │
  Writing (3h) ──────────────────→─┴──┘
  = 4 hours total (max path in DAG)

3x speedup just from parallelization.

Executor: "My experiment shows 15% improvement"
Reviewer: "That's not statistically significant"

Options:
1. Adversarial loop: Agents debate until aligned
   → "You're right, let me run with more samples"
   → "Good, now p < 0.05"

2. Escalate: If they can't agree after 3 iterations
   → Route to a Human-in-the-Loop reviewer
   → Or apply a pre-defined tiebreaker policy

3. Consensus: Multiple reviewers vote
   → If 2/3 approve, pass
   → If 2/3 reject, fix

It pairs:
  Executor (e.g., Claude 3.5 Sonnet)
  with
  Reviewer (e.g., GPT-4o)

Why different families?
Because models share blind spots with themselves.
A mixed-model configuration produces varied critiques,
breaking the "self-play" confirmation bias.

1. Input hypothesis
   ↓
2. Executor plans experiments
   ↓
3. Executor runs experiments (in sandbox)
   ↓
4. Reviewer audits results
   ├─ Is the experiment correct?
   ├─ Do results match claims?
   ├─ Are claims supported by evidence?
   ↓
5. If Reviewer satisfied → Move to next task
   If Reviewer unsatisfied → Send back to Executor for revision
   ↓
6. Final output: Only verified, audited results

┌─────────────────────────────────────────────────────┐
│  ASSURANCE LAYER (Verification, Auditing)          │
│  - Integrity verification                           │
│  - Result-to-claim mapping                          │
│  - Claim auditing                                   │
├─────────────────────────────────────────────────────┤
│  ORCHESTRATION LAYER (Workflow Management)         │
│  - DAG execution                                    │
│  - Task routing                                     │
│  - Effort level management                          │
├─────────────────────────────────────────────────────┤
│  EXECUTION LAYER (Skills & Tools)                  │
│  - 65+ reusable Markdown-defined skills            │
│  - MCP integrations                                │
│  - Persistent research wiki                         │
└─────────────────────────────────────────────────────┘

Skill examples:
- run_hyperparameter_sweep: Objective, parameter ranges → Best params + metrics
- statistical_test: Data1, Data2, test_type → p-value, significance
- load_dataset: Dataset name, filters → Loaded data with metadata
- plot_results: Results dict, style → Publication-quality plots
- write_section: Topic, findings → Markdown section with citations
- compare_models: Model list, metrics → Comparison table + analysis

Agents can:
- Query academic databases (ArXiv, Papers with Code)
- Access computational resources (GPU clusters)
- Interact with data storage (S3, databases)
- Call external APIs (GitHub, Hugging Face)
- All safely, via defined MCP servers

Agent action: "Search for recent papers on quantization"
MCP Integration: Calls Papers with Code API
Result: [title, abstract, code_link, citation_count]
Agent continues: Now has grounding in the literature

Research Wiki contents:
- Experimental results (baseline accuracies, hyperparameters)
- Failed approaches (with reasons why they failed)
- Literature summary (papers read, key findings)
- Code artifacts (reusable implementations)
- Intermediate hypotheses (discarded ideas)

Benefit:
If running a new experiment, ARIS doesn't re-discover old results.
It builds on prior knowledge.

The Orchestrator:
1. Accepts a high-level goal
2. Breaks it into a DAG of subtasks
3. Identifies dependencies
4. Routes tasks to appropriate agents
5. Monitors progress
6. Handles re-routing if a task fails

High effort (thorough):
- Run experiments with multiple seeds
- Statistical significance testing
- Cross-validation
- Ablation studies
- Use when: Critical results

Medium effort (standard):
- Run experiments once
- Basic validation
- Use when: Intermediate results

Low effort (quick):
- Heuristic estimation
- No validation
- Use when: Early exploration

For code:
- Does it run without errors?
- Do outputs match expected types?
- Are edge cases handled?

For results:
- Are metrics correctly computed?
- Are dimensions correct (shape mismatch detected)?
- Are units consistent?

Action: Run the artifact in a sandbox, observe outputs.

Claim: "Our method improves accuracy by 5%"

Verification:
- What was measured? (Accuracy on which dataset, which metric?)
- What was compared against? (Which baseline?)
- Is the 5% improvement correct? (Sanity check the arithmetic)
- Is this the right direction? (Did we test improvement vs. degradation?)

Action: Trace the claim to the data that supports it.

Checks:
- Is the improvement statistically significant? (p-value test)
- Was variance measured? (Std dev, confidence intervals)
- Are there confounding factors? (Did we control for them?)
- Is this reproducible? (Multiple runs, different seeds)
- Does this align with priors? (Does it match what domain experts expect?)

Action: Apply statistical rigor and domain knowledge.

"We propose a quantization strategy that reduces model size by 4x
with only 0.5% accuracy loss."

Proof:
✓ Model size: Measured in bytes, confirmed reduction
✓ 4x: Arithmetic verified (from X bytes to X/4 bytes)
✓ Accuracy loss: Measured on test set, p < 0.05
✓ 0.5%: Confirmed via cross-validation (mean ± std)

from langgraph.graph import StateGraph, END
from typing import TypedDict, Optional, List

# 1. Define the shared state between agents
class ResearchState(TypedDict):
    hypothesis: str
    code_artifact: Optional[str]
    experimental_results: Optional[str]
    reviewer_critique: Optional[str]
    execution_log: List[str]
    is_approved: bool
    iterations: int
    max_iterations: int

# 2. The Executor Agent (writes and runs code)
def executor_node(state: ResearchState):
    """
    Executor: Writes code, runs experiments.
    Takes critique and improves code iteratively.
    """

    # If there's prior critique, use it to improve
    if state.get("reviewer_critique"):
        instruction = f"""
        Previous feedback: {state['reviewer_critique']}

        Hypothesis: {state['hypothesis']}
        Current code: {state['code_artifact']}

        Fix the code based on the feedback.
        """
    else:
        # First iteration: generate from scratch
        instruction = f"""
        Design and implement an experiment for:
        {state['hypothesis']}
        """

    # Generate code
    artifact = generate_code_via_llm(instruction)

    # Execute in sandbox
    results, error = execute_sandbox(artifact)

    # Log execution
    execution_log = state.get('execution_log', [])
    execution_log.append(f"Iteration {state['iterations']}: {error or 'Success'}")

    return {
        "code_artifact": artifact,
        "experimental_results": results,
        "execution_log": execution_log,
        "iterations": state.get("iterations", 0) + 1,
        "is_approved": False  # Reset approval status
    }

# 3. The Reviewer Agent (different model, different background)
def reviewer_node(state: ResearchState):
    """
    Reviewer: Audits Executor's work.
    Uses multi-model approach (e.g., GPT-4o, not Claude).
    Conducts three-stage audit.
    """

    # Stage 1: Integrity Verification
    integrity_check = verify_integrity(
        state["code_artifact"],
        state["experimental_results"]
    )

    if not integrity_check.passed:
        critique = f"""
        INTEGRITY FAILURE: {integrity_check.error}

        Fix: {integrity_check.suggestion}
        """
        return {
            "is_approved": False,
            "reviewer_critique": critique
        }

    # Stage 2: Result-to-Claim Mapping
    claim_check = map_results_to_claims(
        state["hypothesis"],
        state["experimental_results"]
    )

    if not claim_check.valid:
        critique = f"""
        CLAIM-RESULT MISMATCH: {claim_check.issue}

        Your results show: {claim_check.actual}
        Your claim says: {claim_check.stated}

        Revise the code or the claim.
        """
        return {
            "is_approved": False,
            "reviewer_critique": critique
        }

    # Stage 3: Claim Auditing (statistical rigor)
    audit_check = audit_claims(
        state["hypothesis"],
        state["experimental_results"]
    )

    if not audit_check.sound:
        critique = f"""
        STATISTICAL ISSUE: {audit_check.issue}

        Problem: {audit_check.detail}

        Resolution: {audit_check.fix}
        """
        return {
            "is_approved": False,
            "reviewer_critique": critique
        }

    # All checks passed
    return {
        "is_approved": True,
        "reviewer_critique": None
    }

# 4. Orchestrate the Swarm
workflow = StateGraph(ResearchState)

# Add the nodes (agents)
workflow.add_node("Executor", executor_node)
workflow.add_node("Reviewer", reviewer_node)

# 5. Routing and Conflict Resolution
def route_after_review(state: ResearchState):
    """
    Conflict resolution logic.
    Decides where to go after Reviewer speaks.
    """

    if state.get("is_approved"):
        # Reviewer satisfied, move forward
        return END

    if state.get("iterations", 0) >= state.get("max_iterations", 3):
        # Hit iteration limit, escalate to human or declare failure
        return "Escalate"

    # Not approved, iterate
    return "Executor"

# Connect the nodes
workflow.add_edge("Executor", "Reviewer")
workflow.add_conditional_edges("Reviewer", route_after_review)
workflow.set_entry_point("Executor")

# 6. Compile and execute
research_swarm = workflow.compile()

# Run the adversarial loop
initial_state = {
    "hypothesis": "Selective head quantization improves efficiency",
    "code_artifact": None,
    "experimental_results": None,
    "reviewer_critique": None,
    "execution_log": [],
    "is_approved": False,
    "iterations": 0,
    "max_iterations": 3
}

final_result = research_swarm.invoke(initial_state)

print(f"Execution Log: {final_result['execution_log']}")
print(f"Final Code: {final_result['code_artifact']}")
print(f"Verified Results: {final_result['experimental_results']}")

Executor: "I ran this experiment 1,000 times"
Reviewer: "That's overkill. 100 runs is sufficient."

Executor: "But I want low variance..."
Reviewer: "Show me the variance reduction from 100 to 1,000"

Executor: *computes* "Variance reduced by 0.02%"
Reviewer: "That's negligible for our purpose. 100 runs is fine."

Consensus: Use 100 runs, save 90% compute.

After 3 iterations, agents still disagree.
System escalates to human reviewer.

Human makes call: "Run 500 times, split the difference."

Instead of 2 agents, use 3+ reviewers.

Results:
- Claude: "Approve"
- GPT-4o: "Request changes"
- LLaMA: "Approve"

Voting rule: 2/3 approve → proceed

Multiple Executors propose different solutions.
Reviewer ranks them.
Best solution advances.

Use case: Hyperparameter optimization

Agent 1: Literature review
Agent 2: Experimental design (awaits Agent 1)
Agent 3: Execution (awaits Agent 2)
Agent 4: Analysis (parallel with Agent 3)
Agent 5: Writing (awaits Agent 4)
Agent 6: Review (awaits Agent 5)

Depends-on edges create a DAG of execution.

Manager Agent: Routes tasks
├─ Research Sub-Agent: Runs experiments
├─ Writing Sub-Agent: Generates prose
├─ Review Sub-Agent: Quality checks
└─ Integration Sub-Agent: Combines results

ML Expert Agent: "Use XGBoost"
Domain Expert Agent: "Use linear model (interpretability)"

System: Runs both, benchmarks on domain metrics.
Consensus: "Use XGBoost with SHAP for interpretability."

Sequential: 12 hours
Parallel orchestrated: 4 hours