Aura-Quant: Building an Agentic Trading Research System

Aura-Quant is an autonomous research tool that separates trade hypothesis generation from risk gatekeeping using a multi-agent LLM pipeline, deterministic backtesting, and persistent caching. It was built to answer a simple question: what if your trading AI had someone to argue with?

What It Does and Who It's For

Aura-Quant takes a stock ticker and runs it through a research pipeline. You enter AAPL, the system fetches real market data, computes technical indicators, proposes a trade hypothesis via a local LLM, then has a second AI vet the trade for risk. If it passes, the trade backtests against 1 year of historical data and undergoes a 1,000-run Monte Carlo stress test. Every state change streams to a dashboard in real time.

Input:  AAPL
        ↓
[Fetch data] → [Compute signals] → [LLM proposes] → [AI gatekeeps]
        ↓
       YES (passed Critic)
        ↓
[30-day backtest] → [Walk-forward validation] → [1000× Monte Carlo]
        ↓
Output: Trade hypothesis + risk metrics + dashboard visualization
        (streamed live)

It's built for quant researchers, students learning about trading systems, and anyone who wants to understand how ML agents can enforce constraints on other ML systems. Not for live trading (it executes nothing), but as a research and learning tool.

The Problem That Made Me Build It

Most trading backtests are a black box: you plug in a strategy, get a number, and trust it. Most LLM trading bots have no gatekeeper — if the model proposes a trade with inverted logic or a stop-loss wider than the profit target, it executes anyway.

I wanted to build something that:

1. Made every decision visible (not a black box)
2. Had a verifiable gatekeeper (risk can't be bypassed by clever prompting)
3. Showed why a trade was proposed, not just that it was

The system that resulted is opinionated: it enforces that risk-reward ratios must be at least 1.5:1, stops can't be tighter than 0.5% or wider than 12%, and trades can't contradict the technical signals. These aren't configurable — they're hard rules.

Architecture: How It Works

The pipeline has four nodes in a LangGraph directed graph:

┌─────────────────────────────────────────────────────────┐
│                    USER INPUT (Ticker)                  │
└────────────────────────┬────────────────────────────────┘
                         │
                         ▼
            ┌────────────────────────┐
            │   RESEARCHER NODE      │
            │  Fetch data + signals  │
            │  (yfinance, 7 indicators)│
            └────────────┬───────────┘
                         │
                         ▼
            ┌────────────────────────┐
            │   ANALYST NODE         │
            │  Llama 3.2 (local)     │
            │  Propose trade (JSON)  │
            └────────────┬───────────┘
                         │
                         ▼
            ┌────────────────────────┐
            │   CRITIC NODE          │
            │  Claude / GPT-4o / Rules│
            │  Validate & constrain  │
            └────────┬───────────────┘
                     │
          ┌──────────┴──────────┐
          │                     │
        FAIL                  PASS
      (retry 3×)               │
          │                    ▼
          └──→ loop back    ┌──────────────────┐
                            │ BACKTEST EXEC    │
                            │ 30-day forward   │
                            │ + Walk-forward   │
                            │ + Monte Carlo    │
                            └────────┬─────────┘
                                     │
                                     ▼
                        ┌────────────────────────┐
                        │   Frontend Dashboard   │
                        │  (SSE stream, live viz)│
                        └────────────────────────┘

Researcher fetches the last year of OHLCV data from yfinance and computes seven technical indicators: RSI (Wilder's smoothing), MACD (12/26/9), ATR, Bollinger %B, historical volatility, volume trend, and SMA 20/50 crossover. This node produces a quant signals snapshot.

Analyst (local Llama 3.2 via Ollama) reads the signals and proposes a structured trade: entry price, exit price, stop-loss, strategy type (mean-reversion or trend-following), and a rationale. The prompt forces JSON output to eliminate regex parsing brittleness.

Critic validates the trade. First it tries Claude API (if available), then GPT-4o, then falls back to a deterministic validator that checks six conditions: all prices > 0, direction logic is sound, R:R ≥ 1.5:1, stop distance is between 0.5% and 12%, and the entry signal aligns with the strategy type. Failed trades loop back to the Analyst (up to 3 iterations).

Backtest Executor simulates the approved trade over the 30 days following the analysis date, applying 0.1% fees per leg and a 2×ATR stop. It reports final P&L.

After the pipeline completes, a separate walk-forward engine runs the same entry/exit rules (not the LLM proposal) over multi-year history, computing Sharpe, Sortino, max drawdown, CAGR, and Calmar ratio. Then a Monte Carlo engine stress-tests the trade 1,000 times with stochastic slippage.

The win probability for the Monte Carlo is derived dynamically from strategy type (trend-following empirically wins ~35% of the time vs. mean-reversion ~58%), risk-reward ratio, and historical volatility — not hardcoded to 60% for everything.

All state transitions stream to the frontend via Server-Sent Events (SSE), so the dashboard animates as the agents work.

The Hardest Technical Decision

Should the Analyst use a local LLM (Ollama, Llama 3.2) or a frontier model (GPT-4o)?

                    Cost        Reliability    Reasoning
Local (Llama 3.2)   $0          Medium         Medium
GPT-4o API          $0.005/call High          High

Local is free, private, and fast — I can call it 10 times a second without worrying about API costs or rate limits. Frontier models are more reliable and have better reasoning but cost money per call.

The deterministic validator is the safety net. Even if both LLMs fail to load, the backtest still runs and the deterministic rules still gate the trade. This was non-negotiable — a feature that requires the internet to work is a bug waiting to happen.

Critic Node (tiered fallback):
1. Try Claude API (if key available)
   └─ fail → 2. Try GPT-4o API (if key available)
              └─ fail → 3. Run deterministic validator (always works)

All three paths produce the same output format, so the downstream code doesn't know which one ran. This design is defensive: the system degrades gracefully, not fails loudly.

What I Measured

Parser reliability: The original version parsed entry prices from free-form LLM text using regex (r"Entry[^0-9]*([0-9]+\.?[0-9]*)") — it failed ~70% of the time on small variations in wording. Forcing JSON output and extracting with json.loads() reduced failures to near-zero. Not a glamorous metric, but it's the difference between "this works in a demo" and "this actually works."

Cache performance: Every agent run calls fetch_real_data() multiple times. With live yfinance fetches, one run took 5.30 seconds. With SQLite caching (24-hour TTL + coverage checks), repeat runs hit the cache in 0.013 seconds. That's a 408× speedup. Not theoretical — measured on real hardware.

Live Fetch (yfinance):     5.30 seconds    |████████████████████|
Cache Hit (SQLite):        0.013 seconds   |

Speedup:                   408×

Short trade P&L accuracy: The original P&L formula for shorts double-counted the initial capital: capital + (capital - exit_price × shares) instead of (entry_price - exit_price) × shares. On a $10k account, the error was ~$10,000 per trade. This was caught by reasoning about the accounting, not testing.

Strategy performance (real data): Running mean-reversion and trend-following on 3 years of AAPL revealed stark differences. Same ticker, same period, different regime fitness.

MEAN-REVERSION (AAPL, 3 years)
Sharpe Ratio:   0.67         ████████░░░░░░░░░░░
Win Rate:       61%          ███████████░░░░░░░░░
Return:         +44%

TREND-FOLLOWING (AAPL, 3 years)
Sharpe Ratio:   -0.35        ░░░░░░░░░░░░░░░░░░░░ (negative)
Win Rate:       17%          ██░░░░░░░░░░░░░░░░░░
Return:         -8%

Insight: AAPL 2023-2026 was range-bound → mean-reversion thrives, trend-following fails.

This is honest — no cherry-picking. The 0.67 Sharpe on mean-reversion would disappear on a trending market.

What I'd Do Differently

The vectorization is a close second. For a research tool, 30 seconds on a backtest is fine. For a portfolio of 100 tickers, it's unacceptable. I'd prioritize this before multi-asset mode.

Conclusion

Building Aura-Quant taught me that the hardest part of an AI system isn't the LLM — it's enforcing constraints. Every line of the Critic node could be questioned: "What if we loosen the R:R check?" "What if we allow wider stops?" Each loosening makes the system more 'flexible' and less trustworthy.

The insight is that a constrained system is more useful than a flexible one. A trader can reason about why a trade was rejected. An unconstrained system that "just works" is a black box, and black boxes are untrustworthy.

The caching layer (Phase 4) was the practical payoff. Because data is cached and keyed by date, experiments run 400× faster after the first fetch, turning backtesting from a "run it once and wait" operation into interactive exploration. That's where the tool stopped being a demo and became something worth using.