Backtesting System

This document covers the comprehensive backtesting framework that captures and analyzes all trading decisions and their real-market outcomes. The system provides complete audit trails from AI decision inputs through execution to final P&L results, enabling strategy evaluation and machine learning dataset generation.

For information about the core trading decision pipeline, see AI Trading Engine. For details about mathematical analysis services that provide input data, see Mathematical Analysis Services.

Data Structure and Event Model

The backtesting system uses a comprehensive event-sourcing model to capture every aspect of trading activity. The core data structure is the BacktestEventRow interface, which records all trading decisions and their outcomes.

BacktestEventRow Schema

The system tracks eight types of trading events with complete metadata:

Field	Type	Purpose
resultId	string | null	Links to complete LLM interaction logs
signalId	string | null	Groups events into trading chains
date	Date	Precise timestamp of decision/execution
takeProfit	boolean | null	Successful target achievement
stopLoss	boolean | null	Maximum loss protection trigger
closeProfit	boolean | null	Early profitable closure
closeLoss	boolean | null	Early loss-cutting closure
watch	boolean | null	Position monitoring activity
wait	boolean | null	Market analysis with no action
buy	boolean | null	Position opening execution
profitLossPercent	number | null	Actual percentage return
profitLossUsd	number | null	Dollar profit/loss amount
currentPrice	number | null	Market price at decision time

Backtesting Event Flow

Trade Outcome Categories

The system classifies all completed trades into four distinct outcome categories based on exit conditions and profitability:

Trade Classification Matrix

Trade Lifecycle Tracking

Signal Chain Reconstruction

The BacktestMeasureService reconstructs complete trading narratives by grouping events under common signalId values. Each signal represents a complete trading cycle from initial analysis through final closure.

Signal Chain Analysis Architecture

Event Timestamp Analysis

The system maintains precise temporal relationships between AI decisions and market execution. Each resultId links backtest events to complete LLM interaction logs, providing full decision context reconstruction.

Example signal chain from actual backtest data:

Date	Event Type	ResultId	Purpose
2025-09-09 05:16:29	buy	7f40d0db-40b9-4000-adc7-72a78a38d000	Position opening
2025-09-09 05:20:44	watch	19cc912c-3ef2-4800-a6ba-37c2d33e4800	Position monitoring
2025-09-09 06:04:22	takeProfit	N/A	Target achievement

Performance Analysis and Reporting

BacktestMeasureService Implementation

The BacktestMeasureService provides comprehensive trade performance analysis through statistical aggregation and outcome classification.

Performance Analysis Pipeline

Statistical Metrics

The system calculates key performance indicators for strategy evaluation:

• Success Rate: Percentage of profitable trades (Full Profit + Partial Profit)
• Average Return: Mean percentage return across all completed trades
• Maximum Drawdown: Worst loss scenario analysis
• Trade Duration: Time from buy to terminal event
• P&L Distribution: Profit/loss histogram across outcome categories

Data Integration and Export

ResultId Linking System

Every backtest event includes a resultId that links to complete AI decision records, enabling full context reconstruction for machine learning applications.

Multi-Dimensional Data Linkage

Export Infrastructure

The system provides multiple export formats for research and development applications:

// Export API endpoint structure
app.post('/export/backtest', async (req, res) => {
    const { symbol, period } = req.body;
    
    const dataset = await backtestMeasureService.getBacktestMeasure(symbol, period);
    const report = await backtestMeasureService.generateBacktestMeasureReport(symbol, period);
    
    return {
        structured_data: dataset,     // JSON for ML pipelines
        human_readable: report,       // Markdown for analysis
        metadata: {
            total_trades: dataset.totalTrades,
            success_rate: dataset.successRate,
            avg_return: dataset.averageReturn,
            max_drawdown: dataset.maxDrawdown
        }
    };
});
Copy

Development Applications

Machine Learning Dataset Generation

The backtesting system creates comprehensive supervised learning datasets with complete feature vectors and outcome labels:

Training Data Structure:

• Input Features: Technical indicators, volume data, timeframe analysis, market conditions, social signals
• Target Labels: Optimal actions (trade/wait/close/hold), position types (long/short), success probabilities
• Performance Metrics: Actual returns, trade duration, maximum drawdown, success categories

Reinforcement Learning Environment:

• State Space: Market features, position state, historical performance
• Action Space: Signal actions, risk actions, position sizing
• Reward Function: P&L-based with risk penalties and timing bonuses

Strategy Validation Framework

The system supports comprehensive strategy backtesting through historical data analysis:

1. Historical P&L Reconstruction: Complete profit/loss tracking for strategy evaluation
2. Feature Flag Testing: Analysis of different AI behavior configurations
3. Model Performance Comparison: A/B testing of different LLM providers
4. Risk Assessment Validation: Verification of risk management effectiveness

Integration Support

The backtesting system exports data in formats compatible with major analysis frameworks:

• Python ML Libraries: Pandas, NumPy, Scikit-learn compatible formats
• Deep Learning Frameworks: TensorFlow, PyTorch ready datasets
• Statistical Analysis: R, MATLAB compatible time-series data
• Visualization Tools: Plotly, D3.js compatible JSON structures