Design

Design Document

Overview

The Market Maker Dashboard is a static website generator that creates interactive visualizations of market maker performance, regime analysis, and trading recommendations. The system reads historical metrics data from YAML files and generates a complete static website with HTML, CSS, and JavaScript that can be viewed without a web server.

The dashboard consists of two main components:

1. Main Dashboard: Provides historical overview and real-time monitoring of grid trading performance across multiple markets
2. Spot Check Dashboard: Generates detailed analysis pages for each recommendation, preserving raw exchange data and analysis results for validation and re-analysis

Architecture

The system follows a static site generation pattern with data-driven rendering:

graph TB
    subgraph "Data Sources"
        METRICS[Metrics YAML Files]
        SPOTCHECK[Spot Check JSON Snapshots]
        CONFIG[Grid Configuration]
    end
    
    subgraph "Static Site Generator"
        SCANNER[File Scanner]
        PARSER[YAML/JSON Parser]
        PROCESSOR[Data Processor]
        RENDERER[Template Renderer]
    end
    
    subgraph "Generated Output"
        INDEX[index.html - Main Dashboard]
        SPOTPAGES[spotcheck_*.html - Spot Check Pages]
        SPOTINDEX[spotchecks/index.html - Spot Check Index]
        ASSETS[CSS/JS/Chart Libraries]
        DATA[Preprocessed JSON Data]
    end
    
    subgraph "Browser"
        BROWSER[Web Browser]
        CHARTS[Chart.js / Plotly]
    end
    
    METRICS --> SCANNER
    SPOTCHECK --> SCANNER
    CONFIG --> SCANNER
    SCANNER --> PARSER
    PARSER --> PROCESSOR
    PROCESSOR --> RENDERER
    RENDERER --> INDEX
    RENDERER --> SPOTPAGES
    RENDERER --> SPOTINDEX
    RENDERER --> ASSETS
    RENDERER --> DATA
    
    INDEX --> BROWSER
    SPOTPAGES --> BROWSER
    SPOTINDEX --> BROWSER
    ASSETS --> BROWSER
    DATA --> BROWSER
    BROWSER --> CHARTS

Components and Interfaces

Static Site Generator

Responsibilities:

• Scan metrics and spot check directories for data files
• Parse YAML metrics files and JSON spot check snapshots
• Process and optimize data for visualization
• Generate HTML pages from templates
• Copy/bundle CSS, JavaScript, and chart libraries

Key Functions:

• scan_metrics_directory(): Discovers all available metric files
• scan_spotcheck_directory(): Discovers all spot check snapshots
• parse_yaml_metrics(): Extracts pricing, regime, and grid data
• parse_spotcheck_snapshot(): Loads raw exchange data and analysis results
• preprocess_data(): Optimizes data structures for client-side rendering
• render_main_dashboard(): Generates main dashboard HTML
• render_spotcheck_page(): Generates individual spot check HTML pages
• render_spotcheck_index(): Generates index of all spot checks
• bundle_assets(): Copies shared CSS/JS libraries

Data Processor

Responsibilities:

• Transform raw data into visualization-ready formats
• Calculate derived metrics (trends, aggregations)
• Filter data by time ranges
• Handle missing or incomplete data gracefully

Key Functions:

• extract_price_series(): Converts OHLCV data to chart format
• extract_regime_series(): Prepares regime classification timeline
• extract_grid_metrics(): Aggregates grid performance data
• calculate_account_balance_series(): Tracks capital deployment over time
• prepare_spotcheck_charts(): Formats spot check data for detailed visualizations

Template Renderer

Responsibilities:

• Apply data to HTML templates
• Generate responsive layouts
• Embed preprocessed data as JavaScript
• Link to shared CSS/JS libraries

Templates:

• main_dashboard.html.j2: Main dashboard layout
• spotcheck_page.html.j2: Individual spot check page
• spotcheck_index.html.j2: List of all spot checks
• chart_component.html.j2: Reusable chart component

Client-Side Rendering

Responsibilities:

• Load preprocessed JSON data
• Render interactive charts using Chart.js or Plotly
• Handle user interactions (zoom, pan, time range selection)
• Update visualizations dynamically
• Export charts and data

Key JavaScript Modules:

• dashboard.js: Main dashboard logic
• spotcheck.js: Spot check page logic
• charts.js: Chart rendering utilities
• data-loader.js: Loads and caches JSON data
• export.js: Handles chart/data export

Spot Check Dashboard Design

Overview

The Spot Check Dashboard provides detailed analysis visualization for each recommendation generated by the Regime Management System. Unlike the main dashboard which shows historical trends, spot check pages focus on a single point in time, displaying all the underlying data and analysis factors that led to a specific recommendation.

Data Snapshot Structure

Each spot check is stored as a JSON file with the following structure:

{
  "metadata": {
    "timestamp": "2026-01-08T14:30:00Z",
    "symbol": "ETH-USDT",
    "recommendation_id": "rec-abc123",
    "collection_mode": "real-time",
    "decision_record_path": "../decisions/2026-01-08_rec-abc123.yaml"
  },
  "raw_exchange_data": {
    "timeframes": {
      "1m": [
        {"timestamp": "2026-01-08T13:30:00Z", "open": 3150.25, "high": 3152.10, "low": 3149.80, "close": 3151.50, "volume": 125.3},
        ...
      ],
      "15m": [...],
      "1h": [...],
      "4h": [...]
    },
    "source": "KuCoin",
    "api_endpoint": "/api/v1/market/candles"
  },
  "analysis_results": {
    "regime_classification": {
      "verdict": "TRANSITION",
      "confidence": 0.55,
      "strength": "WEAK",
      "competing_verdicts": [...]
    },
    "regime_scores": {
      "TrendScore": 0.45,
      "MeanRevScore": 0.38,
      "RangeQualityScore": 0.42,
      "VolLevelScore": 0.65,
      "VolChangeScore": 0.72
    },
    "discovered_range": {
      "upper_bound": 3200.00,
      "lower_bound": 2900.00,
      "discovery_method": "ROLLING_HIGHLOW",
      "range_confidence": 0.85,
      "support_strength": 0.82,
      "resistance_strength": 0.79
    },
    "gate_evaluations": {
      "gate1_directional_energy_decay": {
        "status": "FAILED",
        "metrics": {...}
      },
      "gate2_mean_reversion_return": {
        "status": "NOT_EVALUATED",
        "metrics": null
      },
      "gate3_tradable_volatility": {
        "status": "NOT_EVALUATED",
        "metrics": null
      }
    },
    "technical_indicators": {
      "ADX": [...],
      "ATR": [...],
      "bollinger_bands": {...},
      "autocorrelation": [...],
      "ou_half_life": [...]
    }
  },
  "configuration_snapshot": {
    "grid_config": {...},
    "thresholds": {...},
    "system_settings": {...}
  }
}

Spot Check Page Generation

Generation Process:

1. Snapshot Creation (during recommendation generation):

   • Regime Management System generates recommendation
   • Collects all raw KuCoin OHLCV data (1m, 15m, 1h, 4h)
   • Captures analysis results (scores, gates, ranges, verdicts)
   • Snapshots configuration state (grid params, thresholds)
   • Writes JSON file to spotchecks/ directory
   • Naming: spotcheck_{timestamp}_{symbol}_{recommendation_id}.json

2. Page Generation (during static site build):

   • Static site generator scans spotchecks/ directory
   • For each JSON snapshot:
     • Parses snapshot data
     • Applies data to spotcheck_page.html.j2 template
     • Generates HTML page with embedded data
     • Saves to output directory
   • Generates index page listing all spot checks

3. Client-Side Rendering (in browser):

   • Browser loads spot check HTML page
   • JavaScript loads embedded snapshot data
   • Renders all charts using shared chart library
   • Enables interactive features (zoom, export)

File Organization:

market-maker-data/
├── spotchecks/
│   ├── spotcheck_2026-01-08T14-30-00_ETH-USDT_rec-abc123.json
│   ├── spotcheck_2026-01-08T15-00-00_ETH-USDT_rec-def456.json
│   └── ...
├── metrics/
│   └── ...
└── decisions/
    └── ...

dashboard-output/
├── index.html (main dashboard)
├── spotchecks/
│   ├── index.html (spot check index)
│   ├── spotcheck_2026-01-08T14-30-00_ETH-USDT_rec-abc123.html
│   ├── spotcheck_2026-01-08T15-00-00_ETH-USDT_rec-def456.html
│   └── ...
├── css/
│   └── styles.css (shared)
└── js/
    ├── dashboard.js
    ├── spotcheck.js
    └── charts.js (shared)

Spot Check Visualizations

The spot check page displays comprehensive analysis data organized into sections:

1. Header Section:

• Recommendation metadata (timestamp, symbol, recommendation ID)
• Link to decision record
• Regime verdict with confidence score
• Quick summary of key metrics

2. Price Analysis Section:

• Multi-timeframe price charts (1m, 15m, 1h, 4h)
• Discovered range bounds overlaid
• Configured grid bounds overlaid (if exists)
• Price position indicators
• Technical indicators (Bollinger Bands, ATR bands, moving averages)

3. Regime Classification Section:

• Final verdict display with confidence
• Competing verdicts with weights
• Time-series charts for each regime factor:
  • Mean reversion indicators (autocorrelation, OU half-life, z-score reversion)
  • Directional persistence (ADX, normalized slope, trend strength)
  • Range quality (discovered bounds, range ratio, efficiency ratio)
  • Volatility state (ATR%, expansion/contraction, BB bandwidth)

4. Gate Evaluation Section (if applicable):

• Gate status indicators (pass/fail) for all 3 gates
• Detailed metrics for each gate component
• Time-series of gate status over evaluation period
• Threshold values and actual values comparison

5. Grid Comparison Section (if grid exists):

• Discovered range vs configured grid bounds
• Price position relative to both ranges
• Grid level visualization
• Proposed grid configuration (if recommendation includes one)

6. Configuration Section:

• Grid configuration parameters
• System thresholds
• Settings active at recommendation time

7. Export Section:

• Export all charts as PNG/SVG
• Export raw data as JSON/CSV
• Download snapshot JSON

Re-analysis Capability

The spot check system supports re-analyzing historical recommendations with updated algorithms:

Re-analysis Process:

1. Load Snapshot: Read JSON snapshot from spotchecks/ directory
2. Extract Raw Data: Get original KuCoin OHLCV data from snapshot
3. Apply New Algorithm: Run updated regime analysis code on raw data
4. Compare Results: Display old vs new analysis side-by-side
5. Update Visualization: Regenerate spot check page with new analysis

Use Cases:

• Algorithm improvements: Test new regime classification logic
• Threshold tuning: Evaluate different threshold values
• Bug fixes: Verify fixes against historical data
• Auditing: Compare system decisions over time

Implementation:

def reanalyze_spotcheck(snapshot_path, new_algorithm):
    # Load original snapshot
    snapshot = load_json(snapshot_path)
    raw_data = snapshot['raw_exchange_data']
    
    # Run new analysis
    new_analysis = new_algorithm.analyze(raw_data)
    
    # Compare with original
    original_analysis = snapshot['analysis_results']
    comparison = compare_analyses(original_analysis, new_analysis)
    
    # Generate updated page
    render_spotcheck_page(
        snapshot=snapshot,
        new_analysis=new_analysis,
        comparison=comparison
    )

Data Models

Metrics Data Structure

# metrics/2026/01/08/14_ETH-USDT.yaml
timestamp: "2026-01-08T14:00:00Z"
symbol: "ETH-USDT"
collection_mode: "real-time"
 
market:
  summary:
    open: 3145.20
    close: 3151.50
    high: 3158.30
    low: 3142.10
    volume: 15234.5
    price_change_pct: 0.2
  minute_prices: [3145.20, 3146.10, ..., 3151.50]  # 60 values
 
regime:
  verdict: "TRANSITION"
  confidence: 0.55
  strength: "WEAK"
  range_analysis:
    upper_bound: 3200.00
    lower_bound: 2900.00
    range_integrity: 0.65
  competing_verdicts:
    - verdict: "RANGE_WEAK"
      weight: 0.30
      reasons: [...]
 
grid:
  enabled: true
  status: "ACTIVE"
  configuration:
    grid_id: "eth-grid-1"
    upper_bound: 3150.00
    lower_bound: 2850.00
    levels: 7
    amount_per_grid: 0.1
 
account:
  total_usdt: 271.62
  available_usdt: 271.62
  locked_usdt: 0.00

Spot Check Snapshot Structure

See “Spot Check Dashboard Design” section above for complete JSON structure.

Error Handling

Missing Data Handling

Scenario: Metric file is corrupted or incomplete Handling:

• Log error with file path and error details
• Skip corrupted file and continue processing
• Display warning in dashboard UI indicating missing data for that period
• Show last known good data with timestamp

Scenario: Spot check snapshot missing raw exchange data Handling:

• Display warning on spot check page
• Show available analysis results
• Indicate which data sources are missing
• Disable re-analysis functionality for that snapshot

Parse Errors

Scenario: YAML/JSON parsing fails Handling:

• Log detailed error (line number, syntax issue)
• Skip file and continue
• Add to error report in dashboard
• Provide link to problematic file for manual inspection

Chart Rendering Failures

Scenario: Chart library fails to render Handling:

• Display error message in chart container
• Provide raw data table as fallback
• Log error to browser console
• Offer data export as alternative

Testing Strategy

Unit Tests

Static Site Generator:

• Test file scanning and discovery
• Test YAML/JSON parsing
• Test data preprocessing
• Test template rendering
• Test asset bundling

Data Processor:

• Test data transformations
• Test time range filtering
• Test aggregation calculations
• Test missing data handling

Integration Tests

End-to-End Generation:

• Test complete dashboard generation from sample data
• Verify all HTML pages are created
• Verify assets are copied correctly
• Verify links between pages work

Spot Check Generation:

• Test snapshot creation from recommendation
• Test spot check page generation
• Test index page generation
• Verify data integrity in generated pages

Acceptance Tests

Main Dashboard:

• Load dashboard in browser
• Verify all charts render correctly
• Test time range selection
• Test market switching
• Test export functionality
• Test responsive design on mobile

Spot Check Dashboard:

• Load spot check page in browser
• Verify all analysis sections display
• Test chart interactions
• Test export functionality
• Verify links to decision records work
• Test re-analysis functionality

Main Dashboard Enhancements

Recent Recommendations Section

The main dashboard includes a “Recent Recommendations” section that provides quick access to spot check dashboards:

Location: Full-width section below the main charts

Content:

• Table/list of recent recommendations (last 20-30)
• Each row displays:
  • Timestamp
  • Symbol (ETH-USDT, BTC-USDT, etc.)
  • Regime verdict (RANGE_OK, RANGE_WEAK, TRANSITION, TREND)
  • Confidence score
  • Action (RUN_GRID, STOP_GRID, CREATE_GRID, etc.)
  • Status (PENDING, ACCEPTED, DECLINED, TIMEOUT)
  • Link to spot check dashboard

Features:

• Sortable by timestamp, symbol, verdict, confidence
• Filterable by symbol, verdict type, action type
• Color-coded by regime verdict (using standard color scheme)
• Click row to open spot check dashboard
• Hover shows quick preview of key metrics

Implementation:

// recommendations-list.js
function renderRecommendationsList(recommendations) {
  return recommendations.map(rec => ({
    timestamp: rec.timestamp,
    symbol: rec.symbol,
    verdict: rec.regime.verdict,
    confidence: rec.regime.confidence,
    action: rec.action,
    status: rec.status,
    spotCheckLink: `/spotchecks/spotcheck_${rec.timestamp}_${rec.symbol}_${rec.id}.html`
  }));
}

Data Source:

• Scans decision records directory
• Matches decision records to spot check snapshots
• Aggregates into recommendations list
• Updates on dashboard rebuild