Spec

0003 - Market Maker Dashboard: Spot Check Pages

Summary

This specification covers the implementation of spot check page generation and visualizations for the market maker dashboard. Spot check pages provide detailed, standalone HTML views of all underlying market data and analysis factors that led to a specific recommendation, enabling traders to validate system reasoning and make informed decisions.

Problem Statement

The market maker dashboard currently displays aggregate metrics and trends, but traders need detailed visibility into the specific data and analysis that drove each recommendation. When the system recommends starting, stopping, or repositioning a grid, traders cannot easily inspect:

1. Raw market data: The actual OHLCV candlestick data across multiple timeframes (1m, 15m, 1h, 4h) that informed the analysis
2. Discovered ranges: Support/resistance levels detected by the system and their relationship to configured grid bounds
3. Gate evaluations: Which restart gates passed or failed, with specific metrics and thresholds
4. Competing verdicts: Alternative regime classifications that were considered and their relative weights
5. Recommendation context: How all these factors combined to produce the final recommendation

The snapshot creation infrastructure exists (SpotCheckSnapshot class), but the dashboard lacks the ability to scan these snapshots, parse them, generate HTML pages, and render the specialized visualizations needed for spot check analysis.

Goals

1. Create SpotCheckScanner class (Task 10.1): Scan spotchecks directory to discover JSON snapshot files, index them by timestamp/symbol/recommendation ID, and match to decision records
2. Create SpotCheckParser class (Task 10.2): Parse JSON snapshots, validate structure, and extract raw exchange data, analysis results, and configuration
3. Create spot check HTML template (Task 10.3): Design template with sections for metadata/verdict, multi-timeframe price charts, regime classification factors, gate evaluations, grid comparison, and configuration
4. Create SpotCheckGenerator class (Task 10.4): Generate individual HTML pages from snapshots using shared CSS/JS libraries from main dashboard
5. Create spot check index page generator (Task 10.5): Generate index.html listing all spot checks with sortable/filterable table and links
6. Create SpotCheckChartManager class (Task 11.1): JavaScript class for rendering multi-timeframe charts, regime factor charts, gate evaluations, grid comparison, and technical indicator overlays
7. Implement discovered range visualization (Task 11.2): Overlay support/resistance levels on price chart with rejection frequency indicators and range stability duration
8. Implement gate evaluation display (Task 11.3): Show gate status for all 3 gates with specific metrics/thresholds and time-series of gate status
9. Implement competing verdicts display (Task 11.4): Show alternative regime classifications with weights and supporting data using consistent color coding
10. Create RecommendationsScanner class (Task 12.1): Scan decision records, match to spot check snapshots, and extract recommendation metadata for the main dashboard’s recent recommendations list

Non-Goals

• Modifying the SpotCheckSnapshot class (already implemented in metrics-service)
• Adding new regime classification algorithms
• Implementing real-time streaming updates for spot check pages
• Adding new chart types to the main dashboard (this spec is spot-check focused)
• Implementing export functionality (covered in a separate task)
• Re-analysis capability (Task 13, out of scope for this spec)
• Integration tests (Task 14 checkpoint, out of scope for this spec)

Technical Approach

Phase 1: Spot Check Scanning and Parsing (Tasks 10.1, 10.2)

Create Python classes in dashboard/build/ to discover and parse spot check snapshots:

# SpotCheckScanner - discovers JSON snapshot files
class SpotCheckScanner:
    def scan_spotcheck_directory(self, directory: Path) -> List[SpotCheckMetadata]
    def index_by_timestamp(self) -> Dict[datetime, SpotCheckMetadata]
    def index_by_symbol(self) -> Dict[str, List[SpotCheckMetadata]]
    def match_to_decision_records(self, decision_records_dir: Path) -> Dict[str, str]
 
# SpotCheckParser - parses individual snapshot files
class SpotCheckParser:
    def parse_snapshot(self, filepath: Path) -> SpotCheckData
    def validate_structure(self, data: dict) -> ValidationResult
    def extract_raw_exchange_data(self) -> Dict[str, List[OHLCV]]
    def extract_analysis_results(self) -> AnalysisResults
    def extract_configuration(self) -> ConfigSnapshot

Snapshot file naming convention: spotcheck_{timestamp}_{symbol}.json

Phase 2: HTML Template and Page Generation (Tasks 10.3, 10.4, 10.5)

Create Jinja2 templates in dashboard/templates/:

spotcheck.html - Individual spot check page template:

• Header: Timestamp, symbol, recommendation ID, verdict badge with confidence
• Multi-timeframe price charts section (containers for 1m, 15m, 1h, 4h charts)
• Regime factors section (mean reversion, directional persistence, range quality, volatility)
• Gate evaluation section (3 gates with pass/fail indicators)
• Grid comparison section (discovered range vs configured grid bounds)
• Configuration section (grid config, thresholds, system settings)
• Link to decision record file

spotcheck-index.html - Index page listing all spot checks:

• Sortable table with columns: timestamp, symbol, verdict, confidence, action, status
• Filter controls for symbol, verdict type, action type
• Links to individual spot check pages
• Color coding consistent with main dashboard regime colors

# SpotCheckGenerator - generates HTML pages
class SpotCheckGenerator:
    def generate_spotcheck_page(self, snapshot: SpotCheckData, output_dir: Path) -> str
    def generate_index_page(self, all_snapshots: List[SpotCheckMetadata], output_dir: Path) -> str
    def copy_shared_assets(self, output_dir: Path)  # CSS, JS from main dashboard

Phase 3: JavaScript Visualization (Tasks 11.1, 11.2, 11.3, 11.4)

Create dashboard/js/spotcheck-chart-manager.js extending visualization capabilities:

class SpotCheckChartManager {
    // Multi-timeframe price charts
    renderMultiTimeframeCharts(rawExchangeData)
 
    // Regime classification factor charts
    renderMeanReversionFactors(indicators)  // Autocorrelation, OU half-life, z-score
    renderDirectionalPersistenceFactors(indicators)  // ADX, slope, trend strength
    renderRangeQualityFactors(indicators)  // Range bounds, ratio, efficiency
    renderVolatilityFactors(indicators)  // ATR%, expansion/contraction, BB bandwidth
 
    // Discovered range visualization
    renderDiscoveredRange(priceChart, discoveredRange)  // Overlay support/resistance
    renderRejectionFrequency(levels)  // Visual indicators at S/R levels
    renderRangeStabilityDuration(range)  // Duration indicator
 
    // Gate evaluation display
    renderGateEvaluations(gates)  // 3 gates with pass/fail indicators
    renderGateMetrics(gate)  // Specific metrics and thresholds
    renderGateTimeSeries(gateHistory)  // Gate status over evaluation period
 
    // Competing verdicts
    renderCompetingVerdicts(verdicts)  // Alternative classifications with weights
 
    // Grid comparison
    renderGridComparison(discoveredRange, gridConfig)  // Overlay on price chart
 
    // Technical indicator overlays
    renderTechnicalIndicators(priceChart, indicators)  // BB, ATR bands, MAs
}

Visualization requirements:

• Consistent color scheme with main dashboard (regimeColors mapping)
• Zoom/pan synchronization across charts where applicable
• Tooltips showing detailed values on hover
• Responsive design for various screen sizes

Phase 4: Recommendations List Integration (Task 12.1)

Create dashboard/build/recommendations_scanner.py:

class RecommendationsScanner:
    def scan_decision_records(self, directory: Path) -> List[DecisionRecord]
    def match_to_spotchecks(self, spotchecks: List[SpotCheckMetadata]) -> Dict[str, str]
    def extract_metadata(self, record: DecisionRecord) -> RecommendationMetadata
    def get_recent_recommendations(self, limit: int = 30) -> List[RecommendationMetadata]

The main dashboard will use this to populate the “Recent Recommendations” section with links to spot check pages.

Integration with Static Site Generator

Update StaticSiteGenerator.generate_site() to:

1. Call SpotCheckScanner to discover snapshots
2. Call SpotCheckGenerator to create spot check HTML pages
3. Generate spot check index page
4. Copy shared assets (CSS, JS) to output directory
5. Include recommendations list data in main dashboard context

Success Criteria

1. Spot Check Discovery: Scanner finds all JSON snapshots in spotchecks directory and correctly indexes by timestamp, symbol, and recommendation ID
2. Snapshot Parsing: Parser correctly extracts all OHLCV data for all timeframes, analysis results, and configuration
3. Page Generation: Individual spot check pages render with all sections populated and styled consistently with main dashboard
4. Index Generation: Index page lists all spot checks with working sort/filter and navigation links
5. Multi-timeframe Charts: All 4 timeframe charts (1m, 15m, 1h, 4h) render correctly with price data
6. Discovered Range Visualization: Support/resistance levels display as overlays on price chart with rejection indicators
7. Gate Evaluation Display: All 3 gates show pass/fail status with metrics and thresholds visible
8. Competing Verdicts: Alternative verdicts display with weights and consistent color coding
9. Recommendations Integration: Main dashboard shows recent recommendations with working links to spot check pages
10. Shared Assets: Spot check pages use same CSS/JS as main dashboard for consistent look and feel

Dependencies

• SpotCheckSnapshot class: Already implemented in metrics-service/src/spotcheck/snapshot.py
• Main dashboard infrastructure: StaticSiteGenerator, ChartManager, CSS, templates
• Chart.js library: Used by main dashboard, reused for spot check visualizations
• Jinja2 templates: Template infrastructure exists in dashboard/templates/
• Snapshot file format: JSON format established by SpotCheckSnapshot.write_snapshot_file()

Risks

Risk	Impact	Mitigation
Large snapshot files slow page generation	Medium	Implement lazy loading for charts, paginate index
Missing or incomplete snapshot data	Medium	Parser validates structure, pages indicate missing data
JavaScript bundle size growth	Low	SpotCheckChartManager extends existing ChartManager
Inconsistent snapshot file naming	Low	Scanner handles variations, logs warnings
Decision records not found for snapshots	Low	Pages still render, indicate “no decision record”