Techcle Wiki

Plan

10 min read

0004 - Implementation Plan: Regime Management - Snapshot Architecture

Reference

  • Specification: /home/coder/src/workspaces/codev/codev/specs/0004-regime-snapshot-architecture.md
  • Task Breakdown: /home/coder/src/workspaces/codev/repos/market-making/.kiro/specs/regime-management/tasks.md
  • Repository: /home/coder/src/workspaces/codev/repos/market-making/

Current State Summary

The codebase has substantial implementation already in place:

Completed Infrastructure:

  • SnapshotStorage class for storing JSON snapshots (per-hour directory structure)
  • MetricsCollector with snapshot creation and regime analysis validation
  • RangeDiscovery module with multiple discovery methods
  • FeatureCalculator for raw indicators and percentile ranking
  • ScoreAggregator for regime score calculation
  • Restart gates implementation (all 3 gates, state tracking, configuration)
  • Grid comparison logic in grid_comparison.py

Partially Implemented:

  • RegimeEngine.analyze_from_snapshot() exists but returns placeholder data
  • Snapshot files are created but regime analysis reads from live APIs
  • Property tests defined in tasks.md but not yet written

Remaining Work:

  • Wire analyze_from_snapshot() to use actual discovery/calculation modules
  • Complete separation of data collection from regime analysis
  • Create standalone regime analysis service
  • Update dashboard for snapshot-based display
  • Write checkpoint property tests (Tasks 4.6, 7.2, 7.6, 10, 14)

Phase 1: Complete Snapshot Reader for Regime Analysis

Objective: Enable regime analysis to read from pre-collected snapshot files containing minute-level price data and historical OHLCV.

Tasks

1.1. Create SnapshotReader class in metrics-service/src/metrics/snapshot_reader.py

  • Implement load_snapshot(symbol, timestamp) method
  • Implement load_range(symbol, start, end) for time ranges
  • Support reading from hourly directory structure (snapshots/YYYY/MM/DD/HH/)
  • Parse JSON files: ETH-USDT_1m.json, ETH-USDT_1h.json, etc.

1.2. Add snapshot validation to SnapshotReader

  • Validate 60 minute-level prices per hour
  • Check required fields present (market_summary, metadata)
  • Validate timestamp consistency across related files

1.3. Create helper methods for OHLCV conversion

  • Convert integer-stored prices back to floats (divide by 100)
  • Build price_data format expected by existing analysis methods

Files to Create/Modify

  • Create: /home/coder/src/workspaces/codev/repos/market-making/metrics-service/src/metrics/snapshot_reader.py
  • Modify: /home/coder/src/workspaces/codev/repos/market-making/metrics-service/src/metrics/__init__.py

Success Criteria

  • SnapshotReader can load snapshot files from any valid timestamp
  • Validation catches incomplete snapshots (< 60 minute prices)
  • Unit tests pass for snapshot loading and validation

Dependencies

  • None (foundational phase)

Phase 2: Wire analyze_from_snapshot to Existing Modules

Objective: Connect the analyze_from_snapshot() method to actually use the existing range discovery, feature calculation, and score aggregation modules.

Tasks

2.1. Refactor analyze_from_snapshot() in engine.py to:

  • Convert historical_data from snapshot format to price_data format
  • Call RangeDiscovery.discover_range() with converted data
  • Call FeatureCalculator.calculate_raw_features() and to_percentile_ranks()
  • Call ScoreAggregator methods for TrendScore, MeanRevScore, etc.

2.2. Implement hierarchical regime classification using scores

  • Step 1: Check RangeQualityScore gate (< 30 → RANGE_WEAK)
  • Step 2: Check VolChangeScore for TRANSITION (> 70)
  • Step 3: Check TrendScore for TREND (>= 70 with confirmation)
  • Step 4: Evaluate RANGE_OK vs RANGE_WEAK
  • Step 5: Apply GridCapacity safety check

2.3. Add grid comparison when grid_config provided

  • Use compare_discovered_vs_configured() from grid_comparison.py
  • Include comparison results in output

2.4. Calculate competing verdicts dynamically

  • Use existing _calculate_competing_verdicts() logic
  • Ensure primary verdict excluded from competing list

2.5. Calculate decision factors with numerical support

  • Use existing _calculate_decision_factors() logic
  • Include supporting data from calculated metrics

Files to Modify

  • /home/coder/src/workspaces/codev/repos/market-making/metrics-service/src/regime/engine.py
  • /home/coder/src/workspaces/codev/repos/market-making/metrics-service/src/regime/classifier.py (if needed)

Success Criteria

  • analyze_from_snapshot() returns actual calculated values (not placeholders)
  • Regime verdicts match expected classification for known market conditions
  • All scores (TrendScore, MeanRevScore, etc.) populated with real values
  • Grid comparison included when grid_config provided

Dependencies

  • Phase 1 (SnapshotReader for loading historical data)

Phase 3: Separate Metrics Collector from Regime Analysis

Objective: Ensure metrics collector focuses only on data collection while regime analysis operates on pre-collected snapshots.

Tasks

3.1. Modify _aggregate_system_data() in collector.py

  • Remove direct _collect_regime_state() call for live API regime analysis
  • Replace with call to analyze_from_snapshot() using collected data
  • Ensure snapshot is stored BEFORE regime analysis runs

3.2. Update regime analysis flow

  • Collector creates complete snapshot with market data
  • Snapshot stored to JSON files
  • Regime analysis reads from snapshot data (not live APIs)
  • Results added to snapshot before final storage

3.3. Add regime analysis validation

  • Verify snapshot data is complete before analysis
  • Log warnings if analysis skipped due to missing data

Files to Modify

  • /home/coder/src/workspaces/codev/repos/market-making/metrics-service/src/metrics/collector.py

Success Criteria

  • Metrics collector produces complete snapshots without embedded regime analysis
  • Regime analysis operates solely on snapshot data
  • Existing tests continue to pass

Dependencies

  • Phase 2 (analyze_from_snapshot working)

Phase 4: Create Standalone Regime Analysis Service

Objective: Extract regime analysis into a standalone service that can analyze any snapshot file.

Tasks

4.1. Create RegimeAnalysisService class

  • Accept SnapshotReader as dependency
  • Provide analyze_timestamp(symbol, timestamp) method
  • Provide analyze_range(symbol, start, end) for batch analysis
  • Support analyzing historical periods

4.2. Create CLI entry point for standalone analysis

  • Add task regime:analyze command
  • Accept timestamp and symbol parameters
  • Output regime analysis results to stdout or file

4.3. Add batch analysis support for backtesting

  • Process multiple hours efficiently
  • Support parallel analysis where appropriate
  • Generate summary reports

Files to Create/Modify

  • Create: /home/coder/src/workspaces/codev/repos/market-making/metrics-service/src/regime/analysis_service.py
  • Create: /home/coder/src/workspaces/codev/repos/market-making/metrics-service/analyze_regime.py
  • Modify: /home/coder/src/workspaces/codev/repos/market-making/metrics-service/Taskfile.yml

Success Criteria

  • Can run regime analysis on any historical snapshot via CLI
  • Batch analysis completes for week-long periods
  • Results match live analysis for same time periods

Dependencies

  • Phase 3 (separation complete)

Phase 5: Update Dashboard for Snapshot-Based Display

Objective: Modify dashboard generation to read regime results from snapshot files.

Tasks

5.1. Update data_aggregator.py to read from snapshot structure

  • Load regime analysis from JSON snapshot files
  • Aggregate results across time periods
  • Support both recent and historical data

5.2. Update chart generation for snapshot data

  • Ensure minute-level price charts use snapshot data
  • Display regime classifications from snapshot analysis
  • Show discovered ranges on price charts

5.3. Add historical regime visualization

  • Show regime classification over time
  • Display confidence trends
  • Include grid comparison events

Files to Modify

  • /home/coder/src/workspaces/codev/repos/market-making/dashboard/build/data_aggregator.py
  • /home/coder/src/workspaces/codev/repos/market-making/dashboard/build/static_site_generator.py

Success Criteria

  • Dashboard displays data from snapshot files
  • Historical regime analysis visible in charts
  • Dashboard loads without requiring live API access

Dependencies

  • Phase 4 (standalone analysis service)

Phase 6: Checkpoint Tests - Grid Configuration Management (Task 4.6)

Objective: Complete property tests 39-48 for grid configuration management.

Tasks

6.1. Write Property Test 39: Grid Configuration Parameter Completeness

  • Test all grid parameters are captured in configuration
  • Validate parameter types and ranges

6.2. Write Property Test 40: Grid Configuration Update Integrity

  • Test version history preservation
  • Validate configuration change tracking

6.3. Write Property Tests 39.1-39.3: Probationary Grid Parameters

  • Test allocation (50-60%)
  • Test level count (4-6)
  • Test wider spacing (0.45-0.55 x ATR)

6.4. Write Property Tests for validation criteria and quick stop triggers

  • Tests 39.2, 39.3 as defined in tasks.md

Files to Create/Modify

  • Create: /home/coder/src/workspaces/codev/repos/market-making/metrics-service/tests/property/test_grid_configuration.py

Success Criteria

  • All property tests 39-48 written and passing
  • Tests cover probationary grid parameters, validation, quick stops
  • task test passes with grid configuration tests

Dependencies

  • None (can run in parallel with other phases)

Phase 7: Checkpoint Tests - Enhanced Metrics Integration (Task 7.2)

Objective: Complete integration tests for the metrics collection workflow.

Tasks

7.1. Write integration test for collection-to-dashboard workflow

  • Trigger metrics collection
  • Verify snapshot file creation
  • Verify dashboard update

7.2. Write n8n webhook integration test

  • Test webhook endpoint response format
  • Verify trigger_time handling
  • Test error response formatting

7.3. Write per-market file creation test

  • Verify file naming (HH_SYMBOL.yaml)
  • Verify minute-level data presence
  • Test multi-symbol collection

Files to Create/Modify

  • Modify: /home/coder/src/workspaces/codev/repos/market-making/metrics-service/tests/integration/test_metrics_workflow.py

Success Criteria

  • Integration tests cover full collection workflow
  • n8n webhook tests verify response format
  • Per-market file tests pass

Dependencies

  • Phase 3 (metrics collector separation)

Phase 8: Checkpoint Tests - Enabled Status Awareness (Task 7.6)

Objective: Complete property tests 49-53 for enabled status awareness.

Tasks

8.1. Write Property Test 49: Enabled Status Consideration

  • Test recommendations consider grid enabled status
  • Verify enabled status in recommendation output

8.2. Write Property Test 50: Disabled Grid with Favorable Conditions

  • Test “enable grid” recommendations generated
  • Verify actionable flag set correctly

8.3. Write Property Test 51: Disabled Grid with Unfavorable Conditions

  • Test no-action logic for TRANSITION/TREND
  • Verify informational-only marking

8.4. Write Property Tests 52-53: Grid Repositioning

  • Test price-outside-bounds detection
  • Test reposition recommendation generation

Files to Create/Modify

  • Create: /home/coder/src/workspaces/codev/repos/market-making/metrics-service/tests/property/test_enabled_status.py

Success Criteria

  • All property tests 49-53 written and passing
  • Tests cover all enabled status scenarios
  • task test passes with enabled status tests

Dependencies

  • None (can run in parallel)

Phase 9: Checkpoint Tests - Restart Gates (Task 10)

Objective: Complete property tests 54-74 for restart gates.

Tasks

9.1. Write Property Tests 54-55: Gate State and Sequential Evaluation

  • Test gate state initialization (54)
  • Test sequential evaluation requirement (55)

9.2. Write Property Tests 56-58: Gate 1 Behavior

  • Test passage logic consistency (56)
  • Test condition evaluation completeness (57)
  • Test blocking behavior (58)

9.3. Write Property Tests 59-62: Gate 2 Behavior

  • Test Gate 1 to Gate 2 progression (59)
  • Test condition evaluation completeness (60)
  • Test blocking behavior (61)
  • Test Gate 2 to Gate 3 progression (62)

9.4. Write Property Tests 63-66: Gate 3 and Regime Integration

  • Test condition evaluation completeness (63)
  • Test failure behavior (64)
  • Test regime transition (65)
  • Test grid creation permission (66)

9.5. Write Property Tests 67-74: Advanced Gate Features

  • Test fresh grid parameters (67)
  • Test output completeness (68)
  • Test state initialization (69)
  • Test per-symbol customization (70)
  • Test bar-by-bar tracking (71)
  • Test time floor enforcement (72)
  • Test confidence monotonicity (73)
  • Test confidence ignored for restart (74)

Files to Create/Modify

  • Modify: /home/coder/src/workspaces/codev/repos/market-making/metrics-service/tests/regime/test_restart_gates.py

Success Criteria

  • All property tests 54-74 written and passing
  • Tests cover all gate behaviors documented in spec
  • task test passes with restart gates tests

Dependencies

  • None (restart gates already implemented)

Phase 10: Checkpoint Tests - Snapshot Architecture (Task 14)

Objective: Complete full test suite for snapshot-based architecture.

Tasks

10.1. Write Property Test 77: Snapshot-Based Regime Detection Consistency

  • Test regime detection produces consistent results
  • Compare snapshot-based vs historical analysis

10.2. Write Property Test 78: Snapshot Data Integrity

  • Test 60 minute prices per hour
  • Test data consistency across files

10.3. Write Property Test 79: Regime-Analysis-Ready Snapshot Completeness

  • Test hourly snapshots contain all required data
  • Test snapshots consumable by regime detection

10.4. Write integration tests for snapshot architecture

  • Test data collection → snapshot creation → regime analysis workflow
  • Verify snapshot-based results match API-based results
  • Test historical analysis using snapshot files

Files to Create/Modify

  • Create: /home/coder/src/workspaces/codev/repos/market-making/metrics-service/tests/property/test_snapshot_architecture.py
  • Modify: /home/coder/src/workspaces/codev/repos/market-making/metrics-service/tests/integration/test_snapshot_integration.py

Success Criteria

  • All snapshot architecture property tests passing
  • Integration tests verify end-to-end workflow
  • task test passes with all snapshot tests

Dependencies

  • Phase 4 (standalone regime analysis service)

Phase 11: Update Backtesting for Snapshot-Based Analysis

Objective: Enable backtesting to run regime analysis on historical snapshot files.

Tasks

11.1. Update collect_historical_range.py to create regime-ready snapshots

  • Ensure historical collection creates compatible snapshots
  • Add validation for snapshot completeness

11.2. Create backtesting runner using RegimeAnalysisService

  • Process historical snapshots in date range
  • Generate regime classification distribution
  • Compare results to known market conditions

11.3. Add backtesting report generation

  • Regime classification accuracy metrics
  • Confidence score distribution
  • Regime transition analysis

Files to Modify

  • /home/coder/src/workspaces/codev/repos/market-making/metrics-service/collect_historical_range.py
  • Create: /home/coder/src/workspaces/codev/repos/market-making/metrics-service/backtest_regime.py

Success Criteria

  • Historical collection creates regime-analysis-ready snapshots
  • Backtesting runs on week-long and month-long periods
  • Backtesting report shows regime distribution

Dependencies

  • Phase 4 (standalone regime analysis service)

Phase 12: Final Validation and Documentation

Objective: Verify all components work together and document the architecture.

Tasks

12.1. Run full test suite

  • Execute task test for all unit tests
  • Execute integration tests
  • Verify all checkpoint tests pass

12.2. Validate snapshot/API result equivalence

  • Run parallel comparison of snapshot-based vs API-based analysis
  • Document any discrepancies
  • Ensure results match within acceptable tolerance

12.3. Update inline documentation

  • Document snapshot file format
  • Document regime analysis service API
  • Update module docstrings as needed

Files to Modify

  • Various source files for documentation updates

Success Criteria

  • All tests pass (task test)
  • Snapshot-based analysis produces equivalent results to API-based
  • Documentation current and complete

Dependencies

  • All previous phases complete

Summary: Phase Dependencies

Phase 1 (Snapshot Reader)
    |
    v
Phase 2 (Wire analyze_from_snapshot)
    |
    v
Phase 3 (Separate Metrics Collector)
    |
    v
Phase 4 (Standalone Regime Service)
    |
    +--> Phase 5 (Dashboard Update)
    |
    +--> Phase 10 (Snapshot Architecture Tests)
    |
    +--> Phase 11 (Backtesting Update)
    |
    v
Phase 12 (Final Validation)

Parallel phases (no dependencies):
- Phase 6 (Task 4.6 - Grid Config Tests)
- Phase 7 (Task 7.2 - Integration Tests) [after Phase 3]
- Phase 8 (Task 7.6 - Enabled Status Tests)
- Phase 9 (Task 10 - Restart Gates Tests)

Key Implementation Notes

  1. Preserve Existing Code: The range discovery, feature calculation, score aggregation, and restart gates modules are already implemented. This plan wires them together rather than rewriting.

  2. Incremental Integration: Each phase produces independently testable functionality that can be committed separately.

  3. Property Tests Optional: Per the task file, property tests marked with * are optional for faster MVP but should be completed for full validation.

  4. Snapshot Format Stable: The JSON snapshot format is established. No changes to file format are planned.

  5. Backward Compatibility: Existing get_current_regime() continues to work via live APIs. Snapshot-based analysis is additive.

Graph View