Techcle Wiki

Tasks

40 min read

Implementation Plan: Market making system

Overview

This implementation plan implements a data collection & regime detection system with clear separation of concerns: data collection creates comprehensive market snapshots, while regime detection analyzes these pre-collected snapshot files rather than calling APIs directly. This architecture enables better separation of concerns, improved testability, more reliable regime analysis, and the ability to run regime detection on historical data.

The approach focuses on advanced metrics collection with minute-level price granularity, per-market file organization, and detailed regime analysis while maintaining the existing decision-making and trading functionality. The regime classification uses percentile-ranked features, split volatility roles, range discovery, and grid capacity safety checks for robust cross-asset performance. The system uses n8n-triggered collection and generates static HTML dashboards with minute-level chart capabilities for detailed market analysis.

Tasks

  • 1. Set up metrics collection infrastructure

    • Create metrics collector service with webhook endpoint for n8n triggers
    • Implement system snapshot data aggregation from existing components
    • Set up YAML storage with Git integration for metrics history
    • Requirements: 15.1, 15.6

  • 1.2 Update hourly metrics collection to create regime-analysis-ready snapshots

    • Ensure hourly metrics collection creates snapshot files with all data needed for regime analysis
    • Verify that hourly snapshots include complete market data, grid status, and configuration
    • Add validation that hourly snapshots can be consumed by snapshot-based regime analysis
    • Test that hourly collection produces snapshots compatible with the new architecture
    • Remove any regime analysis logic from the hourly collection process (data collection only)
    • Requirements: 15.1, 15.2, 15.8, 15.9

  • [ ]* 1.3 Write property test for regime-analysis-ready snapshot creation

    • Property 79: Regime-Analysis-Ready Snapshot Completeness
    • Validates: Requirements 15.1, 15.8, 15.9
    • Test that hourly snapshots contain all data required for regime analysis
    • Test that snapshots can be successfully consumed by snapshot-based regime detection
    • Test that snapshot format is consistent between hourly and historical collection

  • [ ]* 1.4 Write property test for metrics collection webhook

    • Property 19: Metrics Collection Webhook
    • Validates: Requirements 15.1, 15.8

  • 2. Implement comprehensive data collection

    • 2.1 Implement grid status data collection

      • Collect grid_id, symbol, active status, capital allocated, and profit/loss
      • Integrate with existing Grid Manager component
      • Requirements: 15.2

    • 2.1.1 Fix grid status collection in metrics collector

      • Verify metrics collector is properly querying the Grid Manager component for active grid status
      • Ensure grid status data is being correctly retrieved and parsed from Grid Manager
      • Fix any issues causing active grids to be reported as NO_GRIDS when grids exist in Grid Manager
      • Requirements: 15.17, 15.18, 15.19

    • [ ]* 2.1.2 Write property test for grid detection accuracy

      • Property 20.1: Grid Detection Accuracy
      • Validates: Requirements 15.17, 15.18, 15.19

    • [ ]* 2.2 Write property test for grid metrics completeness

      • Property 20: Grid Metrics Completeness
      • Validates: Requirements 15.3

    • 2.3 Implement account balance data collection

      • Collect total USDT, available USDT, locked USDT, and deployable capital
      • Integrate with existing Exchange Interface
      • Requirements: 15.5

    • [ ]* 2.4 Write property test for account metrics completeness

      • Property 21: Account Metrics Completeness
      • Validates: Requirements 15.5

    • 2.5 Implement market data collection

      • Collect current price, 24h change, and volume for tracked symbols
      • Use existing Market Data Service
      • Requirements: 15.6

    • [ ]* 2.6 Write property test for market data completeness

      • Property 22: Market Data Completeness
      • Validates: Requirements 15.6

    • 2.6.1 Fix market context data to reflect analyzed period instead of execution time

      • Update market data collection to use closing price from analyzed period end
      • Replace live execution-time data with period-specific market context
      • Ensure current_context reflects the analyzed time window, not script execution time
      • Requirements: 15.7, 15.15

    • 2.6.2 Consolidate market_summary and current_context data structures

      • Merge current_context data into market_summary section to eliminate execution-time data mixing
      • Ensure all market data reflects the analyzed period, not script execution time
      • Remove redundant current_context section and consolidate into single market_summary
      • Update data structure to have single source of market data for analyzed period
      • Requirements: 15.8, 15.15, 15.16

    • [ ]* 2.6.3 Write property test for period-specific market context

      • Property 22.1: Period-Specific Market Context
      • Validates: Requirements 15.8, 15.15

    • [ ]* 2.6.4 Write property test for consolidated market data structure

      • Property 22.2: Consolidated Market Data Structure
      • Validates: Requirements 15.16

    • [-] 2.7 Implement regime state data collection

      • Collect verdict, confidence score, and active recommendations
      • Integrate with existing Regime Engine
      • The regime engine should be focused on discoverying the current regime & grid based on exchange data, functionality related to any configured grid(s) should be moved out of here
      • Requirements: 15.9

    • 2.7.1 Update regime state collection to use snapshot-based analysis

      • Modify regime state collection to read from snapshot files rather than calling regime engine directly
      • Ensure regime analysis results are stored in snapshot files for later access
      • Update regime state data structure to include snapshot file references
      • Capture refined classification scores (TrendScore, MeanRevScore, VolLevelScore, VolChangeScore, RangeQualityScore, GridCapacity) from snapshot analysis
      • Requirements: 15.9, 1.1, 1.6

    • [ ]* 2.8 Write property test for regime state completeness

      • Property 23: Regime State Completeness
      • Validates: Requirements 15.9

  • 2.9 Implement regime data retrieval (historical price data)

    • Add historical OHLCV data fetching to KuCoin exchange interface

    • Implement price history methods for regime engine technical analysis

    • Add support for configurable time periods (1h, 4h, 1d candles)

    • Requirements: 15.9

    • [ ]* 2.10 Write property test for historical data retrieval

      • Property 24: Historical Data Retrieval
      • Validates: Requirements 15.9

  • 2.11 Implement refined regime classification engine

    • 2.11.1 Implement range discovery logic

      • Create range discovery module that identifies support/resistance from market data
      • Implement multiple discovery methods: ROLLING_HIGHLOW, BB_BANDS, VOLUME_PROFILE, PRICE_CLUSTERS
      • Calculate range confidence, integrity, and duration metrics
      • Output discovered range independent of any configured grid
      • Requirements: 1.1, 1.6

    • [ ]* 2.11.2 Write property test for range discovery independence

      • Property 54: Range Discovery Independence
      • Validates: Requirements 1.1, 1.6
      • Test that discovered range is independent of configured grid bounds

    • 2.11.3 Implement feature calculation and percentile ranking

      • Calculate raw indicators: ADX, ER, slope/ATR, autocorr, OU half-life, z-score decay, ATR%, BB bandwidth
      • Implement percentile_rank() function for scale-invariant transforms
      • Convert all raw indicators to 0-100 percentile ranks over rolling window
      • Requirements: 1.1, 1.3, 1.4, 1.5

    • [ ]* 2.11.4 Write property test for percentile rank scale invariance

      • Property 55: Percentile Rank Scale Invariance
      • Validates: Requirements 1.1
      • Test that percentile ranks work consistently across different price scales

    • 2.11.5 Implement TrendScore aggregation

      • Combine ADX, ER, slope/ATR using weighted average (0.40, 0.35, 0.25)
      • Use percentile ranks as inputs
      • Output normalized TrendScore (0-100)
      • Requirements: 1.1, 1.3

    • [ ]* 2.11.6 Write property test for TrendScore aggregation

      • Property 56: TrendScore Aggregation Consistency
      • Validates: Requirements 1.1, 1.3

    • 2.11.7 Implement MeanRevScore aggregation

      • Combine neg_autocorr, inv_half_life, z_decay using median
      • Use percentile ranks as inputs
      • Output normalized MeanRevScore (0-100)
      • Requirements: 1.1, 1.3

    • [ ]* 2.11.8 Write property test for MeanRevScore aggregation

      • Property 57: MeanRevScore Aggregation Consistency
      • Validates: Requirements 1.1, 1.3

    • 2.11.9 Implement volatility split (VolLevelScore and VolChangeScore)

      • Calculate VolLevelScore from ATR%, BB bandwidth (absolute volatility level)
      • Calculate VolChangeScore from vol_expansion, bandwidth_delta, vol_percentile_change (volatility dynamics)
      • Keep scores separate for different purposes (tradability vs regime change)
      • Requirements: 1.1, 1.4, 1.7

    • [ ]* 2.11.10 Write property test for volatility split independence

      • Property 58: Volatility Split Independence
      • Validates: Requirements 1.1, 1.4, 1.7
      • Test that VolLevel and VolChange can diverge (high stable vol vs low expanding vol)

    • 2.11.11 Implement RangeQualityScore calculation

      • Calculate range_ratio and oscillation_width from discovered range
      • Aggregate into RangeQualityScore using percentile ranks
      • Implement MIN_RANGE_QUALITY threshold (default 30)
      • Requirements: 1.1, 1.6

    • [ ]* 2.11.12 Write property test for RangeQuality gate behavior

      • Property 59: RangeQuality Gate Behavior
      • Validates: Requirements 1.1, 1.6
      • Test that tight discovered ranges trigger RANGE_WEAK regardless of other scores

    • 2.11.13 Implement GridCapacity safety check

      • Calculate ExpectedMove from ATR and BB half-width
      • Calculate GridSpacing from regime-specific k_s multiplier
      • Compute GridCapacity = ExpectedMove / GridSpacing
      • Implement downgrade_regime() function for capacity > MAX_LEVELS_SAFE
      • Requirements: 1.1, 1.6

    • [ ]* 2.11.14 Write property test for GridCapacity downgrade

      • Property 60: GridCapacity Downgrade Mechanism
      • Validates: Requirements 1.1, 1.6
      • Test that excessive capacity downgrades regime by one notch

    • 2.11.15 Implement hierarchical regime classification decision tree

      • Step 1: Check RangeQualityScore gate (< MIN_RANGE_QUALITY → RANGE_WEAK)
      • Step 2: Check VolChangeScore for TRANSITION (> 70 or breakout triggers)
      • Step 3: Check TrendScore for TREND (≥ 70 with 3/5 bar confirmation)
      • Step 4: Evaluate RANGE_OK vs RANGE_WEAK based on TrendScore, MeanRevScore, VolLevelScore
      • Step 5: Apply GridCapacity safety check and downgrade if needed
      • Requirements: 1.1, 1.2, 1.3, 1.4, 1.5, 1.6

    • [ ]* 2.11.16 Write property test for hierarchical classification

      • Property 61: Hierarchical Classification Consistency
      • Validates: Requirements 1.1, 1.2, 1.3, 1.4, 1.5
      • Test that classification follows decision tree order

    • 2.11.17 Implement hysteresis and confirmation logic

      • Add 3/5 bar confirmation requirement for TREND classification
      • Implement asymmetric exit thresholds (enter at 70, exit at 50)
      • Add minimum dwell time for regime stability
      • Requirements: 1.1, 3.5

    • [ ]* 2.11.18 Write property test for hysteresis behavior

      • Property 62: Hysteresis Prevents Flapping
      • Validates: Requirements 1.1, 3.5
      • Test that regimes don’t flip rapidly back and forth

  • 2.12 Implement discovered vs configured grid comparison

    • 2.12.1 Implement grid comparison logic

      • Create compare_discovered_vs_configured() function
      • Calculate price_outside_grid, center_shift_pct, range_width_change
      • Implement decision logic for REPOSITION vs CONTINUE
      • Output structured comparison with action, reason, urgency
      • Requirements: 5.9, 15.25

    • [ ]* 2.12.2 Write property test for grid comparison triggers

      • Property 63: Grid Comparison Repositioning Triggers
      • Validates: Requirements 5.9, 15.25
      • Test that price outside bounds triggers HIGH urgency reposition
      • Test that center shift >10% triggers MEDIUM urgency reposition
      • Test that width mismatch triggers MEDIUM urgency reposition

    • 2.12.3 Implement grid comparison output in recommendations

      • Add grid_comparison block to recommendation output
      • Include discovered_range, configured_grid, alignment_analysis
      • Output recommendation with action, reason, urgency
      • Requirements: 5.9, 6.1, 6.2, 6.3, 6.4

    • [ ]* 2.12.4 Write property test for grid comparison output completeness

      • Property 64: Grid Comparison Output Completeness
      • Validates: Requirements 5.9, 6.1

    • 2.12.5 Integrate grid comparison into recommendation generator

      • Call grid comparison when configured grid exists
      • Generate GRID_REPOSITION recommendations when needed
      • Include proposed_grid with discovered range bounds
      • Requirements: 5.9, 6.1, 6.2, 6.3, 6.4, 15.25

    • [ ]* 2.12.6 Write property test for grid repositioning recommendations

      • Property 65: Grid Repositioning Recommendation Generation
      • Validates: Requirements 5.9, 6.1, 15.25

  • 3. Checkpoint - Ensure regime classification and grid comparison tests pass

    • Run unit tests for regime classification and grid comparison
    • Run integration tests to verify component interactions
    • Run acceptance tests to validate end-to-end functionality
    • All three test layers must pass before proceeding
    • Ask the user if questions arise at any test layer

  • 4. Implement enhanced metrics storage and history management

    • 4.1 Implement enhanced timing logic for previous hour analysis

      • Modify collection logic to analyze previous complete hour (HH:00:00 to HH:59:59)
      • Update file naming to use hour prefix for time-based grouping (HH_SYMBOL.yaml)
      • Add support for re-running collections to update existing files
      • Requirements: 15.2, 15.11

    • [ ]* 4.2 Write property test for previous hour analysis window

      • Property 33: Previous Hour Analysis Window
      • Validates: Requirements 15.2

    • [ ]* 4.3 Write property test for collection idempotency

      • Property 36: Collection Idempotency
      • Validates: Requirements 15.11

    • 4.4 Implement minute-level price data collection

      • Collect 60 minute-level closing prices for each analyzed hour
      • Store prices as integer arrays (price * 100) for efficiency
      • Implement price data aggregation from exchange API
      • Requirements: 15.5

    • [ ]* 4.5 Write property test for minute-level price completeness

      • Property 34: Minute-Level Price Completeness
      • Validates: Requirements 15.5

    • 4.6 Implement per-market file structure

      • Create separate YAML files for each market with hour prefix
      • Implement file organization for time-based grouping
      • Add cross-market summary files (optional)
      • Requirements: 15.8

    • [ ]* 4.7 Write property test for per-market file structure

      • Property 35: Per-Market File Structure
      • Validates: Requirements 15.8

    • 4.8 Enhance regime analysis with detailed supporting data

      • Expand regime analysis to include range analysis with discovered bounds
      • Add mean reversion metrics, volatility metrics (split into level and change)
      • Add time consistency, competing verdicts, and grid comparison data
      • Include all normalized scores (TrendScore, MeanRevScore, VolLevelScore, VolChangeScore, RangeQualityScore, GridCapacity)
      • Requirements: 15.7, 1.1, 1.3, 1.4, 1.6

    • 4.8.1 Fix confidence score calculation to produce realistic values

      • Investigate why confidence score is 0.9999999999999999 instead of expected 0.894-0.994
      • Review _calculate_confidence_score() logic in engine.py for floating point precision issues
      • Ensure confidence scores are properly bounded and realistic (typically 0.5-0.95 range)
      • Add validation to prevent confidence scores from approaching 1.0 unrealistically
      • Test confidence calculation with various market conditions to ensure reasonable outputs
      • Requirements: 3.1, 3.2, 3.3, 3.4, 3.5

    • 4.8.2 Implement conservative confidence calibration with penalty factors

      • Add time-based penalty for ranges < 36 hours (0.85-0.95 multiplier)
      • Add position-based penalty for price in upper/lower half of range (0.90-0.95 multiplier)
      • Add maturity penalty for first range after trend (0.85-0.90 multiplier)
      • Add spacing-based penalty for narrow grids < 2% of range width (0.80-0.90 multiplier)
      • Ensure typical RANGE_OK confidence scores fall between 0.60-0.85 after penalties
      • Hard cap confidence at 0.95 to prevent unrealistic certainty
      • Requirements: 3.8, 3.9, 3.10, 3.11, 3.12, 3.13

    • [ ]* 4.8.3 Write property test for confidence penalty factors

      • Property 48.2: Confidence Penalty Application
      • Validates: Requirements 3.8, 3.9, 3.10, 3.11
      • Test that time penalty is applied for ranges < 36 hours
      • Test that position penalty is applied for extreme range positions
      • Test that maturity penalty is applied for first ranges after trends
      • Test that spacing penalty is applied for narrow grids

    • [ ]* 4.8.4 Write property test for confidence score bounds

      • Property 48.1: Confidence Score Realistic Bounds
      • Validates: Requirements 3.1, 3.2, 3.3, 3.4, 3.5, 3.12, 3.13
      • Test that confidence scores are always between 0.0 and 1.0
      • Test that confidence scores rarely exceed 0.95 (unrealistic certainty)
      • Test that confidence scores respond appropriately to market uncertainty
      • Test that RANGE_OK scores typically fall between 0.60-0.85

    • 4.8.3 Implement dynamic competing verdicts calculation

      • Remove hardcoded competing_verdicts placeholder data (RANGE_WEAK: 0.18, TRANSITION: 0.04)
      • Implement _calculate_competing_verdicts() method in engine.py
      • Calculate weights for alternative verdicts based on actual market analysis
      • Use distance from classification thresholds to determine competing verdict weights
      • Exclude primary verdict from competing verdicts list
      • Generate dynamic reasons based on actual market conditions (not generic placeholders)
      • Requirements: 15.13, 15.14

    • [ ]* 4.8.4 Write property test for competing verdicts calculation

      • Property 48.2: Competing Verdicts Dynamic Calculation
      • Validates: Requirements 15.13, 15.14
      • Test that competing verdicts are calculated dynamically, not hardcoded
      • Test that primary verdict is excluded from competing verdicts
      • Test that competing verdict weights sum to less than 1.0 (primary has remaining weight)
      • Test that competing verdicts include range bounds when applicable

    • 4.8.5 Replace hardcoded decision_factors with calculated metrics

      • Remove hardcoded placeholder values (support_strength: 0.82, resistance_strength: 0.79, etc.)
      • Implement _calculate_support_resistance_strength() method to compute actual values
      • Implement _calculate_mean_reversion_metrics() for reversion_strength, avg_reversion_time, etc.
      • Implement _calculate_volatility_metrics() for ATR, volatility_expansion_ratio, etc.
      • Implement _calculate_time_consistency_metrics() for regime_agreement_pct, regime_flips, etc.
      • Use actual market data to populate all decision_factors fields
      • Requirements: 15.7, 3.2, 3.3, 3.4, 3.5

    • [ ]* 4.8.6 Write property test for decision factors calculation

      • Property 48.3: Decision Factors Calculated from Market Data
      • Validates: Requirements 15.7, 3.2, 3.3, 3.4, 3.5
      • Test that all decision_factors are calculated from actual market data
      • Test that decision_factors change appropriately with different market conditions
      • Test that decision_factors provide quantitative justification for verdicts

    • 4.8.7 Add supporting data to competing verdicts

      • Enhance competing verdicts structure to include detailed supporting_data with numerical evidence
      • Add metrics like volatility_expansion_ratio, boundary_test_count, volume_spike_ratio for each competing verdict
      • Ensure supporting_data provides quantitative justification for why each competing verdict was considered
      • Include range bounds (upper_bound, lower_bound) for range-based competing verdicts
      • Requirements: 15.8, 15.14

    • [ ]* 4.8.8 Write property test for competing verdicts supporting data completeness

      • Property 48.4: Competing Verdicts Supporting Data Completeness
      • Validates: Requirements 15.8, 15.14

    • 4.9 Create YAML serialization for enhanced snapshots

      • Implement per-market file serialization with minute-level data
      • Add Git commit automation for version control with new file structure
      • Update index management for per-market tracking
      • Requirements: 15.6, 15.9

    • [ ]* 4.10 Write property test for enhanced metrics storage format

      • Property 25: Metrics Storage Format
      • Validates: Requirements 15.6, 15.9

    • 4.11 Add error handling and n8n response formatting

      • Return structured status responses to n8n webhook calls
      • Handle partial data collection gracefully with new file structure
      • Update response format to include per-market file information
      • Requirements: 15.7, 15.11, 15.12

    • [ ]* 4.12 Write property test for collection error response

      • Property 26: Collection Error Response
      • Validates: Requirements 15.11

    • [ ]* 4.13 Write property test for n8n response format

      • Property 27: N8N Response Format
      • Validates: Requirements 15.12

  • [-] 4.5. Implement enhanced grid configuration management

    • 4.5.1 Create Grid Configuration Manager component

      • Implement detailed grid parameter storage and retrieval
      • Add support for version history with timestamps
      • Create configuration validation against actual grid behavior
      • Requirements: 17.1, 17.2, 17.4

    • [ ]* 4.5.2 Write property test for grid configuration parameter completeness

      • Property 39: Grid Configuration Parameter Completeness
      • Validates: Requirements 17.1, 17.2

    • 4.5.3 Implement grid configuration update and versioning

      • Add configuration update methods with version tracking
      • Implement configuration change history preservation

    • [~] 4.5.4 Implement probationary grid management system

      • Add probationary grid detection logic based on confidence scores (0.60-0.80)
      • Implement conservative grid parameter calculation (60% allocation, 5 levels, wider spacing)
      • Add validation criteria tracking for transitioning to normal grids
      • Create probationary grid status monitoring and progression logic
      • Requirements: 18.1, 18.2, 18.3, 18.4

    • [~] 4.5.5 Implement probationary grid validation and transition logic

      • Add time validation (36-48 hours successful operation)
      • Add boundary rejection validation (multiple clean rejections)
      • Add inventory oscillation validation (no persistent bias)
      • Add confidence stability validation (stable without sharp increases)
      • Implement automatic transition to normal grid parameters when criteria met
      • Requirements: 18.5, 18.6, 18.7, 18.8

    • [~] 4.5.6 Implement probationary grid quick stop triggers

      • Add one-sided inventory detection (>70% bias to one side)
      • Add volatility expansion detection (>20% increase)
      • Add range breach detection for discovered bounds
      • Implement quick stop recommendations as feature, not failure
      • Requirements: 18.6, 18.9

    • [ ]* 4.5.7 Write property test for probationary grid parameter calculation

      • Property 39.1: Probationary Grid Parameters
      • Validates: Requirements 18.2, 18.3, 18.4
      • Test that probationary grids use 50-60% allocation
      • Test that probationary grids use fewer levels (4-6)
      • Test that probationary grids use wider spacing (0.45-0.55 × ATR)

    • [ ]* 4.5.8 Write property test for probationary grid validation criteria

      • Property 39.2: Probationary Grid Validation
      • Validates: Requirements 18.5, 18.7, 18.8
      • Test that validation requires minimum duration
      • Test that validation requires boundary rejections
      • Test that validation requires inventory oscillation without bias

    • [ ]* 4.5.9 Write property test for probationary grid quick stop triggers

      • Property 39.3: Probationary Grid Quick Stop
      • Validates: Requirements 18.6, 18.9
      • Test that one-sided inventory triggers quick stop
      • Test that volatility expansion triggers quick stop
      • Test that range breach triggers quick stop
      • Add configuration integrity validation during updates
      • Requirements: 17.3, 17.4

    • [ ]* 4.5.4 Write property test for grid configuration update integrity

      • Property 40: Grid Configuration Update Integrity
      • Validates: Requirements 17.3, 17.4

    • 4.5.5 Enhance performance calculations with actual parameters

      • Update performance calculation logic to use actual grid parameters
      • Replace estimates with real configuration data (amount per grid, profit percentages)
      • Add validation that calculations reference configuration parameters
      • Requirements: 17.5

    • [ ]* 4.5.6 Write property test for performance calculation accuracy

      • Property 41: Performance Calculation Accuracy
      • Validates: Requirements 17.5

    • [~] 4.5.7 Integrate grid configuration into snapshot-based regime analysis

      • Update regime analysis to read grid configuration from snapshot files
      • Ensure discovered range (not configured grid bounds) is used for classification in snapshot-based analysis
      • Add grid comparison logic to snapshot-based regime analysis
      • Include grid_comparison block in regime output with alignment analysis from snapshot data
      • Update regime analysis to recommend repositioning when discovered range shifts (using snapshot data)
      • Requirements: 17.6, 17.7, 5.9, 15.25

    • [ ]* 4.5.8 Write property test for grid-aware regime analysis

      • Property 42: Grid-Aware Regime Analysis
      • Validates: Requirements 17.6, 17.7

    • 4.5.9 Add stop-loss monitoring integration

      • Integrate stop-loss price monitoring with market data
      • Add stop-loss risk assessment based on current market conditions
      • Implement stop-loss alerts when configured thresholds are approached
      • Requirements: 17.7

    • [ ]* 4.5.10 Write property test for stop-loss monitoring activation

      • Property 43: Stop-Loss Monitoring Activation
      • Validates: Requirements 17.7

    • [~] 4.5.11 Implement reinvestment calculation logic

      • Add compounding effect calculations for grids with reinvestment enabled
      • Update performance tracking to account for reinvestment mode
      • Implement reinvestment impact analysis in performance reports
      • Requirements: 17.8

    • [ ]* 4.5.12 Write property test for reinvestment calculation consistency

      • Property 44: Reinvestment Calculation Consistency
      • Validates: Requirements 17.8

    • [~] 4.5.13 Enhance multi-grid configuration management

      • Implement configuration isolation for multiple grids per symbol
      • Add grid instance identification and configuration mapping
      • Ensure configuration changes don’t affect other grid instances
      • Requirements: 17.9

    • [ ]* 4.5.14 Write property test for multi-grid configuration isolation

      • Property 45: Multi-Grid Configuration Isolation
      • Validates: Requirements 17.9

  • 4.6 Enhance recommendation generator for probationary grid support

    • [~] 4.6.1 Update recommendation generator to detect probationary conditions

      • Add logic to detect when confidence scores (0.60-0.80) trigger probationary mode
      • Add detection for early ranges (< 36 hours), position risk, and maturity penalties
      • Integrate with enhanced confidence calculation penalty factors
      • Generate probationary vs normal grid recommendations based on conditions
      • Requirements: 18.1, 18.9

    • [~] 4.6.2 Implement probationary grid recommendation generation

      • Generate GRID_SETUP recommendations with probationary parameters
      • Include probationary rationale and validation criteria in recommendations
      • Add clear marking of probe grids vs commitment grids
      • Include transition plan for scaling up to normal parameters
      • Requirements: 18.9, 18.10

    • [~] 4.6.3 Integrate probationary status into decision records

      • Add probationary grid status to decision record format
      • Include validation progress tracking in decision records
      • Record probationary rationale and transition criteria
      • Track probationary to normal grid transitions
      • Requirements: 18.9, 4.4

    • [ ]* 4.6.4 Write property test for probationary recommendation detection

      • Property 46.1: Probationary Recommendation Detection
      • Validates: Requirements 18.1, 18.9
      • Test that moderate confidence (0.60-0.80) triggers probationary recommendations
      • Test that early ranges trigger probationary mode
      • Test that position risk triggers probationary parameters

    • [ ]* 4.6.5 Write property test for probationary recommendation format

      • Property 46.2: Probationary Recommendation Format
      • Validates: Requirements 18.9, 18.10
      • Test that probationary recommendations include validation criteria
      • Test that probationary recommendations include transition plans
      • Test that probationary recommendations are clearly marked as probe grids

    • [~] 4.5.15 Add configuration version consistency to recommendations

      • Update recommendation system to reference specific configuration versions
      • Ensure recommendations use configuration active at recommendation time
      • Add configuration version tracking in decision records
      • Requirements: 17.10

    • [ ]* 4.5.16 Write property test for configuration version consistency

      • Property 46: Configuration Version Consistency
      • Validates: Requirements 17.10

    • [~] 4.5.17 Enhance metrics collection with grid configuration data

      • Update metrics collector to capture complete grid configuration snapshots
      • Add configuration parameters to hourly metrics files
      • Implement configuration change tracking in metrics history
      • Requirements: 17.11, 15.4

    • [ ]* 4.5.18 Write property test for metrics grid configuration completeness

      • Property 47: Metrics Grid Configuration Completeness
      • Validates: Requirements 17.11, 15.4

    • [~] 4.5.19 Implement historical configuration preservation

      • Ensure grid configuration snapshots are preserved in all metrics files
      • Add configuration change impact analysis capabilities
      • Enable historical configuration comparison and analysis
      • Requirements: 17.12

    • [ ]* 4.5.20 Write property test for historical configuration preservation

      • Property 48: Historical Configuration Preservation
      • Validates: Requirements 17.12

  • [~] 4.6. Checkpoint - Ensure grid configuration management tests pass

    • Ensure all grid configuration management tests pass, ask the user if questions arise.

  • 5. Implement enhanced static dashboard generation

    • [~] 5.1 Create enhanced HTML template with responsive design

      • Build self-contained HTML structure with embedded CSS
      • Add time range controls (24h, 7d, 30d, all time)
      • Add granularity controls (minute, hour, day)
      • Add market-specific tabs and controls
      • Requirements: 16.1, 16.10, 16.11

    • [ ]* 5.2 Write property test for dashboard HTML validity

      • Property 28: Dashboard HTML Validity
      • Validates: Requirements 16.1

    • [~] 5.3 Implement multi-market data aggregation

      • Load and aggregate data from individual per-market files
      • Implement efficient file loading for requested time ranges
      • Add caching for processed data to improve performance
      • Requirements: 16.7

    • [ ]* 5.4 Write property test for multi-market data aggregation

      • Property 38: Multi-Market Data Aggregation
      • Validates: Requirements 16.7

    • [~] 5.5 Implement enhanced JavaScript chart generation

      • Add Chart.js integration for interactive charts with minute-level support
      • Embed aggregated metrics data directly in HTML for offline capability
      • Add support for minute-level granularity and time aggregation
      • Requirements: 16.1, 16.9

    • [~] 5.6 Create account balance charts

      • Time-series graphs for total balance, available balance, deployed capital
      • Add trend lines and key metrics display
      • Requirements: 16.2

    • [ ]* 5.7 Write property test for account chart presence

      • Property 29: Account Chart Presence
      • Validates: Requirements 16.2

    • [~] 5.8 Create enhanced market price charts with minute-level granularity

      • Minute-level price movement charts for each market
      • Regime classification overlays on price charts
      • Support for 1-hour windows with minute-level detail
      • Requirements: 16.4

    • [ ]* 5.9 Write property test for dashboard minute-level granularity

      • Property 37: Dashboard Minute-Level Granularity
      • Validates: Requirements 16.4

    • [~] 5.10 Create grid performance charts

      • Charts for active grid count, capital deployed, profit/loss trends
      • Individual grid performance tracking by market
      • Requirements: 16.3

    • [ ]* 5.11 Write property test for grid chart presence

      • Property 30: Grid Chart Presence
      • Validates: Requirements 16.3

    • [~] 5.12 Create enhanced regime visualization

      • Confidence score trends by market
      • Regime classification distribution over time
      • Cross-market regime correlation analysis
      • Requirements: 16.5, 16.6

    • [ ]* 5.13 Write property test for market chart presence

      • Property 31: Market Chart Presence
      • Validates: Requirements 16.5

    • [ ]* 5.14 Write property test for regime chart presence

      • Property 32: Regime Chart Presence
      • Validates: Requirements 16.6

  • 6. Implement enhanced dashboard update automation

    • [~] 6.1 Add dashboard regeneration trigger with multi-market support

      • Regenerate static HTML after each successful metrics collection
      • Update embedded data from multiple per-market files
      • Refresh charts with minute-level granularity
      • Requirements: 16.8

    • [ ]* 6.2 Write property test for dashboard update timing

      • Property 33: Dashboard Update Timing
      • Validates: Requirements 16.8

    • [~] 6.3 Add enhanced dashboard serving capability

      • Serve static HTML file without requiring web server
      • Handle file access and browser compatibility
      • Support for large datasets with minute-level data
      • Requirements: 16.9

  • 7. Enhanced integration and testing

    • [~] 7.1 Wire enhanced metrics components together

      • Connect metrics collector to existing system components
      • Integrate with n8n webhook infrastructure
      • Test end-to-end metrics collection flow with per-market files
      • Verify minute-level data collection and storage
      • Requirements: 15.1, 15.2, 15.3, 15.5, 15.7

    • 7.1.1 Standardize cronjob to use taskfile methods

      • Install Task runner in the Docker container to enable taskfile usage
      • Create new task collect:git that wraps existing task collect with Git workflow operations
      • Implement Git setup task (collect:git:setup) for cloning and configuring repository
      • Implement Git commit task (collect:git:commit) for committing and pushing results
      • Update cronjob.yaml to use task collect:git instead of complex shell scripts
      • Implement convention-based environment variable overrides for environment.yaml configuration values
      • Test that task collect:git can be run locally to simulate cronjob behavior
      • Verify that task-based collection works properly in the Kubernetes container environment
      • Ensure existing task collect remains unchanged for local development use
      • Requirements: 15.9, 15.10

    • [ ]* 7.1.2 Write property test for cronjob task standardization

      • Property 81: Cronjob Task Standardization
      • Validates: Requirements 15.9, 15.10
      • Test that cronjob uses task collect:git instead of direct Python execution
      • Test that task collect:git properly wraps existing task collect with Git operations
      • Test that local task simulation produces same results as cronjob execution
      • Test that task-based collection works in container environment
      • Test that convention-based environment variable overrides work for any environment.yaml configuration value
      • Test that existing task collect behavior remains unchanged

    • [ ]* 7.2 Write integration tests for enhanced metrics system

      • Test complete collection-to-dashboard workflow with minute-level data
      • Verify n8n webhook integration with new response format
      • Test per-market file creation and aggregation
      • Requirements: 15.1, 16.8

    • [~] 7.3 Add enhanced metrics error handling integration

      • Integrate with existing error handling patterns
      • Ensure metrics failures don’t affect trading operations
      • Handle partial data collection with per-market file structure
      • Requirements: 15.11

    • [~] 7.4 Test snapshot-based regime analysis integration

      • Verify snapshot-based regime analysis data collection and storage
      • Test decision factors with supporting numerical data from snapshot files
      • Validate regime analysis storage and retrieval from snapshot files
      • Ensure regime analysis can be performed on historical snapshot data
      • Requirements: 15.7, 15.9

  • 7.5 Implement enabled status awareness in recommendation generation

    • 7.5.1 Update recommendation generator to consider grid enabled status

      • Modify recommendation logic to check enabled status from system configuration
      • Implement context-aware recommendation generation based on enabled status
      • Add enabled status field to recommendation output
      • Requirements: 5.6, 15.22

    • [ ]* 7.5.2 Write property test for enabled status consideration

      • Property 49: Enabled Status Consideration in Recommendations
      • Validates: Requirements 5.6, 15.22

    • 7.5.3 Implement “enable grid” recommendations for disabled grids with favorable conditions

      • Add logic to generate “enable grid” recommendations when grid is disabled and regime is RANGE_OK or RANGE_WEAK
      • Include actionable flag and appropriate justification in recommendation
      • Requirements: 5.7, 15.23

    • [ ]* 7.5.4 Write property test for disabled grid with favorable conditions

      • Property 50: Disabled Grid with Favorable Conditions
      • Validates: Requirements 5.7, 15.23

    • 7.5.5 Implement no-action logic for disabled grids with unfavorable conditions

      • Add logic to skip actionable recommendations when grid is disabled and regime is TRANSITION or TREND
      • Mark analysis as informational-only in these cases
      • Requirements: 5.8, 15.24

    • [ ]* 7.5.6 Write property test for disabled grid with unfavorable conditions

      • Property 51: Disabled Grid with Unfavorable Conditions
      • Validates: Requirements 5.8, 15.24

    • 7.5.7 Implement grid repositioning recommendations

      • Add logic to detect when price is significantly outside grid bounds
      • Generate recommendations to stop current grid and create new grid with updated bounds
      • Calculate appropriate new bounds based on current market price
      • Requirements: 5.9, 15.25

    • [ ]* 7.5.8 Write property test for grid repositioning

      • Property 52: Grid Repositioning Recommendation
      • Validates: Requirements 5.9, 15.25

    • 7.5.9 Update metrics collection to include all configured grids

      • Ensure metrics collector includes both enabled and disabled grids
      • Add enabled status to grid configuration data in metrics output
      • Requirements: 15.21

    • [ ]* 7.5.10 Write property test for metrics collection completeness

      • Property 53: Metrics Collection Completeness for All Grids
      • Validates: Requirements 15.21

  • [~] 7.6 Checkpoint - Ensure enabled status awareness tests pass

    • Ensure all enabled status awareness tests pass, ask the user if questions arise.

  • 8. Implement backtesting for hourly analysis mechanism

    • 8.1 Create bulk historical collection script

      • Create collect_historical_range.py script that accepts start and end dates
      • Loop through each hour in the date range and call existing metrics collection
      • Add progress reporting, error handling, and resume capability
      • Add rate limiting to avoid overwhelming the exchange API
      • Example usage: task metrics:collect:range -- --start-date 2026-01-01 --end-date 2026-01-06
      • Requirements: 15.2, 15.5, 15.8

    • [~] 8.1.2 Update bulk historical collection to create regime-analysis-ready snapshots

      • Ensure bulk historical collection creates snapshot files with all data needed for regime analysis
      • Verify that historical snapshots include minute-level price data, market context, and grid configuration
      • Add validation that historical snapshots can be consumed by snapshot-based regime analysis
      • Test that historical collection produces snapshots compatible with the new architecture
      • Requirements: 15.2, 15.5, 15.8, 15.9

    • [ ]* 8.1.3 Write property test for historical snapshot regime-analysis readiness

      • Property 80: Historical Snapshot Regime-Analysis Readiness
      • Validates: Requirements 15.2, 15.5, 15.8, 15.9
      • Test that historical snapshots contain all data required for regime analysis
      • Test that historical snapshots are compatible with snapshot-based regime detection
      • Test that historical and hourly snapshots have consistent format and structure

    • [ ]* 8.1.4 Write property test for bulk collection completeness

      • Property 38.1: Bulk Collection Completeness
      • Validates: Requirements 15.2, 15.5

    • [~] 8.2 Create historical data collection framework

      • Implement historical OHLCV data fetching for backtesting periods
      • Add support for fetching minute-level data for specified date ranges
      • Create data validation and gap detection for historical datasets
      • Requirements: 15.5

    • [~] 8.3 Implement week-long backtesting using snapshot-based analysis (7 days)

      • Run snapshot-based regime analysis over 168 hours of historical snapshot files
      • Generate per-market files for each hour using historical minute-level prices from snapshots
      • Validate regime analysis accuracy against known market conditions using snapshot data
      • Create backtesting report with regime classification distribution from snapshot analysis
      • Requirements: 15.2, 15.5, 15.8, 15.9

    • [ ]* 8.4 Write property test for week-long backtesting completeness

      • Property 39: Week-Long Backtesting Completeness
      • Validates: Requirements 15.2, 15.5

    • [~] 8.5 Implement month-long backtesting using snapshot-based analysis (30 days)

      • Run snapshot-based regime analysis over 720 hours of historical snapshot files
      • Generate comprehensive per-market file structure for full month using snapshot data
      • Analyze regime stability and transition patterns over extended period from snapshot analysis
      • Create performance metrics for regime classification accuracy using snapshot-based results
      • Requirements: 15.2, 15.5, 15.8, 15.9

    • [ ]* 8.6 Write property test for month-long backtesting completeness

      • Property 40: Month-Long Backtesting Completeness
      • Validates: Requirements 15.2, 15.5

    • [~] 8.7 Create backtesting analysis and validation tools for snapshot-based regime detection

      • Implement regime classification accuracy metrics using snapshot-based analysis results
      • Add confidence score distribution analysis from snapshot data
      • Create regime transition frequency analysis using snapshot-based regime detection
      • Generate backtesting dashboard with historical performance from snapshot analysis
      • Requirements: 16.4, 16.5, 15.9

    • [~] 8.8 Validate snapshot file structure and data integrity for regime analysis

      • Verify per-market snapshot file creation for all backtesting periods
      • Validate minute-level price data completeness in snapshot files (60 prices per hour)
      • Test file naming consistency with hour-prefix structure for snapshot files
      • Ensure regime analysis data quality across all time periods using snapshot data
      • Verify that snapshot files contain all data needed for regime analysis
      • Requirements: 15.8, 15.9

    • [ ]* 8.9 Write property test for backtesting data integrity

      • Property 41: Backtesting Data Integrity
      • Validates: Requirements 15.8, 15.9

    • [~] 8.10 Create backtesting performance benchmarks

      • Establish baseline performance metrics for regime classification
      • Create accuracy thresholds for different market conditions
      • Document expected regime distribution patterns
      • Generate performance comparison reports
      • Requirements: 15.7

  • 9. Implement Grid Restart Gates

    • 9.1 Implement Gate State Tracking

      • Create GateState class to track gate evaluation state across bars
      • Implement record_stop() method to initialize gate tracking when grid stops
      • Implement record_evaluation() method to store bar-by-bar gate results
      • Add get_status() method to query current gate status
      • Requirements: 17.1, 17.16

    • [ ]* 9.1.1 Write property test for gate state initialization

      • Property 69: Gate State Initialization
      • Validates: Requirements 17.16

    • 9.2 Implement Gate 1: Directional Energy Decay

      • Create evaluate_gate_1() function with all condition checks
      • Calculate TrendScore, ADX, normalized slope, Efficiency Ratio
      • Implement swing detection for higher highs/lower lows
      • Support configurable passage logic (consecutive vs majority-of-window)
      • Return passed status, metrics, and reason
      • Requirements: 17.3, 17.4

    • [ ]* 9.2.1 Write property test for Gate 1 condition evaluation

      • Property 57: Gate 1 Condition Evaluation Completeness
      • Validates: Requirements 17.4

    • [ ]* 9.2.2 Write property test for Gate 1 passage logic consistency

      • Property 56: Gate 1 Passage Logic Consistency
      • Validates: Requirements 17.3

    • 9.3 Implement Gate 2: Mean Reversion Return

      • Create evaluate_gate_2() function with all condition checks
      • Calculate MeanRevScore, lag-1 autocorrelation, OU half-life
      • Implement z-score reversion counting
      • Implement oscillation detection across candidate grid spacing
      • Return passed status, metrics, and reason
      • Requirements: 17.7

    • [ ]* 9.3.1 Write property test for Gate 2 condition evaluation

      • Property 60: Gate 2 Condition Evaluation Completeness
      • Validates: Requirements 17.7

    • 9.4 Implement Gate 3: Tradable Volatility

      • Create evaluate_gate_3() function with all condition checks
      • Calculate ATR percentage, BB bandwidth, volatility percentile
      • Check volatility expansion ratio and slopes
      • Return passed status, metrics, and reason
      • Requirements: 17.10

    • [ ]* 9.4.1 Write property test for Gate 3 condition evaluation

      • Property 63: Gate 3 Condition Evaluation Completeness
      • Validates: Requirements 17.10

    • 9.5 Implement Sequential Gate Evaluation

      • Create evaluate_restart_gates() function
      • Implement sequential evaluation: Gate 1 → Gate 2 → Gate 3
      • Implement blocking behavior when gates fail
      • Return verdict and complete gate results
      • Requirements: 17.2, 17.5, 17.6, 17.8, 17.9

    • [ ]* 9.5.1 Write property test for sequential gate evaluation

      • Property 55: Sequential Gate Evaluation Requirement
      • Validates: Requirements 17.2

    • [ ]* 9.5.2 Write property test for Gate 1 blocking behavior

      • Property 58: Gate 1 Blocking Behavior
      • Validates: Requirements 17.5

    • [ ]* 9.5.3 Write property test for Gate 1 to Gate 2 progression

      • Property 59: Gate 1 to Gate 2 Progression
      • Validates: Requirements 17.6

    • [ ]* 9.5.4 Write property test for Gate 2 blocking behavior

      • Property 61: Gate 2 Blocking Behavior
      • Validates: Requirements 17.8

    • [ ]* 9.5.5 Write property test for Gate 2 to Gate 3 progression

      • Property 62: Gate 2 to Gate 3 Progression
      • Validates: Requirements 17.9

    • [ ]* 9.5.6 Write property test for Gate 3 failure behavior

      • Property 64: Gate 3 Failure Behavior
      • Validates: Requirements 17.11

    • 9.6 Implement Gate Configuration Management

      • Create configuration schema for all three gates
      • Support per-symbol and per-timeframe threshold customization
      • Implement configuration validation
      • Add default configuration values
      • Requirements: 17.17

    • [ ]* 9.6.1 Write property test for per-symbol per-timeframe customization

      • Property 70: Per-Symbol Per-Timeframe Threshold Customization
      • Validates: Requirements 17.17

    • 9.7 Implement Bar-by-Bar Gate Status Tracking

      • Add bar-by-bar tracking for multi-bar gate requirements
      • Implement consecutive-bar checking logic
      • Implement majority-of-window checking logic
      • Store historical gate status for analysis
      • Requirements: 17.18

    • [ ]* 9.7.1 Write property test for bar-by-bar tracking

      • Property 71: Bar-by-Bar Gate Status Tracking
      • Validates: Requirements 17.18

    • 9.8 Integrate Gates into Regime Classification

      • Modify regime classification to call evaluate_restart_gates() after grid stop
      • Update verdict based on gate evaluation results
      • Transition to RANGE_OK only when all gates pass
      • Block CREATE_GRID recommendations until gates pass
      • Requirements: 17.2, 17.12

    • [ ]* 9.8.1 Write property test for all gates pass regime transition

      • Property 65: All Gates Pass Regime Transition
      • Validates: Requirements 17.12

    • [ ]* 9.8.2 Write property test for all gates pass grid creation permission

      • Property 66: All Gates Pass Grid Creation Permission
      • Validates: Requirements 17.13

    • 9.9 Implement Gate Evaluation Output and Recommendation Integration

      • 9.9.1 Add restart_gates section to regime analysis output
        • Include gate status, metrics, and reasons for each gate
        • Record bars since stop and stopping verdict
        • Format output for metrics collection and dashboard
        • Requirements: 17.15
      • 9.9.2 Integrate restart gates into recommendation generation
        • Update _generate_recommendations() to accept grid_running status (not just grid_enabled config)
        • Update _generate_recommendations() to accept restart_gates state
        • Implement restart-gate-aware recommendation logic:
          • If grid is running → Use normal GRID_MANAGE recommendations
          • If grid is stopped and gates not started → “No recommendation - waiting for market stabilization”
          • If grid is stopped and gates evaluating → “No recommendation - restart gates X, Y passed but Z blocking: [reason]”
          • If grid is stopped and all gates passed → “Recommend CREATE_GRID at discovered range [bounds]”
        • Remove generic fallback recommendations (“Continue normal operations”, etc.)
        • Only output concrete, actionable recommendations or explicit “No recommendation” statements
        • Requirements: 17.13, 17.15, 5.6, 5.7, 5.8
      • 9.9.3 Update metrics collector to pass grid running status
        • Query actual grid status from Grid Manager (ACTIVE, STOPPED, DISABLED)
        • Pass grid_running boolean to regime engine based on history entries
        • Use history array as single source of truth for grid state (enabled timestamp without disabled = running)
        • Requirements: 15.23, 15.24, 15.25

    • [ ]* 9.9.4 Write property test for gate evaluation output completeness

      • Property 68: Gate Evaluation Output Completeness
      • Validates: Requirements 17.15

    • [ ]* 9.9.5 Write property test for restart-gate-aware recommendations

      • Property 69: Restart Gate Recommendation Logic
      • Validates: Requirements 17.13, 5.6, 5.7, 5.8
      • Test that recommendations reflect gate status accurately
      • Test that no generic fallback recommendations are generated
      • Test that “No recommendation” is explicit when no action is warranted

    • 9.10 Implement Fresh Grid Parameter Calculation

      • Create compute_new_grid_after_gates_pass() function
      • Calculate grid parameters from current market structure
      • Use discovered range (not stopped grid bounds)
      • Apply conservative initial parameters
      • Ensure no reuse of stopped grid parameters
      • Requirements: 17.14

    • [ ]* 9.10.1 Write property test for fresh grid parameter calculation

      • Property 67: Fresh Grid Parameter Calculation
      • Validates: Requirements 17.14

    • 9.11 Add Grid Stop Event Recording

      • Record grid stop events with timestamp and verdict
      • Initialize gate tracking state on stop
      • Store stop event in decision records
      • Link stop event to gate evaluation history
      • Requirements: 17.1

    • [ ]* 9.11.1 Write property test for grid stop event recording

      • Property 54: Grid Stop Event Recording
      • Validates: Requirements 17.1

    • 9.12 Implement Minimum Time Floor After Stop

      • Add min_bars_after_stop configuration parameter
      • Implement should_evaluate_gates() check before gate evaluation
      • Prevent gate evaluation until minimum time floor passes
      • Record time floor status in gate evaluation output
      • Requirements: 17.19

    • [ ]* 9.12.1 Write property test for minimum time floor enforcement

      • Property 72: Minimum Time Floor Enforcement
      • Validates: Requirements 17.19

    • 9.13 Implement Confidence Monotonicity Guard

      • Add confidence_max_delta configuration parameter
      • Implement apply_confidence_monotonicity() function
      • Apply constraint only when regime unchanged
      • Allow full confidence change when regime changes
      • Document that this is for UI/display only, not trading logic
      • Requirements: 17.20

    • [ ]* 9.13.1 Write property test for confidence monotonicity guard

      • Property 73: Confidence Monotonicity Guard
      • Validates: Requirements 17.20

    • 9.14 Document Confidence Ignored for Restart Grid Creation

      • Add explicit documentation that confidence is ignored for CREATE_GRID after gates pass
      • Document where confidence IS used (STOP_GRID override, risk scaling)
      • Document where confidence is NOT used (CREATE_GRID permission, gate override)
      • Add configuration comments explaining the design decision
      • Requirements: 17.21

    • [ ]* 9.14.1 Write property test for confidence ignored in restart grid creation

      • Property 74: Confidence Ignored for Restart Grid Creation
      • Validates: Requirements 17.21

  • [~] 10. Checkpoint - Ensure restart gates tests pass

    • Ensure all restart gates tests pass, ask the user if questions arise.

  • 11. Clean up grid configuration and fix recommendation logic

    • 11.1 Remove redundant enabled field from grid configuration

      • Remove the top-level enabled: true field from all grid definitions in system_config_v1.yaml
      • Update configuration loading logic to use only history array for grid state determination
      • Ensure backward compatibility during transition period
      • Requirements: Grid config cleanup

    • 11.2 Implement single source of truth for grid state

      • Create is_grid_running() function that examines most recent history entry
      • If latest history entry has ‘enabled’ timestamp without ‘disabled’ timestamp, grid is running
      • If no history or latest entry has ‘disabled’ timestamp, grid is stopped
      • Replace all grid_enabled vs grid_running parameter distinctions with single state check
      • Requirements: Requirement 1, Requirement 3

    • 11.3 Fix recommendation logic to use history-based state

      • Update _generate_recommendations() method to use is_grid_running() function
      • Remove grid_enabled parameter and exchange API dependency for grid status
      • Generate correct recommendations based on actual grid state from history
      • Eliminate spurious ENABLE_GRID recommendations when grids are already running
      • Requirements: Requirement 2, Requirement 3

    • 11.4 Update metrics collection to use history-based grid state

      • Modify grid status collection to use history array instead of enabled field
      • Ensure active_grids status reflects actual grid state from history
      • Update grid configuration data structure in metrics output
      • Requirements: Requirement 2, Requirement 5

    • [~] 11.5 Validate recommendation accuracy with historical data

      • Run metrics collection on known historical periods with documented grid states
      • Verify that recommendations match expected behavior (no ENABLE_GRID when grid is running)
      • Add logging for recommendation reasoning and decision factors
      • Create audit trail for recommendation logic changes
      • Requirements: Requirement 5

    • [ ]* 11.6 Write property test for grid state determination

      • Property 75: Grid State History-Based Determination
      • Validates: Requirements 1, 3
      • Test that is_grid_running() correctly interprets history entries
      • Test that enabled without disabled means running
      • Test that no history or latest disabled means stopped

    • [ ]* 11.7 Write property test for recommendation accuracy

      • Property 76: Recommendation Logic Accuracy
      • Validates: Requirements 2, 5
      • Test that running grids don’t generate ENABLE_GRID recommendations
      • Test that stopped grids generate appropriate CREATE_GRID recommendations
      • Test that recommendation reasoning is logged and auditable

  • 12. Implement snapshot-based regime detection system

    • [~] 12.1 Create snapshot file reader for regime analysis

      • Implement snapshot file loading and parsing for regime detection
      • Add support for reading minute-level price data from YAML files
      • Create data validation for snapshot file completeness and integrity
      • Support reading multiple market snapshots for cross-market analysis
      • Requirements: 15.2, 15.5, 15.8

    • [~] 12.2 Refactor regime engine to use snapshot data instead of live API calls

      • Update regime classification engine to read from snapshot files rather than calling exchange APIs
      • Modify historical data retrieval to use pre-collected snapshot data
      • Remove direct API dependencies from regime analysis components
      • Ensure regime analysis works with both current and historical snapshot data
      • Requirements: 15.9, 1.1, 1.6

    • [~] 12.3 Implement snapshot-based market data service

      • Create market data service that reads from snapshot files
      • Replace live API calls with snapshot file data access
      • Add caching layer for frequently accessed snapshot data
      • Support time-range queries across multiple snapshot files
      • Requirements: 15.6, 15.7

    • [~] 12.4 Update recommendation generator to use snapshot-based regime analysis

      • Modify recommendation generation to work with snapshot-based regime detection
      • Ensure recommendations can be generated from historical snapshot data
      • Update grid comparison logic to use snapshot-based market data
      • Maintain compatibility with existing recommendation format
      • Requirements: 5.6, 5.7, 5.8, 5.9

    • [~] 12.5 Create snapshot data validation and integrity checks

      • Implement validation for snapshot file completeness (60 minute prices per hour)
      • Add integrity checks for market data consistency across files
      • Create error handling for missing or corrupted snapshot data
      • Add logging for snapshot data quality issues
      • Requirements: 15.8, 15.9

    • [ ]* 12.6 Write property test for snapshot-based regime detection

      • Property 77: Snapshot-Based Regime Detection Consistency
      • Validates: Requirements 15.9, 1.1
      • Test that regime detection produces consistent results when using snapshot data vs live data
      • Test that historical regime analysis works correctly with snapshot files

    • [ ]* 12.7 Write property test for snapshot data integrity

      • Property 78: Snapshot Data Integrity and Completeness
      • Validates: Requirements 15.8, 15.9
      • Test that snapshot files contain complete minute-level data (60 prices per hour)
      • Test that market data is consistent across related snapshot files

  • 13. Update existing components for snapshot-based architecture

    • 13.1 Update metrics collector to focus on data collection only

      • Separate data collection from regime analysis in metrics collector
      • Focus metrics collector on comprehensive snapshot creation
      • Remove regime analysis logic from data collection process
      • Ensure snapshot files contain all data needed for later regime analysis
      • Requirements: 15.1, 15.2, 15.3

    • 13.2 Create separate regime analysis service

      • Extract regime analysis into standalone service that reads snapshot files
      • Implement regime analysis as a separate process from data collection
      • Add support for analyzing historical periods using snapshot files
      • Create API for triggering regime analysis on specific time periods
      • Requirements: 15.9, 1.1, 1.6

    • 13.3 Update dashboard generation to use snapshot-based analysis

      • Modify dashboard generation to read regime analysis from snapshot files
      • Update chart generation to use snapshot-based market data
      • Ensure dashboard can display historical regime analysis results
      • Maintain real-time dashboard updates using latest snapshot data
      • Requirements: 16.1, 16.4, 16.5

    • 13.4 Update backtesting system for snapshot-based analysis

      • Modify backtesting to use snapshot-based regime detection
      • Enable backtesting to run regime analysis on historical snapshot files
      • Update bulk historical collection to create snapshot files suitable for regime analysis
      • Ensure backtesting results are consistent with live regime detection
      • Requirements: 15.2, 15.5, 15.8

    • [ ]* 13.5 Write integration tests for snapshot-based architecture

      • Test complete data collection → snapshot creation → regime analysis workflow
      • Verify that snapshot-based regime detection produces equivalent results to previous API-based approach
      • Test historical analysis capabilities using snapshot files
      • Requirements: 15.1, 15.9

  • 14. Checkpoint - Ensure snapshot-based architecture tests pass

    • Ensure all snapshot-based architecture tests pass, ask the user if questions arise.
    • Verify that regime detection works correctly with snapshot files
    • Confirm that data collection and regime analysis are properly separated

  • 12. Final checkpoint - Ensure all tests pass

    • Ensure all tests pass including backtesting validation, ask the user if questions arise.
    • Ensure all tests pass, ask the user if questions arise.

Notes

  • Tasks marked with * are optional and can be skipped for faster MVP
  • Each task references specific requirements for traceability
  • New tasks 1.2, 8.1.2 ensure that both hourly and historical data collection create regime-analysis-ready snapshots:
    • Hourly metrics collection updated to create snapshots suitable for regime analysis
    • Bulk historical collection updated to ensure snapshot compatibility with new architecture
    • Validation that both collection methods produce snapshots consumable by regime detection
    • Separation of data collection from regime analysis in the collection process
  • New tasks 12 and 13 implement the snapshot-based regime detection architecture with:
    • Clear separation between data collection (snapshot creation) and regime analysis (snapshot consumption)
    • Snapshot file reader for regime analysis that works with pre-collected data
    • Refactored regime engine that uses snapshot data instead of live API calls
    • Support for historical regime analysis using snapshot files
    • Validation and integrity checks for snapshot data completeness
  • Updated tasks 2.7.1, 4.5.7, 7.4, 8.3, 8.5, 8.7, 8.8 reflect the new snapshot-based architecture:
    • Regime state collection now reads from snapshot files
    • Grid configuration integration uses snapshot-based regime analysis
    • Backtesting uses snapshot-based regime detection for historical analysis
    • Data integrity validation ensures snapshot files support regime analysis
  • New tasks 2.11 and 2.12 implement the refined regime classification engine with:
    • Range discovery independent of configured grids
    • Percentile-ranked feature aggregation for scale invariance
    • Split volatility roles (VolLevel for tradability, VolChange for regime transitions)
    • RangeQuality gate to prevent tight-range false positives
    • GridCapacity safety checks linking market movement to grid geometry
    • Discovered vs configured grid comparison for repositioning recommendations
  • New task 9 implements Grid Restart Gates with:
    • Three mandatory sequential gates (Directional Energy Decay, Mean Reversion Return, Tradable Volatility)
    • Gate state tracking across bars with historical evaluation results
    • Configurable passage logic (consecutive bars vs majority-of-window)
    • Per-symbol and per-timeframe threshold customization
    • Fresh grid parameter calculation from current market structure (no reuse of stopped grid parameters)
    • Comprehensive gate evaluation output in regime analysis
    • Integration with regime classification to block CREATE_GRID until all gates pass
  • Enhanced metrics collection includes minute-level price data for detailed analysis
  • Per-market file structure improves performance and organization
  • Hour-prefix file naming enables time-based grouping and analysis
  • Enhanced regime analysis provides detailed supporting data for decision transparency
  • Backtesting tasks validate the snapshot-based regime analysis over extended periods (1 week and 1 month)
  • Historical data backtesting ensures regime classification accuracy and system reliability using snapshot data
  • Property tests validate universal correctness properties across all inputs
  • Unit tests validate specific examples and integration points
  • The system maintains existing architecture while adding advanced observational capabilities and snapshot-based regime detection
  • n8n integration provides centralized scheduling and error handling
  • Static dashboard approach ensures offline capability and simple deployment
  • Minute-level granularity enables detailed price movement analysis and regime correlation
  • Snapshot-based architecture enables regime analysis on historical data and improves system testability

Graph View