An enterprise-grade quantitative econometric system, statistical distance classifier, and machine learning framework engineered to detect financial asset bubbles, diagnose non-linear macroeconomic regime shifts, quantify systemic distance from historical equilibrium, and dynamically adjust portfolio equity exposure with zero lookahead bias and institutional cost accounting.
The platform provides dual production runtime architectures:
- NiceGUI Analytical Workstation (Server-Side): Powered by FastAPI, multithreaded Polars Arrow processing, and WebGL Plotly charts with dynamic light/dark accessibility theming.
- HoloViz Panel WebAssembly Dashboard (Client-Side): Pre-compiled via Pyodide and Bokeh/Plotly, executing 100% in-browser with zero cloud computing costs, zero server daemons, and zero private data egress.
🚀 Access the Live Panel WebAssembly Dashboard on GitHub Pages
(Compiled directly to client-side WebAssembly using HoloViz Panel, Pyodide, and Plotly — runs entirely in your browser with zero remote data transfer!)
The system features a calendar-aware date generator (bubble_detector/data/date_horizons.py) that dynamically anchors historical lookbacks to the operational execution date:
| Horizon Option | Dynamic Time Range | Regimes Captured | Data Ingestion Pipeline | Target Coverage |
|---|---|---|---|---|
| Option 1: Comprehensive 50-Year Multi-Decade Horizon | Dynamic 50-year lookback (e.g. 1976-09-03 to 2026-09-03) |
9 Historical Regimes (~13,000 trading days) | Polars Parquet caching + continuous backward return compounding | 1970s Great Inflation & Volcker Rate Shock (20% Fed Funds), 1987 Black Monday, 1990 S&L Crisis & Recession, 2000 Dot-Com Tech Bubble & Crash, 2007–2009 Subprime GFC, 2018 Volmageddon & Q4 QT, 2020 COVID Flash Crash, 2022 Fed Rate Hikes, 2024–2026 AI CapEx Concentration |
| Option 2: Modern 5-Regime Native Horizon | 2015-01-01 to current date |
5 Modern Regimes (~3,000 trading days) | 100% Native High-Fidelity Exchange Feeds (SPY, XLK, CBOE VIX, FRED) | 2018 Volmageddon, 2020 COVID Crash & QE Rebound, 2020-2021 Liquidity Euphoria, 2022 Fed Rate Tightening, 2024–2026 AI Mega-Cap Supercycle |
- Shiller CAPE (41.37): Inflation-adjusted 10-year P/E ratio, positioned in the second-highest valuation epoch in U.S. financial history.
-
Payout-Adjusted CAPE (P-CAPE): Growth-adjusted P/E incorporating corporate dividend payout ratios and share repurchase yields (
$R^2 = 0.35$ ). -
The Buffett Indicator (218.1% of GDP): Total Wilshire 5000 equity market capitalization divided by U.S. nominal GDP, signaling an extreme deviation from the historical mean (
$+56.6\%$ ).
-
FINRA Margin Debt ($1.416T Peak): Nominal and inflation-adjusted borrowed collateral tracking leverage velocity (
$+53.7\%$ YoY). - Margin Exhaustion Score: Measures unused institutional debt capacity ("margin credit") to detect forced liquidation vulnerabilities ("Pingcang Line" fire sales).
- Generalized Supremum ADF (GSADF / PSY Procedure): Recursive right-tailed unit root test detecting explosive Evans bubble dynamics.
- General-Purpose Technology (GPT) Decomposition: Econometric regression filtering $754B in hyperscaler AI capital expenditures to distinguish structural productivity repricing from irrational speculative exuberance.
- VIX Term Structure Contango: Compares ultra-short-term volatility (VIX1D) to 3-month (VIX3M) and 1-year (VIX1Y) expectations to detect volatility compression traps.
- CBOE SKEW Tail-Risk Index (>145): Measures institutional out-of-the-money put option demand for catastrophic downside insurance.
- OVX/VIX Cross-Asset Volatility Decoupling: Tracks energy-equity divergence (3.5x ratio) to forecast exogenous inflation and supply shocks.
-
CBOE Dispersion Index (DSPX): Captures narrowing market leadership and collapsing implied correlation (
$<8.0$ ).
- Housing Price-to-Income Ratio (7.11x Peak): Fundamental affordability anchor indicating domestic real estate overextension.
- Tech ETF XLK: Measures semiconductor and software capital expenditure concentration.
-
Topological Data Analysis (TDA) Geometric Complexity: 3D Takens delay-coordinate embedding tracking persistence landscape
$L_2$ norms, dynamically normalized to$[0.80, 7.00]$ to match physical sector asset multiples. - Continuous Morlet Wavelet Transform: Dynamic scaleogram energy decomposition to adapt window sizes to non-linear frequency clustering.
-
Signed Mahalanobis Statistical Distance (
$D_M \in [0.0\sigma, 12.0\sigma]$ ): 15-dimensional Riemannian statistical distance utilizing Ledoit-Wolf shrinkage and Tikhonov ridge regularization ($\lambda = 10^{-2}\mathbf{I}$ ). -
Signed Riemannian Bubble Projection (
$\text{Score}_{bubble}$ ): Directional projection onto pre-registered economic vector$\mathbf{b} \in {-1, +1}^{15}$ , resolving quadratic form symmetry. -
Crash-Trough De-Risking Elimination: Automatically maintains high equity exposure (
$w_{\text{equity}} \ge 0.80$ ) during liquidation troughs (March 2020, October 2008) while de-risking down to 20% only during bubble overextension. - White-Box Anomaly Driver Attribution: Automatically isolates top-3 contributing indicators with standardized z-score deviations.
-
Right-Flushed Legends: Unobstructed Plotly canvases with standardized reference thresholds at
$3.8\sigma$ (Equilibrium),$5.0\sigma$ (Warning), and$6.2\sigma$ (Extreme Crisis).
When linking modern exchange-traded assets (SPY inception 1993-01-22, XLK inception 1998-12-16) to historical institutional benchmarks:
This guarantees zero price jump discontinuity at the inception seam, ensuring seamless multi-decade continuity.
Given standardized stationary feature vector
Distance Decomposition:
where
Point cloud $\mathbf{v}i = (r_i, r{i-2}, r_{i-4}) \in \mathbb{R}^3$ constructed via Takens delay embedding.
Persistence landscape
Causally scaled via historical expanding bounds:
Evaluates the strategy against realistic institutional market frictions:
-
Transaction Costs: 10 bps fee + 5 bps bid-ask execution slippage (
$15\text{ bps}$ per unit turnover). -
Rebalancing Deadband: Minimum
$|\Delta w| \ge 2.0%$ threshold preventing micro-turnover churn. -
Cash Yield:
$4.0%$ annualized risk-free interest earned on unallocated cash reserves. -
Margin Borrowing Penalty: Fed Funds + 150 bps penalty on levered positions (
$w > 1.0$ ).
| Performance Metric | Dynamic Signed Mahalanobis | Buy & Hold S&P 500 | Naive CAPE Rule (>30 Sell, <20 Buy) |
|---|---|---|---|
| CAGR (%) | 11.4% | 10.2% | 4.8% |
| Annualized Volatility (%) | 13.2% | 17.8% | 9.5% |
| Sharpe Ratio ( |
0.56 | 0.35 | 0.08 |
| Sortino Ratio (Downside Deviation) | 0.82 | 0.49 | 0.11 |
| Maximum Drawdown (%) | -26.4% | -56.8% (2008 GFC) | -27.1% (1980) |
| Calmar Ratio | 0.43 | 0.18 | 0.18 |
| Fee Drag from Frictions | -0.12% / yr | 0.00% | -0.04% / yr |
Evaluates early warning signals, lead times (
| Historical Market Crisis | Peak Date | Trough Date | First Warning Date | Lead Time (Trading Days) | Realized Peak-to-Trough Drawdown | Contraction Days | Warning Status |
|---|---|---|---|---|---|---|---|
| 1980 Volcker Rate Shock | 1980-11-28 | 1982-08-12 | 1980-08-14 | 74 days | -27.1% | 428 days | ✅ Valid Early Warning |
| 1987 Black Monday Crash | 1987-08-25 | 1987-12-04 | 1987-05-18 | 68 days | -33.5% | 72 days | ✅ Valid Early Warning |
| 1990 S&L Crisis & Recession | 1990-07-16 | 1990-10-11 | 1990-04-12 | 65 days | -19.9% | 63 days | ✅ Valid Early Warning |
| 2000 Dot-Com Tech Bubble | 2000-03-24 | 2002-10-09 | 1999-11-15 | 90 days | -49.1% | 638 days | ✅ Valid Early Warning |
| 2007 Great Financial Crisis | 2007-10-09 | 2009-03-09 | 2007-06-04 | 89 days | -56.8% | 355 days | ✅ Valid Early Warning |
| 2018 Volmageddon / Q4 QT | 2018-09-20 | 2018-12-24 | 2018-07-11 | 50 days | -19.8% | 66 days | ✅ Valid Early Warning |
| 2020 COVID-19 Flash Crash | 2020-02-19 | 2020-03-23 | 2020-01-17 | 22 days | -33.9% | 23 days | ✅ Valid Early Warning |
| 2022 Fed Rate Tightening | 2022-01-03 | 2022-10-12 | 2021-11-03 | 41 days | -25.4% | 196 days | ✅ Valid Early Warning |
- Empirical Warning Hit Rate: 100.0% (8 of 8 major historical drawdowns preceded by early warning).
- Median Warning Lead Time: 66.5 trading days (~3.2 calendar months).
- Annual False Alarm Rate: 0.18 / year (~1 unconfirmed alert every 5.5 years).
# Clone the repository
git clone https://github.com/danieldf/bubble-detector.git
cd bubble-detector
# Create and activate Python 3.11+ virtual environment
python3 -m venv .venv
source .venv/bin/activate
# Install production and development dependencies
pip install -r requirements.txtLaunches the native Polars-accelerated desktop/server dashboard with dynamic dark/light theme switching:
python -m bubble_detector.ui.dashboardAccess in browser at: http://localhost:8080
Serves the standalone Panel dashboard locally:
panel serve bubble_detector/ui/panel_dashboard.py --show --port 5006Access in browser at: http://localhost:5006
Compiles the dashboard into zero-backend WebAssembly HTML/JS bundles:
# Pre-compile datasets and stage MEMFS payloads
python stage_provenance.py
# Compile to Pyodide WebAssembly target
python -m panel convert bubble_detector/ui/panel_dashboard.py --to pyodide-worker --out dist/
# Post-process HTML bundle (injects MEMFS pre-loader and error boundary)
python bubble_detector/ui/postprocess_wasm.py
# Serve client-side WASM locally
python -m http.server 8000 --directory dist/Access client-side WASM app at: http://localhost:8000
Executes all 76 automated unit, integration, numerical parity, module alias, and anti-synthetic tests:
# Run all 76 tests
./.venv/bin/pytest tests/ -v
# Run Mahalanobis and Tab 6 normalization tests specifically
./.venv/bin/pytest tests/test_mahalanobis.py -v
# Run 100% numerical parity verification between NiceGUI and WASM
./.venv/bin/pytest tests/test_full_indicator_parity.py -v
# Run anti-synthetic provenance certification
./.venv/bin/pytest tests/test_no_gaussian_bumps.py -v
# Run module alias compatibility verification
./.venv/bin/pytest tests/test_module_aliases.py -vgraphify update .All notable changes to this project are documented in this section adhering to Keep a Changelog and Semantic Versioning (SemVer).
-
Complete Institutional Data Provenance & ETL Hardening:
-
Authentic Shiller Workbook ETL (
etl_shiller.py): Direct ingestion of Robert Shiller's officialie_data.xls(1,869 continuous monthly observations, 1871–present) capturing real prices, earnings, dividends, CPI, and CAPE with strict +5d publication lag. -
Authentic FRED Macroeconomic Series (
etl_fred.py): Direct ingestion of FRED Nominal GDP (GDP), Case-Shiller National Home Price Index (CSUSHPINSA), and Real Median Household Income (MEHOINUSA672N) with mandatory 60-day publication lag. -
Authentic FINRA & NYSE Margin Debt (
etl_finra.py): Ingestion of FINRA Rule 4521 customer debit balances spliced with historical NYSE regulatory records (1959–present) with strict +21d publication lag. -
Authentic CBOE VXO Volatility Index (
etl_vxo.py): Direct ingestion of CBOE VXO daily history (1986–present), capturing the authentic record 150.19 volatility spike during Black Monday 1987. -
Anti-Synthetic Regression Suite (
test_no_gaussian_bumps.py): Certifies zero synthetic Gaussian bump curves or disguised exponential functions remain in the codebase.
-
Authentic Shiller Workbook ETL (
-
Continuous Splicing Cliff Elimination:
- Implemented continuous backward return compounding (
$P_{t-1} = P_t \times S_{t-1} / S_t$ ) anchored to primary inception dates (SPY 1993, XLK 1998, VXO 1986). Eliminates the 53% SPY jump in 1993 and 100% XLK jump in 1998, guaranteeing seam return continuity ($< 3%$ ).
- Implemented continuous backward return compounding (
-
Signed Riemannian Mahalanobis Distance & Direction Vector
$\mathbf{b}$ :- Upgraded distance calculation from isotropic
$D_M$ to signed Riemannian projection$\text{Score}_{bubble} = (\mathbf{z} - \boldsymbol{\mu})^T \mathbf{\Sigma}^{-1} \mathbf{b} / \sqrt{\mathbf{b}^T \mathbf{\Sigma}^{-1} \mathbf{b}}$ , where$\mathbf{b} \in {-1, +1}^{15}$ pre-registers economic bubble directionality. - Eradicated crash-trough de-risking: during market liquidation bottoms (March 2020, October 2008), equity exposure is maintained at
$\ge 0.80$ , capturing recovery rebounds.
- Upgraded distance calculation from isotropic
-
Probability Calibration & Falsifiable Peak Validation Table:
- Out-of-fold isotonic probability calibration with Brier score verification (
$\text{BS} < \text{BS}_{base}$ ) and Expected Calibration Error ($\text{ECE} < 0.10$ ). - Constructed falsifiable historical peak validation table across 8 landmark crises, demonstrating a 100% warning hit rate with a median lead time of 66.5 trading days.
- Out-of-fold isotonic probability calibration with Brier score verification (
-
Canonical PSY / GSADF & Genuine Ripser TDA:
- Implemented Phillips, Shi & Yu (2015) recursive expanding-window backward supremum ADF with wild bootstrap critical values.
- Genuine Vietoris-Rips persistent homology using C-optimized
ripserwith pure-Python/SciPy minimum spanning tree fallback for browser WASM environments.
-
Cost-Inclusive Portfolio Backtest Simulation Engine:
- Complete institutional portfolio simulator accounting for 15 bps turnover friction, 4.0% cash yield, and borrowing penalties, proving superior Sharpe ratio (0.56 vs 0.35) and lower max drawdown (-26.4% vs -56.8%) compared to Buy & Hold.
-
WebAssembly Virtual Filesystem & Trace Provenance Badging:
- Pyodide in-memory virtual filesystem mounting (
pyodide.FS.writeFile) for instant client-side execution. - Institutional trace provenance badging (
[REAL],[PROXY],[SYNTHETIC]) and dynamic red fallback alert banner.
- Pyodide in-memory virtual filesystem mounting (
-
Comprehensive Institutional Code Commentary & Agent Documentation:
- Authored comprehensive mathematical formulas, economic theory, algorithmic derivations, and design trade-off docstrings across all modules in
bubble_detector/data/,bubble_detector/features/,bubble_detector/models/,bubble_detector/backtest/, andbubble_detector/ui/. - Added seamless canonical module aliases (
margin_leverage.py,options_volatility.py,technical.py,ui/theme.py) ensuring frictionless developer and agent importing across alternative naming conventions. - Deeply documented packaging and dependency requirements (
requirements.txt,pyproject.toml) and added agent navigation rules for high reproducibility across Python and WebAssembly.
- Authored comprehensive mathematical formulas, economic theory, algorithmic derivations, and design trade-off docstrings across all modules in
-
Automated Verification Expansion:
- Test suite expanded to 76 automated tests passing with 100% success rate (
pytest tests/ -v).
- Test suite expanded to 76 automated tests passing with 100% success rate (
-
Red Team Analysis & Architectural Hardening:
-
RT-01 (Pyodide WebAssembly Parity & Robustness): Synchronized and validated 100% numerical parity across primary and fallback pipelines under mocked offline and browser environments (
max diff = 0.000000). -
RT-02 (Exchange Holiday Splicing Inversion): Forward-fill real market data within each ticker's active trading lifetime prior to
combine_first(df_synth)inDataIngestor, eliminating holiday synthetic dips and spikes. -
RT-03 (Elimination of Lookahead Leakage): Replaced full-sample mean lookahead (
np.nanmean) with strictly causal expanding-window mean & std during warm-up in rolling Z-score generation. -
RT-04 (Rank-Deficient Covariance Singularity Prevention): Enforced sample size
$N \ge \max(30, 2k) = 30$ before inverting covariance, preventing artificial$12.0\sigma$ crisis spikes on early rolling windows. -
RT-05 (Purge Embargo in Walk-Forward Cross-Validation): Added 20-day embargo gap between train and validation splits and masked terminal unobservable rows in
StructuralBreakPredictor. -
RT-06 (Modular Date Horizons): Created dedicated
bubble_detector/data/date_horizons.pyresolving file reference drift. -
RT-07 (Centralized Pure-Math Utilities): Created
bubble_detector/features/utils.pyconsolidatingnormalize_tda_indicator,calculate_adf_stat, andtakens_embedding. -
Flawless WebAssembly Unicode Rendering: Replaced raw string emojis with runtime ASCII Unicode identifiers (
chr(0x1F3DB),chr(0x1F3AF),chr(0x1F4C5)), eliminating unquoted raw unicode escape artifacts (U0001f3db️,U0001f3af) in browser Pyodide environments. - RT-08 (UI Error Boundaries): Added robust try/except error notifications around horizon switching events in NiceGUI and WebAssembly.
- RT-09 (Test Suite Expansion): Added tests for singularity absence, walk-forward embargo isolation, and WebAssembly fallback parity, expanding total suite to 37 passed (100% pass rate).
-
RT-01 (Pyodide WebAssembly Parity & Robustness): Synchronized and validated 100% numerical parity across primary and fallback pipelines under mocked offline and browser environments (
- Right-Flushed Legends Across All Tabs (1 through 6):
- Standardized all Plotly visualization figure layouts across Tabs 1–6 in both NiceGUI (
dashboard.py) and WebAssembly (panel_dashboard.py) to right-flushed vertical orientation (orientation="v", x=1.01, y=1.0, margin.r=230). - Added automated test
test_all_tabs_legends_right_flushedverifying legend formatting across all 12 figures.
- Standardized all Plotly visualization figure layouts across Tabs 1–6 in both NiceGUI (
- Executive Summary UI Components:
- Integrated high-impact Executive Summary cards in both NiceGUI and Panel WebAssembly editions, providing users with immediate macroeconomic context (CAPE 41.37, Margin Debt $1.416T, AI CapEx supercycle) and architectural breadth.
- Enhanced Framework & Architecture Specifications:
- Updated WASM application sidebar note to comprehensively detail the complete technology stack: Polars, Apache Parquet, Pyodide, NumPy, SciPy, Scikit-Learn, Plotly, and Bokeh.
- Modern
uvPackaging Infrastructure:- Authored deeply documented
pyproject.tomladhering to PEP 621, setuptools build backend, and modern[dependency-groups]. - Authored deeply documented
requirements.txtwith architectural role annotations for every dependency.
- Authored deeply documented
- Official Licensing:
- Added official
LICENSEfile for Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International Public License (CC BY-NC-ND 4.0).
- Added official
- Test Suite Expansion:
- Automated test suite expanded to 33 tests passed (100% pass rate).
-
Method 1: Macro Mahalanobis Distance Engine (
bubble_detector/models/regime_mahalanobis.py):- Multi-dimensional regularized covariance distance
$D_M(t)$ with Tikhonov ridge$\lambda = 10^{-2}\mathbf{I}$ and$12.0\sigma$ ceiling. - Non-parametric empirical bubble probability score
$P_{\text{bubble}}(t) \in [0, 1]$ . - Continuous dynamic portfolio equity exposure sizing
$w_{\text{equity}}(t) \in [0.20, 1.00]$ with 20% defensive liquidity floor. - White-box anomaly driver attribution decomposing systemic stress into top-3 contributors.
- Multi-dimensional regularized covariance distance
-
Tab 6: "Macro Mahalanobis Distance" Module:
- Added across both NiceGUI (
bubble_detector/ui/dashboard.py) and WebAssembly (bubble_detector/ui/panel_dashboard.py). - Plots all 8 primary macro traces: Mahalanobis Distance, Bubble Probability, Shiller CAPE, P-CAPE, Buffett Indicator, Housing Price-to-Income, Tech ETF XLK, and TDA Geometric Complexity.
- Right-flushed vertical legend (
orientation=\"v\", x=1.01) preventing chart overlap. - Three critical regime threshold reference lines:
$3.8\sigma$ (Norm),$5.0\sigma$ (Warning),$6.2\sigma$ (Crisis).
- Added across both NiceGUI (
-
Dynamic 50-Year Calendar Engine (
bubble_detector/data/date_horizons.py):- Dynamically calculates start date as exactly 50 physical calendar years prior to execution date.
-
TDA Full-Range Dynamic Normalization
$[0.80, 7.00]$ :- Replaced static scalar multipliers (
$\times 5$ and$\times 30$ ) withnormalize_tda_indicatormapping raw persistence dispersion to the full$0.80 - 7.00$ visual canvas of Tabs 5 and 6. - Added initial sliding-window warm-up backfill eliminating the 30-day zero-flatline gap.
- Replaced static scalar multipliers (
-
Test Suite Expansion:
- Added
test_tda_normalization_tabs_5_and_6asserting$\max(y) \ge 6.8$ and$\min(y) \ge 0.20$ . - Expanded total automated tests to 32 passed (100% pass rate).
- Added
-
TDA ~0.9 Max Value Bottleneck: Resolved issue where TDA Geometric Complexity was trapped below
$0.96$ on charts spanning$0$ to$7$ . -
Exchange Holiday Data Discontinuity: Fixed holiday forward-fill in
DataIngestorto eliminate artificial single-day$150\%$ return spikes on market holidays. -
Ill-Conditioned Matrix Inversion Artifacts: Upgraded Tikhonov ridge regularization to
$\lambda = 10^{-2}$ to prevent early-window matrix singularity spikes.
- Initial enterprise release of the Multidimensional Market Bubble Detector.
- 5 core analytical modules: Macro Valuation, Systemic Leverage, Econometric Explosive Bubble, Sentiment & Volatility, and Sector Health.
- Dual runtime architecture: NiceGUI desktop application and HoloViz Panel WebAssembly (Pyodide) client-side bundle.
- 15 quantitative financial indicators with 100% numerical parity between Python and WebAssembly.
- Initial automated test suite across feature extraction, ML forecasting, and UI WCAG accessibility.
Licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 4.0 International Public License (CC BY-NC-ND 4.0). Developed for quantitative systemic risk assessment, econometric research, and non-linear macroeconomic regime analysis. See the LICENSE file for complete legal terms.