Lesson 6.8: Advanced Concept - Fractional Differentiation
In this final lesson of the module, we explore a sophisticated technique from modern quantitative finance. Fractional differentiation, popularized by Dr. Marcos Lopez de Prado, offers a way to achieve stationarity without completely erasing the 'memory' of a time series, providing a potential solution to the classic stationarity vs. memory tradeoff.
Part 1: The Stationarity vs. Memory Dilemma
We've learned that we must difference a non-stationary series (like a stock price) to make it stationary before we can model it. A standard first-difference () creates a series of returns.
The Problem: While differencing achieves stationarity, it also destroys a significant amount of information. The series of returns has almost no "memory" of the original price level. We have thrown the baby out with the bathwater. This is particularly problematic for mean-reversion strategies, which rely on the memory of a long-term equilibrium level.
Is there a middle ground? Can we apply "just enough" differencing to make the series stationary, but not so much that we erase its memory?
Part 2: The Fractional 'd'
The solution is to allow the differencing order, `d`, to be a fractional number instead of just an integer.
Fractional Differencing
Instead of choosing between (no differencing) and (full differencing), fractional differencing allows us to choose a value in between, like or .
The goal is to find the **smallest possible `d`** that makes the resulting series stationary (i.e., passes the ADF test). This creates a series that is:
- Stationary: Suitable for use in ML models.
- Memory-Preserving: The correlation between the fractionally differenced series and the original price series is maximized.
Part 3: The Payoff
Using a fractionally differenced series as features in a machine learning model can lead to significantly better performance, especially for mean-reversion strategies, because the features retain the crucial information about the long-term equilibrium that standard integer differencing destroys.
Congratulations! You Have Completed Module 6
You have now completed a comprehensive tour of time series analysis, from classical ARIMA/GARCH models to the practical application of modern machine learning techniques. You understand the critical importance of stationarity, the pitfalls of backtesting, and the sophisticated feature engineering required to build robust forecasting models.
What's Next in Your Journey?
It's time to enter the world of Deep Learning. In **Module 7: The Neuron's Spark**, we will build neural networks from the ground up, starting with the single neuron and building to multi-layer perceptrons, and we will finally demystify the "backpropagation" algorithm that powers it all.