Lesson 1.8: The Continuous World: PDFs and Smooth CDFs

We now cross the bridge from the discrete (counts) to the continuous (measurements). Here, the old tools break down. We introduce the Probability Density Function (PDF), where probability is AREA, and revisit the CDF, now as a smooth curve defined by calculus. Mastering this is the key to understanding all continuous models, including the famous Normal distribution.

Part 1: The Problem with Continuous Variables

The Core Idea: In the discrete world, we could assign probability "mass" to a specific point. In the continuous world, there are infinitely many points. The probability of landing on any *single exact point* is zero. We must shift our thinking from probability at a point to density over a region.

Consider a stock's daily return, $X$ . What is the probability that the return is *exactly* 1.25%? Not 1.250001% or 1.249999%, but exactly 1.25000...% with infinite precision?

P(X = 1.25) = 0

This means our old friend, the PMF, is useless here. We need a new tool: the **Probability Density Function (PDF)**.

The PDF: Where Area is Probability

f_X(x)

is a curve where the area underneath it corresponds to probability.

Imagine a bell curve. The area shaded between x=a and x=b is the probability.

Definition: Probability Density Function (PDF)

The PDF, $f_X(x)$ , is a function where the probability of $X$ falling between $a$ and $b$ is the **integral (the area)** of the function from $a$ to $b$ .

P(a \le X \le b) = \int_{a}^{b} f_X(x) \, dx

The Rules of a Valid PDF

A function is a valid PDF only if it meets two conditions:

Always Non-Negative: $f_X(x) \ge 0$ for all $x$ . (Density can't be negative).
Total Area is One: $\int_{-\infty}^{\infty} f_X(x) \, dx = 1$ . (The total probability of all outcomes must be 1).

Part 2: The PDF and CDF: Two Sides of the Same Coin

In the discrete world, the CDF was a staircase built from the PMF's bars. In the continuous world, the CDF is a smooth ramp built by the PDF's curve. They are intrinsically linked by the **Fundamental Theorem of Calculus**.

From PDF to CDF (Integrate)

The CDF, $F_X(x)$ , is the total accumulated area under the PDF from the very beginning up to the point $x$ .

F_X(x) = P(X \le x) = \int_{-\infty}^{x} f_X(t) \, dt

From CDF to PDF (Differentiate)

The PDF represents the *rate of change* of cumulative probability. It's the slope of the CDF at any given point.

f_X(x) = \frac{d}{dx} F_X(x) = F_X'(x)

Derivation: Using the CDF to Find Probabilities

Let's prove the most useful property of the CDF: calculating the probability of an interval.

P(a < X \le b) = P(X \le b) - P(X \le a)

Step 1: By definition, $P(X \le b) = F_X(b)$ and $P(X \le a) = F_X(a)$ .

Step 2: In terms of integrals, this is:

\int_{-\infty}^{b} f_X(x)dx - \int_{-\infty}^{a} f_X(x)dx

Step 3: By the properties of definite integrals, subtracting the area up to $a$ from the area up to $b$ leaves just the area between $a$ and $b$ .

= \int_{a}^{b} f_X(x)dx

Conclusion: $P(a < X \le b) = F_X(b) - F_X(a)$ . This is the continuous equivalent of the rule we learned in Lesson 1.4.

Part 3: Why This Matters for Quants and ML

Quant Finance: The Engine of Maximum Likelihood Estimation (MLE)

The PDF is the absolute heart of MLE for continuous data (Module 3). When you have a series of stock returns $R_1, R_2, \dots, R_n$ , you assume they are drawn from a distribution with a certain PDF, like the Normal PDF $f(R_i | \mu, \sigma^2)$ .

The **Likelihood Function** is the joint PDF of all your data points. You find the parameters ( $\mu, \sigma^2$ ) that maximize the value of this function. Without a correctly specified PDF, MLE is impossible.

\mathcal{L}(\mu, \sigma^2) = \prod_{i=1}^n f(R_i | \mu, \sigma^2)

Machine Learning: Quantiles, Percentiles, and Risk

The CDF is the direct gateway to understanding the risk and ranking of continuous predictions.

A risk model might output a continuous loss distribution for a portfolio. The risk manager needs to know the **Value-at-Risk (VaR)**, which is simply a percentile of this distribution. To find the 99% VaR, they must solve for $x$ in the equation:

F_X(x) = 0.01

This requires using the **Inverse CDF** (also called the quantile function). This is one of the most common operations in risk management.

What's Next? Finding the Center in a Continuous World

We have successfully defined the functions that govern the continuous world. We've replaced summation with integration.

Now, we must update our methods for calculating the key summary statistics. How do we find the "center of gravity" (Expected Value) or the "spread" (Variance) when we're dealing with a smooth curve instead of discrete weights?

The next lesson, **Expected Value and Variance via Integration**, will do exactly that.

Lesson 1.7: The Master Tool: Moment Generating Functions (MGFs)

Lesson 1.9: The Calculus of Center and Spread