Laplace Distribution

A sharp-peaked, fat-tailed alternative to the Normal Distribution.

The "Double Exponential" Distribution

The Laplace distribution is a continuous probability distribution that is notable for its sharper peak at the mean and its "fatter" tails compared to the Normal distribution. This means it assigns higher probability to values near the mean and also to extreme outlier events.

In finance and machine learning, this makes it a valuable tool. It can model financial returns that are more prone to extreme events than a normal model would suggest. It is also intrinsically linked to LASSO (L1) regularization, a popular technique in regression for feature selection, because its shape naturally encourages some parameters to go to zero.

Interactive Laplace Distribution

Adjust the location (μ) and scale (b) parameters to see how the distinctive shape of the distribution changes.

Location (μ): 0.0

Scale (b): 1.0

Mean (

\mu

): 0.00

Variance (

\sigma^2

): 2.00

Core Concepts

Probability Density Function (PDF)

f(x | \mu, b) = \frac{1}{2b} \exp\left( -\frac{|x - \mu|}{b} \right)

$\mu$ (mu) is the location parameter, which is also the mean, median, and mode.
$b > 0$ is the scale parameter, which controls the spread or "width" of the distribution.

Expected Value & Variance

Expected Value (Mean)

E[X] = \mu

Variance

Var(X) = 2b^2

Key Derivations

Deriving the Mean & Variance

The mean is straightforward due to symmetry. The variance requires integration by parts.

Deriving the Expected Value (Mean)

Step 1: Use Symmetry

The Laplace PDF is symmetric around $\mu$ . The function $g(x) = (x-\mu)f(x | \mu, b)$ is an odd function with respect to $x=\mu$ . The integral of an odd function over a symmetric interval is zero.

E[X-\mu] = \int_{-\infty}^{\infty} (x-\mu) f(x | \mu, b) dx = 0

Step 2: Solve for E[X]

Using the linearity of expectation, $E[X-\mu] = E[X] - E[\mu] = E[X] - \mu$ .

Therefore, $E[X] - \mu = 0$ , which leads to our result.

Final Mean Formula

E[X] = \mu

Deriving the Variance

We need to solve $Var(X) = E[(X-\mu)^2] = \int_{-\infty}^{\infty} (x-\mu)^2 f(x | \mu, b) dx$ . Let $y=x-\mu$ and use the PDF for $\mu=0$ for simplicity.

Step 1: Set up the Integral for E[Y²]

E[Y^2] = \int_{-\infty}^{\infty} y^2 \frac{1}{2b} e^{-|y|/b} dy

Since the integrand is an even function, we can simplify this to:

E[Y^2] = 2 \int_{0}^{\infty} y^2 \frac{1}{2b} e^{-y/b} dy = \frac{1}{b} \int_{0}^{\infty} y^2 e^{-y/b} dy

Step 2: Apply Integration by Parts (First Pass)

Let $u = y^2$ and $dv = e^{-y/b} dy$ . Then $du = 2y dy$ and $v = -b e^{-y/b}$ . Using $\int u dv = uv - \int v du$ :

\frac{1}{b} \left( \left[ -by^2 e^{-y/b} \right]_0^\infty - \int_0^\infty (-b e^{-y/b})(2y) dy \right)

The first term evaluates to 0. We are left with:

= \frac{1}{b} \int_0^\infty 2by e^{-y/b} dy = 2 \int_0^\infty y e^{-y/b} dy

Step 3: Apply Integration by Parts (Second Pass)

We integrate $\int_0^\infty y e^{-y/b} dy$ . Let $u=y$ and $dv=e^{-y/b}dy$ . Then $du=dy$ and $v=-be^{-y/b}$ .

\left[ -by e^{-y/b} \right]_0^\infty - \int_0^\infty (-b e^{-y/b}) dy

The first term is 0. The second term is:

b \int_0^\infty e^{-y/b} dy = b \left[ -b e^{-y/b} \right]_0^\infty = b (-0 - (-b)) = b^2

Step 4: Combine Results

Substituting the result from Step 3 back into the end of Step 2:

E[Y^2] = 2 \times (b^2) = 2b^2

Since $Y = X-\mu$ and $E[Y]=0$ , we have $Var(X) = Var(Y) = E[Y^2] - (E[Y])^2 = 2b^2 - 0$ .

Final Variance Formula

Var(X) = 2b^2

Applications

Machine Learning: LASSO Regression

In Bayesian statistics, using a Laplace distribution as the prior for regression coefficients is equivalent to performing LASSO (L1) regularization. The sharp peak at zero in the Laplace prior "encourages" coefficients of irrelevant features to be exactly zero, effectively performing feature selection and creating a simpler, more interpretable model.