Overview
Silverman’s rule-of-thumb provides a simple, data-driven bandwidth selection method for kernel density estimation by assuming the underlying distribution is normal. This allows us to compute the unknown quantity \(R(f'')\) analytically and derive a closed-form bandwidth that depends only on the sample standard deviation and sample size.
Step 1: Write the Normal Density and Its Derivatives
Let \(X \sim \mathcal{N}(0, \sigma^2)\), so the probability density function is
\[
f(x) = \frac{1}{\sigma \sqrt{2\pi}} \exp\!\left(-\frac{x^2}{2\sigma^2}\right).
\]
First Derivative:
\[
f'(x) = \frac{d}{dx}\left[\frac{1}{\sigma\sqrt{2\pi}} \exp\!\left(-\frac{x^2}{2\sigma^2}\right)\right]
= -\frac{x}{\sigma^3\sqrt{2\pi}} \exp\!\left(-\frac{x^2}{2\sigma^2}\right).
\]
Second Derivative:
Using the product rule on \(f'(x) = -\frac{x}{\sigma^3\sqrt{2\pi}} e^{-x^2/(2\sigma^2)}\):
\[
f''(x) = \frac{d}{dx}\left[-\frac{x}{\sigma^3\sqrt{2\pi}} e^{-x^2/(2\sigma^2)}\right]
\]
\[
= -\frac{1}{\sigma^3\sqrt{2\pi}} e^{-x^2/(2\sigma^2)}
+ \frac{x^2}{\sigma^5\sqrt{2\pi}} e^{-x^2/(2\sigma^2)}
\]
\[
= \frac{1}{\sigma^5\sqrt{2\pi}} (x^2 - \sigma^2) \exp\!\left(-\frac{x^2}{2\sigma^2}\right).
\]
Step 2: Compute \(R(f'') = \int_{-\infty}^{\infty} [f''(x)]^2 \, dx\)
Squaring \(f''(x)\):
\[
[f''(x)]^2 = \frac{1}{\sigma^{10} \cdot 2\pi} (x^2 - \sigma^2)^2 \exp\!\left(-\frac{x^2}{\sigma^2}\right).
\]
Therefore,
\[
R(f'') = \int_{-\infty}^{\infty} [f''(x)]^2 \, dx
= \frac{1}{2\pi \sigma^{10}} \int_{-\infty}^{\infty} (x^2 - \sigma^2)^2 \exp\!\left(-\frac{x^2}{\sigma^2}\right) dx.
\]
Change of Variables: Let \(z = \frac{x}{\sigma}\), so \(x = \sigma z\) and \(dx = \sigma \, dz\):
\[
R(f'') = \frac{1}{2\pi\sigma^{10}} \int_{-\infty}^{\infty} (\sigma^2 z^2 - \sigma^2)^2 e^{-z^2} \sigma \, dz
\]
\[
= \frac{1}{2\pi\sigma^{10}} \cdot \sigma^4 \cdot \sigma \int_{-\infty}^{\infty} (z^2 - 1)^2 e^{-z^2} dz
\]
\[
= \frac{1}{2\pi\sigma^{5}} \int_{-\infty}^{\infty} (z^2 - 1)^2 e^{-z^2} dz.
\]
Expand \((z^2 - 1)^2\):
\[
(z^2 - 1)^2 = z^4 - 2z^2 + 1.
\]
So we need:
\[
\int_{-\infty}^{\infty} (z^4 - 2z^2 + 1) e^{-z^2} dz
= \int_{-\infty}^{\infty} z^4 e^{-z^2} dz
- 2\int_{-\infty}^{\infty} z^2 e^{-z^2} dz
+ \int_{-\infty}^{\infty} e^{-z^2} dz.
\]
Using Standard Gaussian Integrals:
Recall that for \(n\) even, \[
\int_{-\infty}^{\infty} z^n e^{-z^2} dz = \frac{(n-1)!!}{2^{n/2}} \sqrt{\pi},
\] where \((n-1)!! = (n-1)(n-3)\cdots 3 \cdot 1\).
- For \(n=0\): \(\int e^{-z^2} dz = \sqrt{\pi}\)
- For \(n=2\): \(\int z^2 e^{-z^2} dz = \frac{1}{2}\sqrt{\pi}\)
- For \(n=4\): \(\int z^4 e^{-z^2} dz = \frac{3}{4}\sqrt{\pi}\)
Therefore:
\[
\int_{-\infty}^{\infty} (z^2-1)^2 e^{-z^2} dz = \frac{3\sqrt{\pi}}{4} - 2 \cdot \frac{\sqrt{\pi}}{2} + \sqrt{\pi}
= \frac{3\sqrt{\pi}}{4} - \sqrt{\pi} + \sqrt{\pi} = \frac{3\sqrt{\pi}}{4}.
\]
Substitute back:
\[
R(f'') = \frac{1}{2\pi\sigma^5} \cdot \frac{3\sqrt{\pi}}{4} = \frac{3}{8\sqrt{\pi}\sigma^5}.
\]
Step 4: Evaluate \(C_K\) for the Gaussian Kernel
For the Gaussian kernel:
\[
K(u) = \frac{1}{\sqrt{2\pi}} \exp\!\left(-\frac{u^2}{2}\right),
\]
we need to compute \(R(K)\) and \(\mu_2(K)\).
Compute \(R(K) = \int K(u)^2 \, du\):
\[
K(u)^2 = \frac{1}{2\pi} \exp(-u^2).
\]
\[
R(K) = \int_{-\infty}^{\infty} \frac{1}{2\pi} e^{-u^2} du = \frac{1}{2\pi} \cdot \sqrt{\pi} = \frac{1}{2\sqrt{\pi}}.
\]
Compute \(\mu_2(K) = \int u^2 K(u) \, du\):
\[
\mu_2(K) = \int_{-\infty}^{\infty} u^2 \cdot \frac{1}{\sqrt{2\pi}} e^{-u^2/2} du.
\]
This is the second moment of a standard normal distribution, so:
\[
\mu_2(K) = 1.
\]
Substitute into \(C_K\):
\[
C_K = \left(\frac{8\sqrt{\pi} \cdot \frac{1}{2\sqrt{\pi}}}{3 \cdot 1^2}\right)^{1/5}
= \left(\frac{8\sqrt{\pi}}{2\sqrt{\pi} \cdot 3}\right)^{1/5}
= \left(\frac{4}{3}\right)^{1/5}.
\]
Numerical evaluation:
\[
\left(\frac{4}{3}\right)^{1/5} = (1.3333...)^{0.2} \approx 1.0593.
\]
Therefore:
\[
C_K \approx 1.059
\]
and Silverman’s rule-of-thumb bandwidth for the Gaussian kernel is:
\[
h_r = 1.059 \, \sigma \, n^{-1/5}.
\]
