I have mentioned Stein’s method in passing, a few times on this blog. Today I want to talk about Stein’s method in a bit of detail.

**1. What Is Stein’s Method? **

Stein’s method, due to Charles Stein, is actually quite old, going back to 1972. But there has been a great deal of interest in the method lately. As in many things, Stein was ahead of his time.

Stein’s method is actually a collection of methods for showing that the distribution of some random variable is close to Normal. What makes Stein’s method so powerful is that it can be used in a variety of situations (including cases with dependence) and it tells you how far is from Normal.

I will follow Chen, Goldstein and Shao (2011) quite closely.

**2. The Idea **

Let be iid with mean 0 and variance 1. Let

and let . We want to bound

where is the cdf of a standard Normal. For simplicity, we will assume here

but this is not needed. The key idea is this:

for all smooth if and only if . This suggests the following idea: if we can show that is close to 0, then should be almost Normal.

More precisely, let and let be any function such that . The Stein function associated with is a function satisfying

It then follows that

and showing that is close to normal amounts to showing that is small.

Is there such an ? In fact, letting , the Stein function is

More precisely, is the unique solution to the above equation subject to the side condition .

Let’s be more specific. Choose any and let . Let denote the Stein function for ; thus satisfies

The unique bounded solution to this equation is

The function has the following properties:

where . Also,

Let . From the equation it follows that

and so

We have reduced the problem of bounding to the problem of bounding .

**3. Zero Bias Coupling **

As I said, Stein’s method is really a collection of methods. One of the easiest to explain is “zero-bias coupling.”

Let . Recall that are iid, mean 0, variance 1. Let

where

For each define the leave-one-out quantity

which plays a crucial role. Note that is independent of . We also make use of the following simple fact: for any and , since the density of the Normal is bounded above by .

Recall that if and only if

for all absolutely continuous functions (for which the expectations exists). Inspired by this, Goldstein and Reinert (1997) introduced the following definition. Let be any mean 0 random variable with variance . Say that has the *-zero bias distribution* if

for all absolutely continuous functions for which . Zero-biasing is a transform that maps one random variable into another random variable . (More precisely, it maps the distribution of into the distribution of .) The Normal is the fixed point of this map. The following result shows that exists and is unique.

Theorem 1Let be any mean 0 random variable with variance . There exists a unique distribution corresponding to a random variable such thatfor all absolutely continuous functions for which . The distribution of has density

*Proof:* It may be verified that and integrates to 1. Let us verify that (2) holds. For simplicity, assume that . Given an absolutely continuous there is a such that . Then

Similarly,

Here is a way to construct explicitly when dealing with a sum.

Lemma 2Let be independent, mean 0 random variables and let . Let . Let be independent and zero-bias. Define

where . Then has the -zero bias distribution. In particular, suppose that have mean 0 common variance and let . Let be a random integer from 1 to . Then has the -zero bias distribution.

Now we can prove a Central Limit Theorem using zero-biasing.

Theorem 3Suppose that . Let . Then

*Proof:* Let be independent random variables where is zero-bias for . Let be chosen randomly from and let

Then, by the last lemma, is zero-bias for and hence

for all absolutely continuous . Also note that

So,

By a symmetric argument, we deduce that

Let . From the properties of the Stein function given earlier we have

Combining these inequalities we have

**4. Conclusion **

This is just the tip of the iceberg. If you want to know more about Stein’s method, see the references below. I hope I have given you a brief hint of what it is all about.

** References **

Chen, Goldstein and Shao. (2011). *Normal Approximation by Stein’s Method.* Springer.

Nourdin and Peccati (2012). *Normal Approximations With Malliavin Calculus.* Cambridge.

Ross, N. (2011). Fundamentals of Stein’s method . *Probability Surveys*, 210-293. link