I have mentioned Stein’s method in passing, a few times on this blog. Today I want to talk about Stein’s method in a bit of detail.

**1. What Is Stein’s Method? **

Stein’s method, due to Charles Stein, is actually quite old, going back to 1972. But there has been a great deal of interest in the method lately. As in many things, Stein was ahead of his time.

Stein’s method is actually a collection of methods for showing that the distribution of some random variable is close to Normal. What makes Stein’s method so powerful is that it can be used in a variety of situations (including cases with dependence) and it tells you how far is from Normal.

I will follow Chen, Goldstein and Shao (2011) quite closely.

**2. The Idea **

Let be iid with mean 0 and variance 1. Let

and let . We want to bound

where is the cdf of a standard Normal. For simplicity, we will assume here

but this is not needed. The key idea is this:

for all smooth if and only if . This suggests the following idea: if we can show that is close to 0, then should be almost Normal.

More precisely, let and let be any function such that . The Stein function associated with is a function satisfying

It then follows that

and showing that is close to normal amounts to showing that is small.

Is there such an ? In fact, letting , the Stein function is

More precisely, is the unique solution to the above equation subject to the side condition .

Let’s be more specific. Choose any and let . Let denote the Stein function for ; thus satisfies

The unique bounded solution to this equation is

The function has the following properties:

where . Also,

Let . From the equation it follows that

and so

We have reduced the problem of bounding to the problem of bounding .

**3. Zero Bias Coupling **

As I said, Stein’s method is really a collection of methods. One of the easiest to explain is “zero-bias coupling.”

Let . Recall that are iid, mean 0, variance 1. Let

where

For each define the leave-one-out quantity

which plays a crucial role. Note that is independent of . We also make use of the following simple fact: for any and , since the density of the Normal is bounded above by .

Recall that if and only if

for all absolutely continuous functions (for which the expectations exists). Inspired by this, Goldstein and Reinert (1997) introduced the following definition. Let be any mean 0 random variable with variance . Say that has the *-zero bias distribution* if

for all absolutely continuous functions for which . Zero-biasing is a transform that maps one random variable into another random variable . (More precisely, it maps the distribution of into the distribution of .) The Normal is the fixed point of this map. The following result shows that exists and is unique.

Theorem 1Let be any mean 0 random variable with variance . There exists a unique distribution corresponding to a random variable such thatfor all absolutely continuous functions for which . The distribution of has density

*Proof:* It may be verified that and integrates to 1. Let us verify that (2) holds. For simplicity, assume that . Given an absolutely continuous there is a such that . Then

Similarly,

Here is a way to construct explicitly when dealing with a sum.

Lemma 2Let be independent, mean 0 random variables and let . Let . Let be independent and zero-bias. Define

where . Then has the -zero bias distribution. In particular, suppose that have mean 0 common variance and let . Let be a random integer from 1 to . Then has the -zero bias distribution.

Now we can prove a Central Limit Theorem using zero-biasing.

Theorem 3Suppose that . Let . Then

*Proof:* Let be independent random variables where is zero-bias for . Let be chosen randomly from and let

Then, by the last lemma, is zero-bias for and hence

for all absolutely continuous . Also note that

So,

By a symmetric argument, we deduce that

Let . From the properties of the Stein function given earlier we have

Combining these inequalities we have

**4. Conclusion **

This is just the tip of the iceberg. If you want to know more about Stein’s method, see the references below. I hope I have given you a brief hint of what it is all about.

** References **

Chen, Goldstein and Shao. (2011). *Normal Approximation by Stein’s Method.* Springer.

Nourdin and Peccati (2012). *Normal Approximations With Malliavin Calculus.* Cambridge.

Ross, N. (2011). Fundamentals of Stein’s method . *Probability Surveys*, 210-293. link

## 10 Comments

The wikipedia link at the start of the article is missing an apostrophe, it should link to http://en.wikipedia.org/wiki/Stein%27s_method

I don’t understand the start of section 3. You say that \xi_i = 1/\sqrt{n} X_i and X^i = X_i-\xi_i. So X^i = (1-1/\sqrt{n}) X_i, which is most certainly not independent of X_i. Did you mean X^i = X – \xi_i?

yes on both

i made the corrections

thanks

Larry: Something weird has happened to the LaTeX near the end of the article. The aspect ratio of the characters is screwed up making the formulas hard to read.

It looks fine in my browser.

Maybe it is browser dependent?

My intuition says that one could use Stein’s method to check on the closeness of a RV’s distribution to *any* other distribution, not just Normal, as follows: Suppose we want to check if a random variable X’ has a distribution close to F. Define Z’ as a random variable with distribution F and quantile function q; then Z = Psi(q(Z’)) has a Normal distribution. Does it not follow that X’ will have distribution close to F if X = Psi(q(X’)) has a distribution close to Normal?

Whoops, I think I got the bijection backwards: it should be Z = q(F(Z’)), where q is now the quantile function of the Normal distribution.

Yes Stein’s method extends to other distributions too.

One has to develop the correct differential equation.

There is a chapter on this in the Chen, Goldstein, Shao book.

The Wikipedia page for Stein’s method mentions that it can be used to prove the central limit theorem – does it have any other particularly useful applications?

It is used to prove that the distribution of some random variable X_n

is close to its limiting distribution.

Normal is a special case.

To what other distributions the Stein’s method is applicable to?

## 3 Trackbacks

[…] Stein’s Method […]

[…] Stein’s Method (normaldeviate.wordpress.com) […]

[…] Stein’s Method […]