Comments on: Robins and Wasserman Respond to a Nobel Prize Winner

By: TO CONDITION, OR NOT TO CONDITION, THAT IS THE QUESTION « Normal Deviate

TO CONDITION, OR NOT TO CONDITION, THAT IS THE QUESTION « Normal Deviate — Sun, 06 Jan 2013 16:29:33 +0000

[...] There are many other example such as this one. [...]

By: normaldeviate

normaldeviate — Sun, 02 Sep 2012 21:31:22 +0000

Dear Readers:
Since our reply is rather long, we are posting it as a new blog post. Please see:

http://normaldeviate.wordpress.com/2012/09/02/robins-and-wasserman-respond-to-a-nobel-prize-winner-continued-a-counterexample-to-bayesian-inference/

–LW

By: Robins and Wasserman Respond to a Nobel Prize Winner Continued: A Counterexample to Bayesian Inference? « Normal Deviate

Sun, 02 Sep 2012 21:28:30 +0000

[...] Normal Deviate Thoughts on Statistics and Machine Learning Skip to content About « Robins and Wasserman Respond to a Nobel Prize Winner [...]

By: Chris Sims

Chris Sims — Fri, 31 Aug 2012 20:12:49 +0000

@normaldeivate: Larry: The fact that $\theta$ and $\pi$ both depend on $X$ does not mean that, before we observe any data, they are dependent. After all, all the random variables in the problem are functions of the state in an underlying probability space. Here $X$ might have two components, $X_1$ and $X_2$, that are independent, with $\pi$ a function of $X_1$ alone and $\theta$ a function of $X_2$ alone. And in this case the distribution of $\theta \mid \pi$ would not depend on $\pi$. So it is certainly possible to have a prior that makes $W(\theta\mid\pi)=W(\theta)$. It’s just that here that amounts to assuming at the outset that selection bias is not a problem, while the whole point of the example is that it \emph{is} a problem.

By: Chris Sims

Chris Sims — Fri, 31 Aug 2012 02:34:38 +0000

I should have pointe out that round 3 is, like round 2, in the http://sims.princeton.edu/yftp/WassermanExmpl directory.

By: Chris Sims

Chris Sims — Fri, 31 Aug 2012 02:27:40 +0000

I’ve written up a “round 3″ comment, outlining a Bayesian approach to the problem with X observed. As you’ll see, it’s a lot like what I proposed in round 2, though I also suggest informally how to exploit smoothness if one thinks that likely. The main point again is that the prior on theta, which is a stochastic process with X as index, must have an unknown mean parameter, so that the prior does not implicitly assert precise knowledge of the parameter of interest, psi, before looking at the data.

By: Keith O'Rourke

Keith O'Rourke — Thu, 30 Aug 2012 00:45:38 +0000

Looking bleak, but just for the benefit of any students that might not know

Rather than conditioning on ((p(x),pi(x)), O(X,R,YR)) {p,px being known functions}
what prevents one from conditioning instead on ((p(x),pi(x),MN(Y.i,p.IS(y)),O(X,R)) ?
{MN(Y.i,p.IS(y)) being the “known” importance sampling distribution of Y.i given pi(x.i) is known}

That is, trying to match the use of the importance sampling distribution by the Horwitz-Thompson estimator
by conditioning on that (perhaps stretching the meaning of known.)

By: Entsophy

Entsophy — Wed, 29 Aug 2012 23:17:18 +0000

I take it back. The dependance on pi drops out. Ok, I’m convinced any natural prior for theta wont depend on pi.

By: Entsophy

Entsophy — Wed, 29 Aug 2012 22:37:38 +0000

Well I don’t deny it, but I’ll try my point again from a more concrete angle:

Given the conditions in the problem you can write the joint in the following functional way:

p(x,y,r)=F(y, r, theta(x), pi(x))

for some function F. Given this relationship for a known pi(x) what is a reasonable prior for theta(x)?

We’ll here’s on natural criterion: theta_1 will have a higher prior probability than theta_2 if the entropy of the resulting p(x,y,r) is higher for theta_1 than it is for theta_2.

This natural criterion together with the functional relation F will make the prior for theta depend on pi! I’ll also note that it is easy to calulate F() and the resulting entropy of p(x,y,r) in terms of y,r,theta, and pi. So you can see the dependancy explicitly.

By: normaldeviate

normaldeviate — Wed, 29 Aug 2012 21:33:58 +0000

Chris
Do you agree that the random variables and can be dependent (since they are both functions of X)
while the prior can make them independent functions i.e.
?

Larry