Skip to main content

Mathematical Statistics II · Linear models and projection geometry

All random variables below are square-integrable and centered: 𝔼[x]=𝔼[z]=𝔼[y]=0

Problem

Mathematical Statistics II practice problem

All random variables below are square-integrable and centered:
E[x]=E[z]=E[y]=0\mathbb{E}[x]=\mathbb{E}[z]=\mathbb{E}[y]=0. The structural equation isy=β1x+β2z+u,E[u∣x,z]=0.y=\beta_1 x+\beta_2 z+u, \qquad \mathbb{E}[u\mid x,z]=0.A researcher regresses yy on xx alone and forms the population (and, by the WLLN, probability-limit) slopeplim β^1=Cov(x,y)Var(x).\mathrm{plim}\,\hat\beta_1 =\frac{\mathrm{Cov}(x,y)}{\mathrm{Var}(x)}.1. Prove the omitted-variable identityplim β^1=β1+β2Cov(x,z)Var(x).\mathrm{plim}\,\hat\beta_1 =\beta_1+\beta_2\frac{\mathrm{Cov}(x,z)}{\mathrm{Var}(x)}.2. Specialize to β1=2\beta_1=2, β2=3\beta_2=3, Var(x)=4\mathrm{Var}(x)=4,
and Cov(x,z)=2\mathrm{Cov}(x,z)=2. Proveplim β^1=72.\mathrm{plim}\,\hat\beta_1 =\frac72.3. The same covariance is realized by the linear projection
z=12x+vz=\tfrac12 x+v with Cov(x,v)=0\mathrm{Cov}(x,v)=0. Prove that this representation reproduces Cov(x,z)=2\mathrm{Cov}(x,z)=2 and the same probability limit 7/27/2.

4. Prove that the bias term vanishes if β2=0\beta_2=0 or
Cov(x,z)=0\mathrm{Cov}(x,z)=0, and that the sign of the bias equals the sign of the product β2 Cov(x,z)\beta_2\,\mathrm{Cov}(x,z) (for
Var(x)>0\mathrm{Var}(x)>0).

5. Audit both sentences: (i) “the probability limit is
β1+β2Var(x)/Cov(x,z)\beta_1+\beta_2\mathrm{Var}(x)/\mathrm{Cov}(x,z), or β1+β2Cov(x,z)\beta_1+\beta_2\mathrm{Cov}(x,z), giving 1414 or 88”;
(ii) “bias remains when β2=0\beta_2=0, or the sign follows
β1\beta_1 alone, or z=12x+vz=\tfrac12 x+v changes the limit, or this is a WLS / ridge exercise.”

Stay inside the centered linear structural equation with E[u∣x,z]=0\mathbb{E}[u\mid x,z]=0. Do not introduce an intercept estimation problem, and do not replace the omitted-variable identity by a ridge shrinkage.

Hint

Hint

Cov(x,y)=β1Var(x)+β2Cov(x,z)\mathrm{Cov}(x,y)=\beta_1\mathrm{Var}(x)+\beta_2\mathrm{Cov}(x,z) because Cov(x,u)=0\mathrm{Cov}(x,u)=0. Divide by Var(x)\mathrm{Var}(x).

Check your work

Work the problem yourself first. Then open it in Training to check your answer and read the full worked solution.

The answer check and full solution for this problem come with ProofAnvil Practice membership ($19 USD monthly). See membership. Or start with the free Mathematical Statistics II sample problem: Try the free sample problem.

More Mathematical Statistics II practice problems