Skip to main content

Mathematical Statistics I · Likelihood, score, and Fisher information

Bernoulli information identities hold, then the bound can vanish

Problem

Mathematical Statistics I practice problem

Let X1,…,XnX_1,\ldots,X_n be independent Bernoulli⁡(p)\operatorname{Bernoulli}(p) observations with 0<p<10<p<1 unknown, and write S=∑i=1nXiS=\sum_{i=1}^n X_i. Starting from the joint log-likelihood, derive the score Un(p)U_n(p). Prove the three regular identitiesEp[Un(p)]=0,In(p)=Var⁡p(Un(p))=−Ep[∂pUn(p)],\mathbb E_p[U_n(p)]=0, \qquad I_n(p)=\operatorname{Var}_p(U_n(p)) =-\mathbb E_p[\partial_p U_n(p)],and compute In(p)I_n(p) explicitly. Deduce the Cramér–Rao lower bound for every unbiased estimator of pp, and show that Xˉ\bar X attains it.

Now let τ(p)=p(1−p)\tau(p)=p(1-p). Compute τ′(p)\tau'(p) and the Cramér–Rao expression for unbiased estimators of τ(p)\tau(p). Evaluate that expression at p=1/2p=1/2 and explain why a vanishing bound does not produce a zero-variance estimator of 1/41/4. Contrast this regular, fixed-support calculation with a model in which the support moves with the parameter: the identities above need not hold there, but do not re-derive a shifted-exponential example.

Do not construct a most-powerful test of H0:p=p0H_0:p=p_0.

Hint

Hint

The log-likelihood is Slog⁡p+(n−S)log⁡(1−p)S\log p+(n-S)\log(1-p). Differentiate once and twice.

Check your work

Work the problem yourself first. Then open it in Training to check your answer and read the full worked solution.

The answer check and full solution for this problem come with ProofAnvil Practice membership ($19 USD monthly). See membership. Or start with the free Mathematical Statistics I sample problem: Try the free sample problem.

More Mathematical Statistics I practice problems