4Lecture4:ClassicalEstimationTheory4.1Cramer-RaoInequalityObtainingapointestimatoroftheparameterofinterestisusuallythefirststepinin-ference.SupposeX=(X1;X2;::;Xn)arei.i.d.fromf(x;θ);θ2RandweuseastatisticTn(X)toestimateθ.IfEθ(Tn)=θ+bn(θ)thenthequantitybn(θ)iscalledbias.Notethatitgenerallymaydependonbothθandthesamplesizealthoughthisdependencemaysometimesbesuppressedinthenotation.Wewouldhopeforazerobiasforallθandn,calledunbiasedness.Whenusedrepeatedly,anunbiasedestimator,inthelongrun,willestimatethetruevalueonaverage.Caution:Note,however,thatforsomefamiliesanunbiasedestimatorsmaynotexistor,evenwhentheyexist,maynotbeveryuseful.Forexample,inthecaseofthegeometricdistributionf(x;θ)=θ(1−θ)x−1;x=1;2;:::anunbiasedestimatorofθ,say,T(x)mustP1x−1satisfyx=1T(x)θ(1−θ)=θforallθ2[0;1].Byapolynomialexpansion,theonlyestimatorsatisfyingthisrequirementwouldbeT(1)=1;T(x)=0ifx≥2.Havinginmindtheinterpretationofθ(probabilityofsuccessinasingletry),suchanestimatorisneitherveryreliable,norveryuseful.Whenlookingforanestimatorofa\good"quality,weareinclinedtoanalysethemeansquarederror22MSEθ(Tn)=Eθ(Tn−θ)=VarθTn+(bn(θ)):Asmallmeansquarederrorasacriterionforchoosingapointestimator,isingeneralmoreimportantthanunbiasedness.Toperformoptimally,wewouldtrytofindanestimatorthatminimizestheMSE.Unfortunately,intheclassofallestimators,anestimatorthatminimizestheMSEsimultaneouslyforallθvalues,doesnotexist(theargumentforthiswillbegivenduringlectures).Wayoutofthissituationiseithertorestricttheclassofestimatorsconsidered,ortochangetheevaluationcriterion.Weshallbedealingwiththefirstwayoutrightnow(theotherwaywasdiscussedintheDecisiontheorychapter:Bayesandminimaxestimation).Wechoosetoimposethecriterionofunbiasedness.Thisgreatlysimplifiesthetaskofminimizingthemeansquarederrorbecausethen,weonlyhavetominimizethevariance.Inthe(smallersubsetofunbiasedestimators)onecanveryoftenfindanestimatorwiththesmallestMSE(=Var)forallθvalues.Itiscalledtheuniformlyminimumvarianceunbiasedestimator(UMVUE).