5Lecture5:LikelihoodInference.Firstorderasymp-totics5.1WhyasymptoticsWerealizedthatfindingtheUMVUEforafixedsamplesizencouldbedifficultinsomecasesespeciallywhentheCRboundisnotattainable.Findingthemrequiressomeart,andthereisnoeasytofollowconstructivealgorithmfortheirdetermination.WhileweknowthatsometimetheMLEcouldbebiasedor,evenifnotunbiased,couldnotattaintheCRboundwhenoutsidetheexponentialfamilysetting,stilltheyaretypicallyeasy-to-constructand,asseenonmanyexamples,usuallytheUMVUEarejustabias-correctedMLE.Indeed,theUMVUEfortheprobabilityofsuccessinnindependentBernoulli¯¯n¯¯trialswasX(1−X)n−1whereastheMLEisX(1−X);theUMVUEfortheendpointn+1θofuniform(0;θ)distributionwasnX(n)whereastheMLEisX(n);theUMVUEfortheprobabilityofnooccurrencebasedonnindependentPoissonrandomvariableswas1nX¯¯(1−n)whereastheMLEisexp(−X):Thebias-correctionitselftendstobenegligibleasthesamplesizeincreases.ThereforetheUMVUE0sareeitherMLE0sor"almost"MLE0s.Hence,itisjustifiedtolookforastrongbackingofthepropertiesofMLE0sinageneralsetting.Thiscanbedoneusingasymptoticarguments,i.e.bylookingattheperformanceofMLE0swhenn−!1,i.e.bylettingtheamountofinformationbecomearbitrarilylarge.Statisticalfolkloresaysthenthat\nothingcanbeattheMLEasymptotically".5.2ConvergenceconceptsinasymptoticsWeremindsomestochasticconvergenceconceptsfirst.AnestimatorTnoftheparameterθissaidtobe:i)consistent(orweaklyconsistent)iflimPθ(jTn−θj>)=0n!1Pforallθ2Θandforeveryfixed>0.WedenotethisbyTn!θ.ii)stronglyconsistentifPθflimn!1Tn=θg=1forallθ2Θ.iii)mean-squareconsistentifMSEθ(Tn)−!n!10forallθ2Θ.Itisimportanttonotethatiftheestimatorismean-squareconsistentthenitisalsoconsistent.Thisrelationhasprobablythemostimportantpracticalconsequence.Thereasonisthatmostoftenweareinterestedinweakconsistencyandacommonmethodthatoftenworksinprovingit,isbyshowingmean-squareconsistencyfirst.Tojustifytherelationbetweenmean-squareconsistencyandconsistencywecanusetheChebyshevInequality.ItstatesthatforanyrandomvariableXandany>0