如果您无法下载资料，请参考说明：
1、部分资料下载需要金币，请确保您的账户上有足够的金币
2、已购买过的文档，再次下载不重复扣费
3、资料包下载后请先用软件解压，在使用对应软件打开
18-660:NumericalMethodsforEngineeringDesignandOptimizationXinLiDepartmentofECECarnegieMellonUniversityPittsburgh,PA15213Slide1OverviewLecture7:LinearRegression^Ordinaryleast-squaresregression^Minimaxoptimization^DesignofexperimentsLecture8:ConvexAnalysis^Convexfunction^Convexset^ConvexoptimizationSlide2OrdinaryLeast-SquaresRegressionSolveover-determinedlinearequationbyoptimization⎡⎤⎡⎤⎢⎥⎢⎥⎢⎥⎢⎥Msamples⎢A⎥⋅X=⎢B⎥(M>N)⎢⎥⎢⎥⎢⎥⎢⎥⎣⎢⎦⎥⎣⎢⎦⎥NcoefficientsminA⋅X−B2X2Slide3UnconstrainedNonlinearProgrammingNonlinearcostfunctionwithoutconstraintsminA⋅X−B2X2GeneralnonlinearoptimizationisdifficulttosolveLocaloptimumGlobaloptimumSlide4UnconstrainedQuadraticProgrammingminA⋅X−B2X2However,ordinaryltleast-squaresregressionisdifftdifferentfromgeneralnonlinearprogrammingOptimizationcostisaquadraticfunctionofXanditisalwaysnon-negativeforanygivenX^Thisisauniquepropertythatenablesustosolveleast-squaresregressionefficientlySlide5PositiveSemi-DefiniteTIfaquadraticfunctionXAQXisalwaysnon-negative,thequadraticcoefficientmatrixAQispositivesemi-definite^AssumethatAQissymmetricsothatitseigenvaluesarereal^AnyasymmetricAQcanbeconvertedtoasymmetriconeSlide6PositiveSemi-DefiniteSimpleexample:⎡01⎤AQ=⎢⎥⎣00⎦Slide7PositiveSemi-DefiniteAQispositivesemi-definiteifandonlyifalleigenvaluesofAQarenon-negative(necessaryandsufficientcondition)EigenvaluedecompositionSlide8PositiveSemi-DefiniteEigenvaluedecomposition⎡λ1⎤⎢⎥A⋅V=V⋅λV=VVΣ=λQiii[12L]⎢2⎥⎣⎢O⎦⎥Slide9PositiveSemi-DefiniteIfoneoftheeigenvaluesofAQisnegativeTTAQ=V⋅Σ⋅VwhereVV=ISlide10PositiveSemi-DefiniteIfoneoftheeigenvaluesofAQisnegative⎡×⎤⎢⎥T×(VTX)()⋅⎢⎥⋅VTX⎢O⎥⎢⎥⎣−ε⎦Slide11PositiveSemi-DefiniteTTIfaquadraticfunctionXAQX+BQX+CQisalwaysnon-negative(foranyX),alleigenvaluesofAQarenon-negative^IeI.e.,AQispositivesemi-definite^Why?(Youcanprovethisconclusionbyfollowingthestepsofeigenvaluedecompos
