EfficientProcessingofDataWarehousingQueriesinaSplitExecutionEnvironmentKamilBajda-Pawlikowski1+2,DanielJ.Abadi1+2,AviSilberschatz2,ErikPaulson31HadaptInc.,2YaleUniversity,3UniversityofWisconsin-Madison{kbajda,dna}@hadapt.com;avi@cs.yale.edu;epaulson@cs.wisc.eduABSTRACT1.INTRODUCTIONHadaptisastart-upcompanycurrentlycommercializingMapReduce[19]isemergingasaleadingframeworkfortheYaleUniversityresearchprojectcalledHadoopDB.Theperformingscalableparallelanalyticsanddatamining.companyfocusesonbuildingaplatformforBigDataanalyt-SomeofthereasonsforthepopularityofMapReduceicsinthecloudbyintroducingastoragelayeroptimizedforincludetheavailabilityofafreeandopensourceimplemen-structureddataandbyprovidingaframeworkforexecutingtation(Hadoop)[2],impressiveease-of-useexperience[30],SQLqueriesefficiently.aswellasGoogle's,Yahoo!'s,andFacebook'swideusageThisworkconsidersprocessingdatawarehousingqueries[19,25]andevangelizationofthistechnology.Moreover,oververylargedatasets.Ourgoalistomaximizeperfor-MapReducehasbeenshowntodeliverstellarperformancemancewhile,atthesametime,notgivingupfaulttoleranceonextreme-scalebenchmarks[17,3].Allthesefactorshaveandscalability.WeanalyzethecomplexityofthisproblemresultedintherapidadoptionofMapReduceformanyinthesplitexecutionenvironmentofHadoopDB.Here,in-differentkindsofdataanalysisandprocessing[15,18,32,comingqueriesareexamined;partsofthequeryarepushed29,25,11].downandexecutedinsidethehigherperformingdatabaseHistorically,themainapplicationsoftheMapReducelayer;andtherestofthequeryisprocessedinamoregenericframeworkincludedWebindexing,textanalytics,andMapReduceframework.graphdatamining.Inthispaper,wediscussindetailperformance-orientedNow,however,asMapReduceissteadilydevelopingintoqueryexecutionstrategiesfordatawarehousequeriesinsplitthedefactodataanalysisstandard,itrepeatedlybecomesexecutionenvironments,withparticularfocusonjoinandemployedforqueryingstructureddata|anareatradition-aggregationoperations.Theefficiencyofourtechniquesallydominat