PCA-sift-A-more-distinctive-representation-for-loc

PCA-SIFT:AMoreDistinctiveRepresentationforLocalImageDescriptorsYanKe1,RahulSukthankar1;2{yke,rahuls}@cs.cmu.edu1SchoolofComputerScience,CarnegieMellonUniversity;2IntelResearchPittsburgh˜yke/pcasift/AbstractStablelocalfeaturedetectionandrepresentationisafun-damentalcomponentofmanyimageregistrationandobjectrecognitionalgorithms.MikolajczykandSchmid[14]re-centlyevaluatedavarietyofapproachesandidentifiedtheSIFT[11]algorithmasbeingthemostresistanttocommonimagedeformations.Thispaperexamines(andimprovesupon)thelocalimagedescriptorusedbySIFT.LikeSIFT,ourdescriptorsencodethesalientaspectsoftheimagegra-dientinthefeaturepoint’sneighborhood;however,insteadofusingSIFT’ssmoothedweightedhistograms,weapplyPrincipalComponentsAnalysis(PCA)tothenormalizedgradientpatch.OurexperimentsdemonstratethatthePCA-basedlocaldescriptorsaremoredistinctive,morerobusttoimagedeformations,andmorecompactthanthestandardSIFTrepresentation.Wealsopresentresultsshowingthatusingthesedescriptorsinanimageretrievalapplicationre-sultsinincreasedaccuracyandfastermatching.1.IntroductionLocaldescriptors[6,12,18]arecommonlyemployedinanumberofreal-worldapplicationssuchasobjectrecogni-tion[3,11]andimageretrieval[13]becausetheycanbecomputedefficiently,areresistanttopartialocclusion,andarerelativelyinsensitivetochangesinviewpoint.Therearetwoconsiderationstousinglocaldescriptorsintheseappli-cations.First,wemustlocalizetheinterestpointinpositionandscale.Typically,interestpointsareplacedatlocalpeaksinascale-spacesearch,andfilteredtopreserveonlythosethatarelikelytoremainstableovertransformations.Sec-ond,wemustbuildadescriptionoftheinterestpoint;ide-ally,thisdescriptionshouldbedistinctive(reliablydifferen-tiatingoneinterestpointfromothers),concise,andinvari-antovertransformationscausedbychangesincameraposeandlighting.Whilethelocalizationanddescriptionaspectsofinterestpointalgorithmsareoftendesignedtogether,thesolutionstothesetwoproblemsareindependent[14].Thispaperfocusesonapproachestothesecondaspect–thecon-structionandevaluationoflocaldescriptorrepresentations.MikolajczykandSchmid[14]presentedacomparativestudyofseverallocaldescriptorsincludingsteerablefil-ters[4],differentialinvariants[9],momentinvariants[18],complexfilters[16],SIFT[11],andcross-correlationofdif-ferenttypesofinterestpoints[6,13].Theirexperimentsshowedthattherankingofaccuracyforthedifferentalgo-rithmswasrelativelyinsensitivetothemethodemployedtofindinterestpointsintheimagebutwasdependentontherepresentationusedtomodeltheimagepatcharoundtheinterestpoint.Sincetheirbestmatchingresultswereob-tainedusingtheSIFTdescriptor,thispaperfocusesonthatalgorithmandexploresalternativestoitslocaldescriptorrepresentation.Theremainderofthispaperisorganizedasfollows.Sec-tion2reviewstherelevantaspectsoftheSIFTalgorithm.Section3detailsourPCA-basedrepresentationforlocalfeatures(PCA-SIFT).Section4presentsourevaluationmethodologyandperformancemetrics.Section5providesdetailedexperimentalresultscomparingPCA-SIFTtostan-dardSIFTonfeature-matchingexperimentsandalsointhecontextofanimageretrievalapplication.Section6exam-inesthereasonsbehindPCA-SIFT’saccuracybyexploringtheroleofdifferentcomponentsintherepresentation.Fi-nally,Section7summarizesthecontributionsofthispaperandconcludeswithsomeideasforfutureresearchinthisarea.2.ReviewoftheSIFTAlgorithmSIFT,asdescribedin[12],consistsoffourmajorstages:(1)scale-spacepeakselection;(2)keypointlocalization;(3)orientationassignment;(4)keypointdescriptor.Inthefirststage,potentialinterestpointsareidentifiedbyscan-ningtheimageoverlocationandscale.Thisisimple-mentedefficientlybyconstructingaGaussianpyramidandsearchingforlocalpeaks(termedkeypoints)inaseriesofdifference-of-Gaussian(DoG)images.Inthesecondstage,candidatekeypointsarelocalizedtosub-pixelaccuracyandeliminatediffoundtobeunstable.Thethirdidentifiesthedominantorientationsforeachkeypointbasedonitslocalimagepatch.Theassignedorientation(s),scaleandloca-1tionforeachkeypointenablesSIFTtoconstructacanoni-calviewforthekeypointthatisinvarianttosimilaritytrans-forms.Thefinalstagebuildsalocalimagedescriptorforeachkeypoint,basedupontheimagegradientsinitslocalneighborhood(discussedbelowingreaterdetail).Thefirstthreestageswillnotbediscussedfurtherinthispapersinceourworkmakesnocontributionstothoseareas.Thefinal(keypointdescriptor)stageoftheSIFTalgo-rithmbuildsarepresentationforeachkeypointbasedonapatchofpixelsinitslocalneighborhood.Notethatthepatchhasbeenpreviouslycenteredaboutthekeypoint’slo-cation,rotatedonthebasisofitsdominantorientationandscaledtotheappropriatesize.Thegoalistocreateade-scriptorforthepatchthatiscompact,highlydistinctive(i.e.,patchesfromdifferentkeypointsmaptodifferentrepresen-tations)andyetrobusttochangesinilluminationandcam-eraviewpoint(i.e.,thesamekeypointindifferentimagesmapstosimilarrepresentations).Asdiscussedin[12],ob-viousapproachessuchasnormalizedcorrelationbetweenimagepatchesdonotworksincetheyareoverlysensitivetoregistrationerrorsandnon-rigiddeformations.Thestan-dardkeypointdescriptorusedbySIFTiscreatedbysam-plingthema