Robust wide baseline stereo from maximally stable extremal r(6)

The wide baseline stereo problem ,i.e.the problem of establishing correspondences between apair of images taken from different view points is studied.A new set of image elements that are put into correspondence,the so called extremal regions,is introduced.Extremal regions possess highly desirable properties

766

J.Matasetal./ImageandVisionComputing22(2004)761–767

Table3

Experimentalresults

TC

RoughRoughEGþcorrFineEGFined’MissEGd’

Bookshelf85250.48

151630.091Valbonne49270.17180820.080Wash171420.34220860.082Kampa303780.344221850.082Cyl.box63230.15102670.093Shout15144

0.43

220

86

0.08

1

Fordetails,seethetextatthebeginningofSection4.

Afterestablishingthe‘roughEG’theso-called‘guidedmatching’stepisapplied[2,5].Intheprocessof ndingtentativecorrespondences,atmostasinglecorrespondingregionisassociatedwithoneDR.Oftenthisassociationiserroneous,forinstanceifthereisarepetitivepatterninthescene.MoreoversomeDRsarenotmatchedatallsincetheyfailthe‘mutuallynearest’requirement.Giventhe‘roughEG’,evenratherimprecise,theprocessof ndingtentativematchescanberevisited.Theoriginaltentativecorrespon-dencesarediscardedandallpotentialmatchesconsistentwiththe‘roughEG’areselected.Thematchingnowneednotrelyonrotationalinvariants,sinceepipolarlinespassingthroughapairofmatchingregionsde netheirrelativeorientation[2].The‘guided’tentativecorrespondencesarethereforeselectedusingcorrelation.

Thecolumnheaded‘EGþcorr’givesthenumberofcorrespondencesconsistentwithroughEGthatpassedthecorrelationtest.Noticethatthenumbersaremuchhigherthanthoseinthe‘roughEG’column.The nalnumberofcorrespondencesisgiveninthepenultimatecolumn‘ neEG’.Averagedistancesfromepipolarlinesarepresentedincolumns‘roughd’’and‘ ned’’.Wecansee,thattheprecisionoftheestimatedEGisveryhigh,muchhigherthantheprecisionoftheroughEG.Thelastcolumnshowsthenumberofmismatches(foundmanually).

5.Conclusions

Anewmethodforwide-baselinematchingwasproposed.Thethreemainnoveltiesare:theintroductionofMSERs,robustmatchingoflocalfeaturesandtheuseofmultiplescaledmeasurementregions.

TheMSERsaresetsofimageelements,closedundertheaf netransformationofimagecoordinatesandinvarianttoaf netransformationofintensity.Anef cient(nearlinearcomplexity)andpracticallyfastdetectionalgorithmwaspresented.ThestabilityandhighutilityofMSERswasdemonstratedexperimentally.Anothernoveltyoftheapproachistheuseofarobustsimilaritymeasureforestablishingtentativecorrespondences.Duetotherobust-ness,wewereabletoconsiderinvariantsfrommultiple

measurementregions,evensomethatweresigni cantlylarger(andhenceprobablydiscriminative)thantheassociatedMSER.

GoodestimatesofEGwereobtainedonchallengingwide-baselineproblemswiththerobusti edmatchingalgorithmoperatingontheoutputproducedbytheMSERdetector.Theaveragedistancefromcorrespondingpointstotheepipolarlinewasbelow0.09oftheinter-pixeldistance.Signi cantchangeofscale(3.5£),illuminationconditions,out-of-planerotation,occlusion,locallyaniso-tropicscalechangeand3Dtranslationoftheviewpointareallpresentinthetestproblems.Testimagesincludedbothoutdoorandindoorscenes,somealreadyusedinpublishedwork.

Acknowledgements

TheauthorsweresupportedbytheEuropeanUnionprojectIST-2001-32184,bytheGrantAgencyoftheCzechRepublicprojectGACR102/02/1539andbytheAustrianMinistryofEducationprojectCONEXGZ45.535.TheSHOUTandWASHimageswerekindlymadeavailablebyTinneTuytelaars.

References

[1]A.Baumberg,Reliablefeaturematchingacrosswidelyseparated

views,in:CVPR’00,2000,pp.I:774–781.

[2]O.Chum,T.Werner,T.Pajdla,Jointorientationofepipoles,in:

ProceedingsofBMVC’03,vol.1,BMVA,London,UK,September2003,pp.73–82.

[3]Y.Dufournaud,C.Schmid,R.Horaud,Matchingimageswith

differentresolutions,in:CVPR’00,2000,pp.I:612–618.

[4]W.E.L.Grimson,ObjectRecognition,MITPress,Cambridge,MA,

1990.

[5]R.Hartley,A.Zisserman,MultipleViewGeometryinComputer

Vision,CambridgeUniversityPress,Cambridge,UK,2000.

[6]T.LindebergandJ.Ga

rding,“Shape-adaptedsmoothinginestimationof3-Ddepthcuesfromaf nedistortionsoflocal2-Dstructure”,inProc.3rdEuropeanConferenceonComputerVision,vol.800ofLectureNotesinComputerScience,(Stockholm,Sweden),pp.389–400,SpringerVerlag,1994.

[7]D.Lowe,Objectrecognitionfromlocalscale-invariantfeatures,in:

ICCV’99,1999,pp.1150–1157.

[8]J.Matas,P.B

´lek,O.Chum,Rotationalinvariantsforwide-baselinestereo,in:ProceedingsofCVWW’02,February2002,pp.296–305.

[9]J.Matas,S

ˇ.Obdrzˇ´alek,O.Chum,Localaf neframesforwide-baselinestereo,in:ProceedingsofICPR,vol.4,IEEECS,August2002,pp.363–366.

[10]K.Mikolajczyk,C.Schmid,Indexingbasedonscaleinvariantinterest

points,in:EighthInternationalConferenceonComputerVision,Vancouver,Canada,2001.

[11]K.Mikolajczyk,C.Schmid,Anaf neinvariantinterestpointdetector,

in:ProceedingsofECCV,vol.1,2002,pp.128–142.

[12]F.Mindru,T.Moons,L.vanGool,Recognizingcolorpatterns

irrespectiveofviewpointandillumination,in:CVPR’99,1999,pp.I:368–373.