Salient Regions Detection for Indoor Robots using RGB-D Data(2)

visualcolor,depthvalueandspatiallayoutsigni cantlyimprovestheaccuracyofobjectdetectionalgorithms.

Thecontributionsandadvantagesoftheproposedmethodareasfollows:

Thegraph-basedsegmentationfusesRGBwithdepthdatatomeasureinconsistenciesintheimage.Thealgorithmachievessuperiordetectionoftexturedorinhomogeneouscoloredobjectscomparedtoourpre-viouswork,wherewewereassumingmostlyuniformlycoloredfruits[6]orclosestobjects[7]andwerethusabletosuccessfullyusethehomogeneityandshortestdistanceinthesegmentation.

Giventheassumptionofrarityinregardtothescenerepresentationforobjectsofinterest,weemployanadaptedandcompactestimationofsalientregionswithessentialRGB-Dcharacteristicsusingcolorluminance,depth,spatiallayoutandboundaryinformation.Fore-groundnessisestimatedbydifferenceofcolor,depthandposition.Backgroundnessisproducedbyarea,boundaryconnectivityandtherelativedistancefromthecenterofoneregiontotheimagecornersinsteadofde ninganarrowborderregionasanindicatorforbackground[8].

The nalsaliencymapisacquiredbycombiningfore-groundandbackgroundmeasuresbasedonaGaussian lter.

Theremainderofthepaperisstructuredasfollows:abriefsummaryofrelevantconceptsinsaliencyalgorithmsispresentedinSectionI-B.SectionIIdescribesourproposedmethod,includingthesegmentationapproachandsaliencyalgorithm.Moreover,visualsamplesofsaliencyresultsarepresentedforthecomparisonofdifferentsaliencymethods.Finally,inSectionIIIwepresenttheexperimentalresultsbasedontwodatasetsandconcludethepaperinSectionIV.B.RelatedWork

Inrecentyears,thedevelopmentofmethodsforobjectrecognitionanddetectionhasbeenrapidlyadvancing.Manyresearchershavestudiedtheeffectsofsaliencydetection[1]–[3],[8]–[10].Ingeneral,saliencydetectionalgorithmscanberoughlyclassi edintotwocategories:top-downandbottom-up.Thetop-downmethods[11]obtainasaliencymapbylearningvisualknowledge.Inotherwords,top-downsaliencymethodsrequirealargeamountofannotatedimagesfortraining.Incontrast,thebottom-upapproaches[1]–[3],[8],[9]focusonalow-levelalgorithmbydeterminingcontrastofimageregionsrelativetotheirsurrounding,intensity,colorandorientation.Theseapproachesdonotrequirepriortraining.Ittietal.[12]werethe rsttoadvocateabottom-upapproachinvisualattention.Theyutilizedlocalcontrastandvisuallow-levelfeaturestoacquiresaliency.Subsequently,Achantaetal.[13]acquiredasaliencymapbycomputingthedifferencebetweentheimageandaGaussianblurredversionoftheoriginalimage.Thoughbeingsimpleandcom-putationallyef cient,themethodfailedwhenthesaliencyregionoccupiedmorethanhalfthepixelsoftheimage,orinthepresenceofcomplexbackgrounds.Achantapresenteda

revisedapproachbasedontheideaofmaximumsymmetricsurround[14],whichisderivedfromtheassumptionofarelationbetweenscaleandpositionofthecandidateobjectintheimage.Chengetal.[2]proposedaglobalhistogram-basedcontrastforsaliencydetection.Thedissimilarityofapairofpatchesisdeterminedbycomparingtheircolorhistograms.Saliency lterswerepresentedbyPerazzietal.[3]relyingonestimatinganelementuniquenessanddistributionasafunctionofimagecontrast.Inspiredbyrecentadvancesincontrastanalysis,Zhuetal.[9]proposedasaliencyoptimizationfrombackgrounddetection.Theyutilizeameasuredescribingtheconnectivitybetweenregionandimageboundaries.

Whilethereisawealthofresearchonvisualsaliencymaps,fewattemptshavebeenmadetocombinedepthvaluestoformasaliencymap.Makietal.[15]presentedacomputationalmodelforattentionbyusingdepthcues.Inthisdepth-basedmodel,closertargetsweremappedtohigherpriorityinanattentionalscheme.Ouerhanietal.[16]ngetal.[5]collectedahumaneye xationdatabaseinboth2Dand3DscenesbytheKinectsensor.Theyderivedepthpriorsthatmaybeappliedtosaliencymapsaimingtopredictvisualattentionareasofhumans.AnothermethodforincorporatingvisualsaliencyanddepthinformationwasproposedbyCiptadietal.[4].Thismethodused3Dlayoutandshapefeaturesfromdepthmeasurementstogenerateasaliencymap.Theypresentedpromisingresultsbysaliency-basedsegmentationusingasuperpixelMarkovRandomField(MRF).Ourworkfollowstheparadigmofbottom-upapproachesincorporatingdepthcues.

II.METHOD

Inthissection,thesegmentationalgorithmandsalientmeasuresunderRGB-Ddataaredescribed.Themaintaskofthisworkistolettherobotautomaticallydetectsalientobjectsinascene.Hence,wefocusontheneedforcaptur-ingsalientobjects.Thealgorithmproposedinthissectionincludesthreestepstoaddressandoptimizethisproblem.At rst,weuseagraph-basedRGB-Dsegmentationtoclusterpixelsinanimage.Thisprocessminimizesthesearchspaceandintegratescommoncolors,texturesanddepthinaregion.Then,wepresentamethodwhichcombinessalientforegroundandbackgroundregionstomodelthecorrespondingsaliencymap.Ina nalstep,wecomputethedesiredsaliencymapbyaweightedcombinationofsaliencysub-maps.

A.Graph-BasedRGB-DSegmentation

Thegoalofthesegmentationprocessistoselectpossi-blesalientregioncandidatesfromanintricateenvironmentrepresentedbytheRGB-Ddatastream.Inthispaper,weapplythegraph-basedapproachfrom[17]tolabeldifferentelementsinanRGB-Dimage.First,wetreatanRGB-Dimageasafully-connecteddirectedgraphG=(V,E)withverticesvi∈Vandasetofedges(vi,vj)∈E.Eachedge