Topic segmentation with an aspect hidden Markov model(6)

We present a novel probabilistic method for partially unsupervised topic segmentation on unstructured text. Previous approaches to this problem utilize the hidden Markov model framework (HMM). The HMM treats a document as mutually independent sets of words

3

ThismodelisaneffectivesegmentationframeworkonbothcleanandASRtext.However,itsuffersfromthenaiveBayesassumptionthatthewordswithineachobser-vationaremutuallyindependentgivenatopic.

Asgetslarge,thisassumptionworkswellforcomputing.However,thelargerbecomes,thelessprecisetheresultingsegmentationwillbesincethemodelcanonlyhypothesizetopicbreaksbetweensetsofwords.Thewindow(i.e.)mustbelargeenoughtogiveanaccurateestimateofwhilesmallenoughtodetectasegmentationpointwithgoodgranularity.

4AspectHMMSegmentation

AsegmentingaspectHMM(AHMM)isahiddenMarkovmodelinwhicheachhiddenstateisaninstanceofthelatentvariableinanembeddedaspectmodel.Thisaspectmodeldeterminesboththeobservationemissionprobabilitiesandtrainingsegmentclustersto ndthetransitionprobabilities.AsinthesegmentingHMM,eachobserva-tionisasetofwordsandweusetheViterbialgorithmto ndtopicbreaks.

4.1Theaspectmodelfordocumentsandwords

Inthissectionwesummarizetheaspectmodelasitappliestotext.Foradetaileddiscussion,see[5].

Theaspectmodelisafamilyofprobabilitydistributionsoverapairofdiscreterandomvariables.Intextdata,thispairconsistsofadocumentlabelandaword.Itisimportanttounderstandthatintheaspectmodel,adocumentisnotrepresentedasthesetofitswordsbutsimplyalabelwhichidenti esit.Itisassociatedwithitscorrespondingsetofwordsthrougheachdocument-wordpair.

Thismodelpositsthattheoccurrenceofadocumentandawordareindependentofeachothergivenatopicorfactor.Letdenoteasegmentfromapresegmentedcorpus,denoteaword,anddenoteatopic.Underthisindependenceassumption,thejointprobabilityofgeneratingaparticulartopic,word,andsegmentlabelis

Theparameterisalanguagemodelconditionedonthehiddenfactor.Theparameterisaprobabilitydistributionoverthetrainingsegmentlabels.Thedistributionisathepriordistributiononthehiddenfactor.

Givenacorpusofsegmentsandthewordswithinthosesegments,thetrainingdataforanaspectmodelisthesetofpairsforeachsegmentlabelandeachwordinthosesegments.WecanusetheExpectationMaximization(EM)algorithm[2]tolearnsuchamodelfromanuncategorizedcorpus.IntheE-step,wecomputetheposteriorprobabilityofthehiddenvariablegivenourcurrentmodel.IntheM-step,we