Topic segmentation with an aspect hidden Markov model(7)

We present a novel probabilistic method for partially unsupervised topic segmentation on unstructured text. Previous approaches to this problem utilize the hidden Markov model framework (HMM). The HMM treats a document as mutually independent sets of words

44ASPECTHMMSEGMENTATION

,maximizetheloglikelihoodofthetrainingdatawithrespecttotheparameters

,and.TheE-stepis

whereisthenumberoftimeswordappearsindocument.

Toavoidover ttingthetrainingdata,weusetemperedEMasdescribedin[5].Essentially,weholdoutaportionofourtrainingdataforcrossvalidationpurposesaftertheE-step.Whentheperformancedecreasesonthehold-outdata,wereduceaparameterwhichtemperstheeffectofthenextM-stepontheparametersofthemodel.InthecaseofasegmentingAHMM,wecrossvalidatebycheckingthesegmentationaccuracyonaheldoutsetoftranscriptsasmeasuredbytheCoAP(seesection5.3).Westoptrainingwhenreducingnolongerimprovesperformanceonthesegmentationofthehold-outtrainingdata.

4.2TheaspectHMM

ThesegmentingAHMMisanHMMforwhichthehiddentopicstateistherandomvariableinatrainedaspectmodel.Thisisdepictedin gure2.Generatively,theAHMMworksinexactlythesamewayastheHMMexceptthewordsfromtheselectedhiddenfactoraregeneratedviatheaspectmodelratherthanindependentlygenerated.TotrainanAHMM,wetrainanaspectmodelonasetoftrainingsegmentsasdescribedinsection4.1.Weclusterthetrainingsegmentsbytheparameter.

cluster

Finally,wecomputetransitionprobabilitiesbetweenclustersandinitialprobabilitiesofeachcluster.

Notethattheaspectmodeldoesnotrepresentclustersinthewaythatwecomputethem.Eachisrepresentedby,aprobabilityforeachlatentfactor.Thereisnotheoreticalreasonthatthefactorwithmaximumprobabilityshouldindicateacluster