13-08-2012, 02:22 PM
Hidden Markov Models
[attachment=31280]
Hidden Markov Model (HMM)
HMMs allow you to estimate probabilities of unobserved events
Given plain text, which underlying parameters generated the surface
E.g., in speech recognition, the observed data is the acoustic signal and the words are the hidden parameters
HMMs and their Usage
HMMs are very common in Computational Linguistics:
Speech recognition (observed: acoustic signal, hidden: words)
Handwriting recognition (observed: image, hidden: words)
Part-of-speech tagging (observed: words, hidden: part-of-speech tags)
Machine translation (observed: foreign words, hidden: words in target language)
Noisy Channel Model
In speech recognition you observe an acoustic signal (A=a1,…,an) and you want to determine the most likely sequence of words (W=w1,…,wn): P(W | A)
Problem: A and W are too specific for reliable counts on observed data, and are very unlikely to occur in unseen data
Parameters of an HMM
States: A set of states S=s1,…,sn
Transition probabilities: A= a1,1,a1,2,…,an,n Each ai,j represents the probability of transitioning from state si to sj.
Emission probabilities: A set B of functions of the form bi(ot) which is the probability of observation ot being emitted by si
Initial state distribution: is the probability that si is a start state
[attachment=31280]
Hidden Markov Model (HMM)
HMMs allow you to estimate probabilities of unobserved events
Given plain text, which underlying parameters generated the surface
E.g., in speech recognition, the observed data is the acoustic signal and the words are the hidden parameters
HMMs and their Usage
HMMs are very common in Computational Linguistics:
Speech recognition (observed: acoustic signal, hidden: words)
Handwriting recognition (observed: image, hidden: words)
Part-of-speech tagging (observed: words, hidden: part-of-speech tags)
Machine translation (observed: foreign words, hidden: words in target language)
Noisy Channel Model
In speech recognition you observe an acoustic signal (A=a1,…,an) and you want to determine the most likely sequence of words (W=w1,…,wn): P(W | A)
Problem: A and W are too specific for reliable counts on observed data, and are very unlikely to occur in unseen data
Parameters of an HMM
States: A set of states S=s1,…,sn
Transition probabilities: A= a1,1,a1,2,…,an,n Each ai,j represents the probability of transitioning from state si to sj.
Emission probabilities: A set B of functions of the form bi(ot) which is the probability of observation ot being emitted by si
Initial state distribution: is the probability that si is a start state