语音识别解码

6搜索和解码黎塔[e-mail]语言声学与内容理解重点实验室中国科学院声学研究所2014-04-10提纲概述搜索空间知识源搜索空间构建搜索空间优化解码和词图生成解码算法词图基于WFST语音识别多遍融合解码系统6.1概述语音识别问题模式识别语言模型声学模型发音字典特征搜索问题（解码）给定模型找到与特征相匹配文本6.1概述解码技术快速准确6.1概述whatwasW?knowingO,priorknowledgesource:languagemodel(LM):P(W)(oftenM-gram;)dictionary:words=triphone(W=Ph)acousticmodel:p(O|PH),HMMslanguagegenerationpronunciationspeechproductionwordsequenceW=(w1...wN)sequenceoftriphonePH=(ph1...phN)soundwavestransmissionSRfront-endprocessingobservationvectorsO=(o1...oT)?LM:P(W)dictio-nary}AM:p(O|S)6.1概述贝叶斯框架及viterbi近似6.1概述提纲概述搜索空间知识源搜索空间构建搜索空间优化解码和词图生成解码算法词图基于WFST语音识别多遍融合解码系统6.2搜索空间什么是搜索空间文本搜索InformationRetrieval多媒体检索什么是语音识别的搜索空间语音识别知识源空间完备精简6.2.1知识源语言模型发音字典声学上下文声学模型特征6.2.1知识源语言模型词图/有限状态图N元统计语言模型),|(),,|(21121NNNNNN6.2.1知识源发音字典声学上下文声学模型隐含马尔科夫模型（HMM）特征MFCC,PLP,LPCC….6.2.2搜索空间构建发音字典（线性）6.2.2搜索空间构建发音字典（前缀树）6.2.2搜索空间构建15sihaetdsittsatsadrootNextword6.2.2搜索空间构建6.2.2搜索空间构建WithinWord和CrossWord6.2.2搜索空间构建孤立词识别固定识别词边界6.2.2搜索空间构建受限词图识别6.2.2搜索空间构建大词表连续语音识别（LVCSR）特点大词汇连续语音语言模型延迟难点搜索空间占用内存大搜索速度慢6.2.2搜索空间构建LVCSR搜索空间-Unigram6.2.2搜索空间构建LVCSR搜索空间-Bigram6.2.2搜索空间构建LVCSR搜索空间-backoff6.2.2搜索空间构建对句中的静音处理6.2.3搜索空间的优化优化准则不改变空间完备性减少重复路径知识源信息紧耦合目标减少搜索空间内存提高搜索速度6.2.3搜索空间的优化前向后向归并搜索算法3forwardmergebackwardmerge456σσss1278349σσs1278109σs12786.2.3搜索空间的优化-example例子：受限词图6.2.3搜索空间的优化-example原始词图6.2.3搜索空间的优化-exampleWordgraph-》phonemenetwork6.2.3搜索空间的优化-exampleTriphonenetwork搜索空间构建优化的网络6.2.3搜索空间的优化LVCSR搜索空间动态动态加载知识源解码过程需要查询知识源信息内存占用小，空间构建速度快解码速度不够快静态预先编译好知识源解码过程就是一个FST的搜索问题内存占用大，空间构建速度慢解码速度快6.2.3搜索空间的优化LVCSR搜索空间动态词典树6.2.3搜索空间的优化LVCSR搜索空间静态WFST（G·L）6.2.3搜索空间的优化北极(north-pole)b-ei3-j-i2北京（Beijing）b-ei3-j-ing1南京(Nanjing)n-an2-j-jing1!SENT_PAUSEsil!SENT_ENDsil!SEND_STARTsil三词线性词典ing1-b+ei3词头三音子声学上下文：词内三音子词尾三音子b-ei3+jj-ing1+n初始网络构造Howtoreduceredundancy?8769079422298231623252298232023367北极spwespwe60InitialMid-partFinalFIFOWIWE816ROOT10171819202122232425261662365236951647165412345687690794233北京123413141516623651369816471654272829303132324432593264324932623266spwespwe3536373839404142434451南京529543367451845464748495040104059408652535455400940594086565758593728373136983731911ing1-b+ei3b-ei3+jei3-j+i2i2-b+ei3ing1-n+an2i2-n+an2n-an2+jan2-j+ing1ing1-bj-i2+bj-i2+nj-ing1+bj-ing1+nb-ei3+ji2-bing1-ni2-nj-i2j-ing1ei3-j+ing1SPAWDW876907942S1S2S3ing1-b+ei3三音子状态ID音素决策树映射6.2.3搜索空间的优化LVCSR搜索空间前后向归并8769079422298231623252298232023367北极spwespwe60InitialMid-partFinalFIFOWIWE816ROOT10171819202122232425261662365236951647165412345687690794233北京123413141516623651369816471654272829303132324432593264324932623266spwespwe3536373839404142434451南京529543367451845464748495040104059408652535455400940594086565758593728373136983731911ing1-b+ei3b-ei3+jei3-j+i2i2-b+ei3ing1-n+an2i2-n+an2n-an2+jan2-j+ing1ing1-bj-i2+bj-i2+nj-ing1+bj-ing1+nb-ei3+ji2-bing1-ni2-nj-i2j-ing1ei3-j+ing1SPAWDW231623252298232023367北极spwespwe60InitialMid-partFinalFIFOWE816ROOT1819202164232425263652369545687690794233北京12346766651662365116471654616263306968324432593264324932623266spwespwe35363738394041424344南京5295433674518454647484010408652534009405956577170372836983731ing1-b+ei3b-ei3+jei3-j+i2i2-b+ei3ing1-n+an2i2-n+an2n-an2+jan2-j+ing1ing1-bj-i2+bj-i2+nj-ing1+bj-ing1+nei3-j+ing1i2-bing1-ni2-nj-i2j-ing151WIAWSPDW6.2.3搜索空间的优化LVCSR搜索空间37WI节点提前更有效进行前后向归并算法解码尽快加入语言模型信息8769079422298231623252298232023367北极spwespwe60InitialMid-partFinalFIFOWIWE816ROOT10171819202122232425261662365236951647165412345687690794233北京123413141516623651369816471654272829303132324432593264324932623266spwespwe3536373839404142434451南京529543367451845464748495040104059408652535455400940594086565758593728373136983731911ing1-b+ei3b-ei3+jei3-j+i2i2-b+ei3ing1-n+an2i2-n+an2n-an2+jan2-j+ing1ing1-bj-i2+bj-i2+nj-ing1+bj-ing1+nb-ei3+ji2-bing1-ni2-nj-i2j-ing1ei3-j+ing1SPAWDW231623252298232023367北极spwespwe60InitialMid-partFinalFIFOWE816ROOT1819202164232425263652369545687690794233北京12346766651662365116471654616263306968324432593264324932623266spwespwe35363738394041424344南京5295433674518454647484010408652534009405956577170372836983731ing1-b+ei3b-ei3+jei3-j+i2i2-b+ei3ing1-n+an2i2-n+an2n-an2+jan2-j+ing1ing1-bj-i2+bj-i2+nj-ing1+bj-ing1+nei3-j+ing1i2-bing1-ni2-nj-i2j-ing151WIAWSPDW6.2.3搜索空间的优化231623252298232023367北极spwespwe60InitialMid-partFinalFIFOWE816ROOT1819202164232425263652369545687690794233北京12346766651662365116471654616263306968324432593264324932623266spwespwe35363738394041424344南京5295433674518454647484010408652534009405956577170372836983731ing1-b+ei3b-ei3+jei3-j+i2i2-b+ei3ing1-n+an2i2-n+an2n-an2+jan2-j+ing1ing1-bj-i2+bj-i2+nj-ing1+bj-ing1+nei3-j+ing1i2-bing1-ni2-nj-i2j-ing151WIAWSPDWFI(fan-in)WE(wordend)WI(wordid)AW(aboveWI)DW(downWI)解码空间分层路径管理结构清晰独立剪枝集成语言模型提纲概述搜索空间知识源搜索空间构建搜索空间优化解码和词图生成解码算法词图基于WFST语音识别多遍融合解码系统6.3解码和词图生成解码算法快速搜索空间中找到最可能路径词图识别中间结果信息丰富6.3.1解码算法动态规划（DynamicProgramming）6.3.1解码算法动态时间规整（DTW）6.3.1解码算法基本图搜索算法6.3.1解码算法树搜索空间6.3.1解码算法深度优先搜索（Depthfirst）6.3.1解码算法宽度优先搜索（Breadthfirst）6.3.1解码算法时间