提出一种基于序列标注同时充分利用本体知识增强命名实体识别能力的端到端方法完成对话状态追踪。一方面通过设计命名实体识别指针,基于序列标注方法对对话历史包含的本体知识信息进行标注,有效利用槽值对本体集增强命名实体识别能力;另一方面利用指针网络,保留新槽值识别的优点。实验结果表明,本文提出的方法相比现有模型在命名实体识别的能力上提升了1.2%,并保留槽值识别可扩展的优点。
An end-to-end method based on sequence labeling while making full use of ontology knowledge to enhance the ability of named entity recognition to complete dialogue state tracking is proposed. On the one hand, this method uses a named entity recognition pointer to label the ontology knowledge information contained in the dialogue history based on the sequence labeling method and effectively uses the slot value to enhance the named entity recognition ability of the ontology set. On the other hand, the pointer network is used to retain the ability of the new slot value recognition. The experimental results show that the method proposed in this paper improves the ability of named entity recognition by 1.2% compared with the existing model, and retains the advantages of new slot recognition scalability.
[1] LI F L,QIU M H,CHEN H Q,et al.AliMe assist:an intelligent assistant for creating an innovative E-commerce experience[C]//Proceedings of the 2017 ACM on Conference on Information and Knowledge Management.New York,USA:ACM,2017.
[2] ZHOU L,GAO J F,LI D,et al.The design and implementation of XiaoIce,an empathetic social chatbot[J].Computational Linguistics,2020,46(1):53-93.
[3] WU C S,MADOTTO A,HOSSEINI-ASL E,et al.Transferable multi-domain state generator for task-oriented dialogue systems[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.Florence,Italy:Association for Computational Linguistics,2019
[4] VINYALS O, FORTUNATO M, JAIRLY N. Pointer networks[EB/OL].[2020-06-09].https://arxiv.org/abs/1506.03134.
[5] HENDERSON M,THOMSON B,WILLIAMS J D.The second dialog state tracking challenge[C]//Proceedings of the 15th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL).Baltimore,USA:Association for Computational Linguistics,2014:263-272.
[6] ERIC M,GOEL R,PAUL S,et al.MultiWOZ 2.1:multi-domain dialogue state corrections and state tracking baselines[EB/OL].[2020-07-02].https://arxiv.org/abs/1907.01669.
[7] THOMSON B,YOUNG S.Bayesian update of dialogue state:a POMDP framework for spoken dialogue systems[J].Computer Speech & Language,2010,24(4):562-588.
[8] WANG Z, LEMON O. A simple and generic belief tracking mechanism for the dialog state tracking challenge: on the believability of observed information[C]// The 14th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL).Sofia,Bulgaria:Association for Computational Linguistics, 2013.
[9] WILLIAMS J D.Web-style ranking and SLU combination for dialog state tracking[C]//Proceedings of the 15th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL).Baltimore,USA:Association for Computational Linguistics,2014.
[10] WILLIAMS J D,YOUNG S.Partially observable Markov decision processes for spoken dialog systems[J].Computer Speech & Language,2007,21(2):393-422.
[11] HENDERSON M,THOMSON B,YOUNG S.Word-based dialog state tracking with recurrent neural networks[C]//Proceedings of the 15th Annual Meeting of the Special Interest Group on Discourse and Dialogue (SIGDIAL).Baltimore,USA:Association for Computational Linguistics,2014.
[12] ZILKA L,JURCICEK F.Incremental LSTM-based dialog state tracker[C]//2015 IEEE Workshop on Automatic Speech Recognition and Understanding(ASRU).Scottsdale,Austrilia: IEEE,2015:757-762.
[13] MRKŠIĈ N,ÓSÉAGHDHA D,WEN T H,et al.Neural belief tracker:data-driven dialogue state tracking[C]//Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics.Vancouver,Canada:Association for Computational Linguistics,2017.
[14] ZHANG J G, HASHIMOTO K, WU C S, et al. Find or classify? Dual strategy for slot-value predictions on multi-domain dialog state tracking[EB/OL].[2020-10-08].https://arxiv.org/abs/1910.03544.
[15] QIU L,XIAO Y X,QU Y R,et al.Dynamically fused graph network for multi-hop reasoning[C]//Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics.Florence,Italy:Association for Computational Linguistics,2019.
[16] ZHOU L, SMALL K. Multi-domain dialogue state tracking as dynamic knowledge graph enhanced question answering[EB/OL].[2020-11-07].https://arxiv.org/abs/1911.06192.
[17] DEVLIN J, CHANG M W, LEE K, et al. BERT: pre-training of deep bidirectional transformers for language understanding[EB/OL]. [2020-10-11].https://arxiv.org/abs/1810.04805.
[18] RADFORD A, NARASIMHAN K, SALIMANS T, et al. Improving language understanding by generative pre-training[EB/OL]. [2020-11-13].https://s3-us-west-2.amazonaws.com/openai-assets/research-covers/language-unsupervised/language_understanding_paper.pdf.
[19] CHAO G L,LANE I.BERT-DST:Scalable end-to-end dialogue state tracking with bidirectional encoder representations from transformer[EB/OL].[2020-11-05].https://arxiv.org/abs/1907.03040.
[20] BUDZIANOWSKI P,WEN T H,TSENG B H,et al.MultiWOZ-a large-scale multi-domain wizard-of-oz dataset for task-oriented dialogue modelling[C]//Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing.Melbourne,Austrilia:Association for Computational Linguistics,2018.
[21] KINGMA D P, BA J. Adam: a method for stochastic optimization[J].[2020-11-22].https://arxiv.org/abs/1412.6980.
[22] SRIVASTAVA N, HINTON G , KRIZHEVSKY A, et al. Dropout: a simple way to prevent neural networks from overfitting[J]. Journal of Machine Learning Research, 2014, 15(1):1929-1958.
[23] PENNINGTON J,SOCHER R,MANNING C.Glove:global vectors for word representation[C]//Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP).Doha,Qatar:Association for Computational Linguistics,2014.
[24] CHEN X, XU L, LIU Z, et al. Joint learning of character and word embeddings[C]//Twenty-Fourth International Joint Conference on Artificial Intelligence. Palo Alto,USA: AAAI, 2015.
[25] BOWMAN S R,VILNIS L,VINYALS O,et al.Generating sentences from a continuous space[C]//Proceedings of the 20th SIGNLL Conference on Computational Natural Language Learning.Berlin,Germany:Association for Computational Linguistics,2016.
