CASIA OpenIR  > 毕业生  > 硕士学位论文
在线聊天系统中的跨媒体语义关联
Alternative TitleCross Media Association in Chatting System
张歆明
Subtype工学硕士
Thesis Advisor徐常胜
2011-05-21
Degree Grantor中国科学院研究生院
Place of Conferral中国科学院自动化研究所
Degree Discipline模式识别与智能系统
Keyword多媒体 多媒体聊天 支持向量机 语义概念检测 跨媒体分析 Multimedia Multimedia Chatting Svm Semantic Concept Detection Cross Media Analysis
Abstract互联网技术的飞速发展极大地方便了人们之间的沟通与交流,这其中,便捷易用的通讯工具(例如QQ, MSN, ICQ等)扮演了非常重要的角色。除了基本的文本编辑与显示,目前的绝大多数及时通讯软件还可以通过传输卡通表情以及视频图像等多媒体手段来丰富用户的使用。 但是很少有软件能够将多模态的信息进行有效的关联与同步。我们认为这一功能对于改善不同语言用户之间的交流有特别积极的意义。试想,当一位中国游客询问“去往鱼尾狮雕像的乘车路线”时,通讯系统能够聪明地配以“鱼尾狮”,“公交车”和“地图”等图片信息,那么此时即使是一位不懂中文的交通服务人员也能够很容易地理解对方的问题,从而给出准确的解答与指导。 要实现上述的目标,跨媒体的语义关联,尤其是文本与图片之间的对应,是本文研究的重点。为此我们拟从两个方面对上述问题进行讨论。其一:如何将文本关键字与图片进行合理的关联从而准确地反应用户的对话内容。这方面我们主要是研究了“动词+名词”短语的图片关联。其二:如何组织和展示用户对话的文本图片乃至情感信息从而帮助语言交流存在障碍的用户进行顺畅的沟通。 本文针对多媒体即时通讯平台中的跨媒体关联问题进行了研究,主要的成果和创新之处体现在以下几个方面: 1 针对传统的名词概念检测器不能准确反应口语中所常见的动宾结构内容的问题,我们提出了基于“动词+名词”的动宾短语的语义概念检测方法。其基本思想是在给定一组含有相同实体名词的动宾短语的情况下,把基于支持向量机的分类平面表达式的斜率看成两个部分,即一部分表示这组动宾短语概念含有的共同的名词,另一部分表示相对应的动作,对这一组动宾短语的分类器进行共同地训练从而得到相应的概念检测器。 2 为了验证我们所提出方法的正确性,我们搭建了一个基于跨媒体关联的可视化用户聊天平台。当用输入文本信息时,我们的系统首先利用自然语言处理的工具对用户的谈话内容进行分析,得出谈话的主题元素。然后根据这些文本内容和数据库中的图片进行跨媒体的语义关联,进而为用户呈现相应的多媒体信息。与此同时,系统还会对用户谈话中出现的情感词语进行检测,根据检测结果进一步关联相应的情感图片。最后,系统将自动地组织这些多模态的信息,利用它们来共同地辅助语言交流有障碍的用户进行沟通。
Other AbstractNow the rapid growth of the Internet techniques has made the communication between users convenient. The expedient chatting tools (such as QQ, MSN and ICQ) play a very important role in this aspect. Currently most of the instant messengers can transmit cartoon expression and multimedia items such as videos and images to enrich the users to use the tools except editing and displaying the chat texts. However, there rarely exist softwares that can associate the multi-modal elements with each other and make them synchronous effectively. We believe that this function can make great sense in improving the communication between the users speaking different languages. Imagine this when a Chinese tourist wants to know the route to Merlion statue by bus, if the system can return some information in the form of the images such as “Merlion”, “Bus” and “Map” intelligently, the service providers who cannot speak Chinese can understand the tourist question and then give him some accurate answers and guides. In order to achieve this goal, cross media association, especially the association between texts and images, is the key in our study. We attempt to discuss this problem based on the following two aspects. First, how to associate the keywords with the images properly can reflect the content of the users’ talk accurately. We aim to investigate “verb + noun” phrases in this facet. Second, how to organize and display the images that are related to the keywords and sentimental information aims to help the users who have language obstacle have a fluent communication. This thesis investigates the semantic cross media association in multimedia chatting. The main contributions of this dissertation are summarized as follows: 1. We propose the concept detection based on “Verb + Noun” phrases on the problem that the traditional concept detection based on noun cannot reflect the content of the verb-object phrases in oral speaking. The basic principle of our method is to treat the traditional slope in the expression of the classification plane in Support Vector Machine (SVM) as two parts, i.e. one part representing the individual action and the other part standing for the same noun object in this concept group. We can train these classifiers together in this group to obtain the detectors. 2. In order to verify the validity of our method, we implement a visualized user chatting platform. When the users input the texts, the system will utilize the Natural Language Pro...
shelfnumXWLW1645
Other Identifier200828014628063
Language中文
Document Type学位论文
Identifierhttp://ir.ia.ac.cn/handle/173211/7557
Collection毕业生_硕士学位论文
Recommended Citation
GB/T 7714
张歆明. 在线聊天系统中的跨媒体语义关联[D]. 中国科学院自动化研究所. 中国科学院研究生院,2011.
Files in This Item:
File Name/Size DocType Version Access License
CASIA_20082801462806(2643KB) 暂不开放CC BY-NC-SAApplication Full Text
Related Services
Recommend this item
Bookmark
Usage statistics
Export to Endnote
Google Scholar
Similar articles in Google Scholar
[张歆明]'s Articles
Baidu academic
Similar articles in Baidu academic
[张歆明]'s Articles
Bing Scholar
Similar articles in Bing Scholar
[张歆明]'s Articles
Terms of Use
No data!
Social Bookmark/Share
All comments (0)
No comment.
 

Items in the repository are protected by copyright, with all rights reserved, unless otherwise indicated.