ISSN 1000-1239 CN 11-1777/TP

计算机研究与发展 ›› 2017, Vol. 54 ›› Issue (6): 1133-1143.doi: 10.7544/issn1000-1239.2017.20160804

所属专题: 2017优青专题

• 信息处理 • 上一篇    下一篇

搜索引擎用户满意度评估

刘奕群   

  1. (清华大学计算机科学与技术系 北京 100084) (yiqunliu@tsinghua.edu.cn)
  • 出版日期: 2017-06-01
  • 基金资助: 
    国家自然科学基金优秀青年科学基金项目(61622208)

Satisfaction Prediction of Web Search Users

Liu Yiqun   

  1. (Department of Computer Science & Technology, Tsinghua University, Beijing 100084)
  • Online: 2017-06-01

摘要: 用户满意度评估一直是互联网搜索领域的研究热点,并具有3方面的挑战:1)传统的搜索性能评估方法大多基于对检索结果相关性的标注,但大多数基于相关性标注的评价指标并非针对互联网搜索环境而设计,其结果与搜索用户主观满意度之间的关系缺乏相应研究;2)大多数已有的工作都基于搜索结果同质化的假设,但随着搜索引擎的发展,异质化的搜索结果元素开始频繁地出现在搜索结果列表中;3)已有的关于搜索满意度评估的工作主要基于用户的点击和查询修改行为开展,但实际搜索中会有大量的用户会话中缺失此类信息.总结了近期为解决这些研究问题开展的实验研究工作:1)构建了用户行为实验系统,分析了结果相关性与用户所感知到的结果效用和满意度之间的关系;2)基于仔细设计的异质化搜索结果页面,定量地分析了垂直搜索结果的质量、展现形式、位置等因素对用户满意度的影响;3)受现有的采用鼠标移动信息进行搜索结果相关性预测的工作启发,提出了在搜索结果页面上抽取用户鼠标移动行为模式并进行满意度评估的方法.实验结果表明:在真实搜索环境下,所提出的方法优于现有的模型.

关键词: 搜索满意度, 相关性, 垂直搜索, 鼠标移动信息, 网络搜索引擎

Abstract: User satisfaction is one of the prime concerns for Web search related studies. It is a non-trivial task for three major reasons: 1) Traditional approaches for search performance evaluation mainly rely on editorial judgments of the relevance of search results. The relationship between search satisfaction and relevance-based evaluation still remains under-investigated. 2) Most existing researches are based on the hypothesis that all results on search result pages (SERPs) are homogeneous while a variety of heterogeneous components have been aggregated into modern SERPs to improve search performance. 3) Most existing studies on satisfaction prediction primarily rely on users’ click-through and query reformulation behaviors but there are plenty of search sessions without such information. In this paper, we summarize our recent efforts to shed light on these research questions. Firstly, we perform a laboratory study to investigate the relationship between relevance and users’ perceived usefulness and satisfaction. After that, we also investigate the impact of vertical results with different qualities, presentation styles and positions on search satisfaction with specifically designed SERPs. Finally, inspired by recent studies in predicting result relevance based on mouse movement patterns, we propose novel strategies to extract high quality mouse movement patterns from SERPs for satisfaction prediction. Experimental results show that our proposed method outperforms existing approaches in heterogeneous search environment.

Key words: search satisfaction, relevance, aggregated search, mouse movement, Web search engine

中图分类号: