A Close-to-linear Topic Detection Algorithm using Relative Entropy based Relevance Model and Inverted Indices Retrieval

DOI

10.1080/18756891.2012.718156How to use a DOI?

Keywords

Topic Detection, Link Topic Detection, Retrospective Event Detection, Information Retrieval, Relevance Models, Inverted Indices

Abstract

Although timely access to information is becoming increasingly important and gaining such access is no longer a problem, the capacity for humans to assimilate such huge amounts of information is limited. Topic Detection(TD) is then a promising research area that addresses speedy access of desired information. However, ironically, the time complexity of existing TD algorithms themselves is usually up to the -th power of . Linear performance requirement of real world topic detection has not been significantly addressed. This paper reveals a new patented topic detection algorithm called that combines elevance odel with nformation etrieval technique to improve on time efficiency. Relevance Model(RM) is a theoretical extension of statistical language modeling that was developed for the task of document retrieval. To reduce the costs of fetching RM, we reduce the number of comparisons for stories by a query-based approach that makes similar stories exist in the top-k query results. We also build our query based on inverted indices, which have the complexity close to linear. The time cost of rest of operations in the topic detection process is a constant. Hence, the total complexity of topic detection algorithm should be close to linear as shown in experimental results. In addition, also gains better detection rates and robustness by relative entropy based topic model design

Copyright

© 2017, the Authors. Published by Atlantis Press.

Open Access

This is an open access article distributed under the CC BY-NC license (http://creativecommons.org/licenses/by-nc/4.0/).

Download article (PDF)

TY  - JOUR
AU  - Steve Kansheng Shi
AU  - Lemin Li
PY  - 2012
DA  - 2012/08/01
TI  - A Close-to-linear Topic Detection Algorithm using Relative Entropy based Relevance Model and Inverted Indices Retrieval
JO  - International Journal of Computational Intelligence Systems
SP  - 735
EP  - 744
VL  - 5
IS  - 4
SN  - 1875-6883
UR  - https://doi.org/10.1080/18756891.2012.718156
DO  - 10.1080/18756891.2012.718156
ID  - Shi2012
ER  -