Optimized TF-IDF Algorithm with the Adaptive Weight of Position of Word

DOI

10.2991/aiie-16.2016.28How to use a DOI?

Keywords

text feature extraction; adaptive weight; weight of position; Term Frequency-Inverse Document Frequency(TF-IDF)

Abstract

The classical TF-IDF algorithm only considers the weight of the term frequency and the inverse document frequency, without considering the weights of other feature of word. After the author analyzing summary of Chinese expression habits, an adaptive weight of position of word algorithm based on TF-IDF is proposed in this paper, which can be called TF-IDF-AP algorithm. The TF-IDF-AP algorithm can dynamically determine the weight of position of word according to the position of word. This paper introduced the vector space model (VSM) and designed comparative experiment under the scene of Chinese document clustering. The results show that the F-measure of TF-IDF-AP algorithm has been improved by 12.9% comparing with the classical TF-IDF algorithm.

Copyright

© 2016, the Authors. Published by Atlantis Press.

Open Access

This is an open access article distributed under the CC BY-NC license (http://creativecommons.org/licenses/by-nc/4.0/).

Download article (PDF)

TY  - CONF
AU  - Jie Chen
AU  - Cai Chen
AU  - Yi Liang
PY  - 2016/11
DA  - 2016/11
TI  - Optimized TF-IDF Algorithm with the Adaptive Weight of Position of Word
BT  - Proceedings of the 2016 2nd International Conference on Artificial Intelligence and Industrial Engineering (AIIE 2016)
PB  - Atlantis Press
SP  - 114
EP  - 117
SN  - 1951-6851
UR  - https://doi.org/10.2991/aiie-16.2016.28
DO  - 10.2991/aiie-16.2016.28
ID  - Chen2016/11
ER  -