Proceedings of the 2nd International Conference on Computer Science and Electronics Engineering (ICCSEE 2013)

Design and Implementation of Distributed Crawler System for Opinion Mining

Authors
Yunqi Gao, Chunlin Peng
Corresponding Author
Yunqi Gao
Available Online March 2013.
DOI
10.2991/iccsee.2013.516How to use a DOI?
Keywords
distributed system, crawler, public opinion, PageRank,
Abstract

With the development of Internet, Network public opinion has been serving an import role in reflection of social public opinion. As there are a large number of websites and forums on the Internet, we need a powerful crawler system which can meet the demands of opinion mining. However, common crawler systems concern more about ranking and recommendation algorithms, which is less important in opinion mining. In this article, we introduced the design and implementation of a distributed crawler system for opinion mining. We also introduced some extra parameters such as keywords count and published time into the ranking and refreshing strategies. Experimental results demonstrate that the system can well support different sites, and the improved strategies can greatly enhance the crawling and monitoring efficiency.

Copyright
© 2013, the Authors. Published by Atlantis Press.
Open Access
This is an open access article distributed under the CC BY-NC license (http://creativecommons.org/licenses/by-nc/4.0/).

Download article (PDF)

Volume Title
Proceedings of the 2nd International Conference on Computer Science and Electronics Engineering (ICCSEE 2013)
Series
Advances in Intelligent Systems Research
Publication Date
March 2013
ISBN
978-90-78677-61-1
ISSN
1951-6851
DOI
10.2991/iccsee.2013.516How to use a DOI?
Copyright
© 2013, the Authors. Published by Atlantis Press.
Open Access
This is an open access article distributed under the CC BY-NC license (http://creativecommons.org/licenses/by-nc/4.0/).

Cite this article

TY  - CONF
AU  - Yunqi Gao
AU  - Chunlin Peng
PY  - 2013/03
DA  - 2013/03
TI  - Design and Implementation of Distributed Crawler System for Opinion Mining
BT  - Proceedings of the 2nd International Conference on Computer Science and Electronics Engineering (ICCSEE 2013)
PB  - Atlantis Press
SP  - 2052
EP  - 2053
SN  - 1951-6851
UR  - https://doi.org/10.2991/iccsee.2013.516
DO  - 10.2991/iccsee.2013.516
ID  - Gao2013/03
ER  -