International Journal of Computational Intelligence Systems

Volume 12, Issue 2, 2019, Pages 1613 - 1621

Character-Level Quantum Mechanical Approach for a Neural Language Model

Authors
Zhihao Wang1, *, Min Ren2, Xiaoyan Tian3, Xia Liang1
1School of Management Science and Engineering, Shandong University of Finance and Economics, Jinan 250014, China
2School of Mathematics and Quantitative Economics, Shandong University of Finance and Economics, Jinan 250014, China
3Shandong Police College, Jinan 250014, China
*Corresponding author. Email: wzh861125@hotmail.com
Corresponding Author
Zhihao Wang
Received 22 April 2019, Accepted 9 November 2019, Available Online 25 November 2019.
DOI
10.2991/ijcis.d.191114.001How to use a DOI?
Keywords
Character-level; Quantum theory; Network-in-network; Language model
Abstract

This article proposes a character-level neural language model (NLM) that is based on quantum theory. The input of the model is the character-level coding represented by the quantum semantic space model. Our model integrates a convolutional neural network (CNN) that is based on network-in-network (NIN). We assessed the effectiveness of our model through extensive experiments based on the English-language Penn Treebank dataset. The experiments results confirm that the quantum semantic inputs work well for the language models. For example, the PPL of our model is 10%–30% less than the states of the arts, while it keeps the relatively smaller number of parameters (i.e., 6 m).

Copyright
© 2019 The Authors. Published by Atlantis Press SARL.
Open Access
This is an open access article distributed under the CC BY-NC 4.0 license (http://creativecommons.org/licenses/by-nc/4.0/).

Download article (PDF)
View full text (HTML)

Journal
International Journal of Computational Intelligence Systems
Volume-Issue
12 - 2
Pages
1613 - 1621
Publication Date
2019/11/25
ISSN (Online)
1875-6883
ISSN (Print)
1875-6891
DOI
10.2991/ijcis.d.191114.001How to use a DOI?
Copyright
© 2019 The Authors. Published by Atlantis Press SARL.
Open Access
This is an open access article distributed under the CC BY-NC 4.0 license (http://creativecommons.org/licenses/by-nc/4.0/).

Cite this article

TY  - JOUR
AU  - Zhihao Wang
AU  - Min Ren
AU  - Xiaoyan Tian
AU  - Xia Liang
PY  - 2019
DA  - 2019/11/25
TI  - Character-Level Quantum Mechanical Approach for a Neural Language Model
JO  - International Journal of Computational Intelligence Systems
SP  - 1613
EP  - 1621
VL  - 12
IS  - 2
SN  - 1875-6883
UR  - https://doi.org/10.2991/ijcis.d.191114.001
DO  - 10.2991/ijcis.d.191114.001
ID  - Wang2019
ER  -