Proceedings of the 2016 International Conference on Computer Engineering and Information Systems

Multi-pretraining Deep Neural Network by DBN and SDA

Authors
Zhen Hu, Zhu-Yin Xue, Tong Cui, Shi-Qiang Zong, Cheng-Long He
Corresponding Author
Zhen Hu
Available Online November 2016.
DOI
https://doi.org/10.2991/ceis-16.2016.11How to use a DOI?
Keywords
pretrain; deep belief network; stacked de-noising auto-encoder
Abstract
Pretraining is widely used in deep neutral network and one of the most famous pretraining models is Deep Belief Network (DBN) which is composed by stacking Restricted Boltzmann Machine (RBM). The optimization formulas are different during the pretraining process for different pretraining models. In this paper, we pretrained deep neutral network by different pretraining models and hence investigated the difference between DBN and Stacked De-noising Auto-encoder (SDA) composed by stacking De-noising Auto-encoder (DA) when used as pretraining model. The experimental results show that DBN get a better initial model. However the model converges to a relatively worse model after the finetuning process. Yet after pretrained by SDA for the second time the model converges to a better model if finetuned.
Open Access
This is an open access article distributed under the CC BY-NC license.

Download article (PDF)

Proceedings
2016 International Conference on Computer Engineering and Information Systems
Part of series
Advances in Computer Science Research
Publication Date
November 2016
ISBN
978-94-6252-283-1
DOI
https://doi.org/10.2991/ceis-16.2016.11How to use a DOI?
Open Access
This is an open access article distributed under the CC BY-NC license.

Cite this article

TY  - CONF
AU  - Zhen Hu
AU  - Zhu-Yin Xue
AU  - Tong Cui
AU  - Shi-Qiang Zong
AU  - Cheng-Long He
PY  - 2016/11
DA  - 2016/11
TI  - Multi-pretraining Deep Neural Network by DBN and SDA
BT  - 2016 International Conference on Computer Engineering and Information Systems
PB  - Atlantis Press
UR  - https://doi.org/10.2991/ceis-16.2016.11
DO  - https://doi.org/10.2991/ceis-16.2016.11
ID  - Hu2016/11
ER  -