Journal of Robotics, Networking and Artificial Life

Volume 7, Issue 4, March 2021, Pages 240 - 244

Network with Sub-networks: Layer-wise Detachable Neural Network

Authors
Ninnart Fuengfusin*, Hakaru Tamukoh
Graduate School of Life Science and Systems Engineering, Kyushu Institute of Technology, 2-4 Hibikino, Wakamatsu-ku, Kitakyushu, Fukuoka 808-0196, Japan
*Corresponding author. Email: fuengfusin.ninnart553@mail.kyutech.jp
Corresponding Author
Ninnart Fuengfusin
Received 11 November 2019, Accepted 19 June 2020, Available Online 21 December 2020.
DOI
https://doi.org/10.2991/jrnal.k.201215.006How to use a DOI?
Keywords
Model compression, neural networks, multilayer perceptron, supervised learning
Abstract

In this paper, we introduce a network with sub-networks: a neural network whose layers can be detached into sub-neural networks during the inference phase. To develop trainable parameters that can be inserted into both base-model and sub-models, first, the parameters of sub-models are duplicated in the base-model. Each model is separately forward-propagated, and all models are grouped into pairs. Gradients from selected pairs of networks are averaged and used to update both networks. With the Modified National Institute of Standards and Technology (MNIST) and Fashion-MNIST datasets, our base-model achieves identical test-accuracy to that of regularly trained models. However, the sub-models result in lower test-accuracy. Nevertheless, the sub-models serve as alternative approaches with fewer parameters than those of regular models.

Copyright
© 2020 The Authors. Published by Atlantis Press B.V.
Open Access
This is an open access article distributed under the CC BY-NC 4.0 license (http://creativecommons.org/licenses/by-nc/4.0/).

Download article (PDF)
View full text (HTML)

Journal
Journal of Robotics, Networking and Artificial Life
Volume-Issue
7 - 4
Pages
240 - 244
Publication Date
2020/12
ISSN (Online)
2352-6386
ISSN (Print)
2405-9021
DOI
https://doi.org/10.2991/jrnal.k.201215.006How to use a DOI?
Copyright
© 2020 The Authors. Published by Atlantis Press B.V.
Open Access
This is an open access article distributed under the CC BY-NC 4.0 license (http://creativecommons.org/licenses/by-nc/4.0/).

Cite this article

TY  - JOUR
AU  - Ninnart Fuengfusin
AU  - Hakaru Tamukoh
PY  - 2020
DA  - 2020/12
TI  - Network with Sub-networks: Layer-wise Detachable Neural Network
JO  - Journal of Robotics, Networking and Artificial Life
SP  - 240
EP  - 244
VL  - 7
IS  - 4
SN  - 2352-6386
UR  - https://doi.org/10.2991/jrnal.k.201215.006
DO  - https://doi.org/10.2991/jrnal.k.201215.006
ID  - Fuengfusin2020
ER  -