International Journal of Computational Intelligence Systems

<Previous Article In Issue
Next Article In Issue>
Volume 12, Issue 2, 2019, Pages 1221 - 1231

Composable Instructions and Prospection Guided Visuomotor Control for Robotic Manipulation

Authors
Quanquan Shao, Jie Hu*, Weiming Wang, Yi Fang, Mingshuo Han, Jin Qi, Jin Ma
Institute of Knowledge Based Engineering, School of Mechanical Engineering, Shanghai Jiao Tong University, No. 800 Dongchuan Road, Minhang District, Shanghai, 200240, China
*Corresponding author. Email: hujie@sjtu.edu.cn
Corresponding Author
Jie Hu
Received 17 July 2019, Accepted 16 October 2019, Available Online 30 October 2019.
DOI
10.2991/ijcis.d.191017.001How to use a DOI?
Keywords
Composable instructions; Motion generation; Prospection; Imitation learning; Visuomotor control; Robotic manipulation
Abstract

Deep neural network-based end-to-end visuomotor control for robotic manipulation is becoming a hot issue of robotics field recently. One-hot vector is often used for multi-task situation in this framework. However, it is inflexible using one-hot vector to describe multiple tasks and transmit intentions of humans. This paper proposes a framework by combining composable instructions with visuomotor control for multi-task problems. The framework mainly consists of two modules: variational autoencoder (VAE) networks and long short-term memory (LSTM) networks. Perception information of the environment is encoded by VAE into a small latent space. The embedded perception information and composable instructions are combined by the LSTM module to guide robotic motion based on different intentions. Prospection is also used to learn the purposes of instructions, which means not only predicting the next action but also predicting a sequence of future actions at the same time. To evaluate this framework, a series of experiments are conducted in pick-and-place application scenarios. For new tasks, the framework could obtain a success rate of 91.2%, which means it has a good generalization ability.

Copyright
© 2019 The Authors. Published by Atlantis Press SARL.
Open Access
This is an open access article distributed under the CC BY-NC 4.0 license (http://creativecommons.org/licenses/by-nc/4.0/).

Download article (PDF)
View full text (HTML)

<Previous Article In Issue
Next Article In Issue>
Journal
International Journal of Computational Intelligence Systems
Volume-Issue
12 - 2
Pages
1221 - 1231
Publication Date
2019/10/30
ISSN (Online)
1875-6883
ISSN (Print)
1875-6891
DOI
10.2991/ijcis.d.191017.001How to use a DOI?
Copyright
© 2019 The Authors. Published by Atlantis Press SARL.
Open Access
This is an open access article distributed under the CC BY-NC 4.0 license (http://creativecommons.org/licenses/by-nc/4.0/).

Cite this article

risenwbib
TY  - JOUR
AU  - Quanquan Shao
AU  - Jie Hu
AU  - Weiming Wang
AU  - Yi Fang
AU  - Mingshuo Han
AU  - Jin Qi
AU  - Jin Ma
PY  - 2019
DA  - 2019/10/30
TI  - Composable Instructions and Prospection Guided Visuomotor Control for Robotic Manipulation
JO  - International Journal of Computational Intelligence Systems
SP  - 1221
EP  - 1231
VL  - 12
IS  - 2
SN  - 1875-6883
UR  - https://doi.org/10.2991/ijcis.d.191017.001
DO  - 10.2991/ijcis.d.191017.001
ID  - Shao2019
ER  -