Designing a Safe Motivational System for Intelligent Machines

Waser, Mark R.

doi:agi.2010.21

As machines become more intelligent, more flexible, more autonomous and more powerful, the questions of how they should choose their actions and what goals they should pursue become critically important. Drawing upon the examples of and lessons learned from humans and lesser creatures, we propose a hierarchical motivational system flowing from an abstract invariant super-goal that is optimal for all (including the machines themselves) to low-level reflexive 'sensations, emotions, and attentional effects' and other enforcing biases to ensure reasonably 'correct' behavior even under conditions of uncertainty, immaturity, error, malfunction, and even sabotage.


title:		Designing a Safe Motivational System for Intelligent Machines
publication:		AGI-10
part of series:		Advances in Intelligent Systems Research
ISBN:		978-90-78677-36-9
ISSN:		1951-6851
DOI:		doi:10.2991/agi.2010.21 (how to use a DOI)
author(s):		Mark R. Waser
publication date:		March 2010
abstract:		As machines become more intelligent, more flexible, more autonomous and more powerful, the questions of how they should choose their actions and what goals they should pursue become critically important. Drawing upon the examples of and lessons learned from humans and lesser creatures, we propose a hierarchical motivational system flowing from an abstract invariant super-goal that is optimal for all (including the machines themselves) to low-level reflexive 'sensations, emotions, and attentional effects' and other enforcing biases to ensure reasonably 'correct' behavior even under conditions of uncertainty, immaturity, error, malfunction, and even sabotage.
copyright:		© Atlantis Press. This article is distributed under the terms of the Creative Commons Attribution License, which permits non-commercial use, distribution and reproduction in any medium, provided the original work is properly cited.
full text:		AGI-10-paper_170.pdf (252 K)