98
Enumerated, Evolved, or Corrigible
Corrigibility is the idea of building AI agents that reason as if they are incomplete and potentially
flawed in dangerous ways. Since the AI agent apprehends that it is incomplete, it is encouraged to
maintain a collaborative and not deceptive relationship with its programmers since the programmers
may be able to help provide more complete information, even while both parties maintain different
ethics systems. Thus a highly-advanced AI agent might be built that is open to online value
learning, modification, correction, and ongoing interaction with humans. Corrigibility is proposed
as a reasoning-based alternative to enumerated and evolved computational ethics systems, and also
as an important ‘escape velocity’ project. Escape velocity refers to being able to bridge the
competence gap between the current situation of not yet having human moral concepts reliably
instantiated in AI systems, and the potential future of true moral superintelligences indispensably
orchestrating many complex societal activities.
Lethal Autonomous Weapons
Machine cognition features prominently in lethal autonomous weapons where weapon systems are
increasingly autonomous, making their own decisions in target selection and engagement without
human input. The banning of autonomous weapons systems is currently under debate. On one side,
detractors argue that full autonomy is too much, and that these weapons no longer have “meaningful
human control” as a positive obligation, and do not comply with the Geneva Convention’s Martens
Clause requiring that fully autonomous weapons comply with principles of humanity and
conscience.
On the other side, supporters argue that machine morality might exceed human morality, and be
more accurately and precisely applied. Ethically, it is not clear if weapons systems should be
considered differently than other machine systems. For example, the Nationwide Kidney Exchange
automatically allocates two transplant kidneys per week, where the lack of human involvement has
been seen positively as a response to the agency problem.
See more at:
http://www.kurzweilai.net
