Academic discussion of ethical development stages in AGI systems

12.4 Ethical Synergy 217 Stage Characteristics Pre-ethical e Piagetan infantile to early concrete (aka pre-operational) e Radical selfishness or selflessness may, but do not neces- sarily, occur @ No coherent, consistent pattern of consideration for the rights, intentions or feelings of others e Empathy is generally present, but erratically Conventional Ethics e Concrete cognitive basis e Perry’s Dualist and Multiple stages e The common sense of the Golden Rule is appreciated, with cultural conventions for abstracting principles from behaviors @ One’s own ethical behavior is explicitly compared to that of others e Development of a functional, though limited, theory of mind e Ability to intuitively conceive of notions of fairness and rights e Appreciation of the concept of law and order, which may sometimes manifest itself as systematic obedience or sys- tematic disobedience e Empathy is more consistently present, especially with others who are directly similar to oneself or in situations similar to those one has directly experienced e Degrees of selflessness or selfishness develop based on eth- ical groundings and social interactions. Table 12.4: Integrative Model of the Stages of Ethical Development, Part 1 logical inference step to take in thinking about applying the notions of rights and fairness to a given situation. Gilligan’s stages correspond to increasingly sophisticated control of empathic simulation — which in a CogPrime-type AGI system, is carried out by a specific system component devoted to running internal simulations of aspects of the outside world, which includes a subcomponent specifically tuned for simulating sentient actors. The conventional stage has to do with the raw, uncontrolled capability for such simulation; and the post-conventional stage corresponds to its contextual, goal-oriented control. But controlling empathy, clearly, requires subtle management of various uncertain contextual factors, which is exactly what uncertain logical inference is good at — so, in an AGI system combining an uncertain inference component with a simulative component, it is the inference component that would enable the nuanced control of empathy allowing the ascent to Gilligan’s post-conventional stage. In our integrative perspective, in the context of an AGI system integrating inference and simulation components, we suggest that the ascent from the pre-ethical to the conventional stage may be carried out largely via independent activity of these two components. Empathy is needed, and reasoning about fairness and rights are needed, but the two need not intimately and sensitively intersect — though they must of course intersect to some extent. HOUSE_OVERSIGHT_013133