Technical discussion of AGI inference control and self-modification

11.4 Piaget’s Stages in the Context of Uncertain Inference 203 Stage 4:Reflexive (Post-formal) { Inference Rules New Knowledge 4 } | Knowledge about) i World > aaa Inference Control Strategy | Knowledge about how to control inference > ( Inference Rules ) a i} New Knowledge about how to control inference : Inference Control Strategy 4 ?) \ ¢ \ “ { Inference Rules \ ; ‘ c “ y’ \ \ | Knowledge about New knowledge about ‘ | inference rules [ > 7 inference rules ee Inference Control Strategy wy ~~ ~~ Fig. 11.8: The Reflexive Stage stage adds an entire new feedback loop for reasoning about inference control itself, as shown in Figure 11.8. As a very concrete example, in later chapters we will see that, while PLN is founded on probability theory, it also contains a variety of heuristic assumptions that inevitably introduce a certain amount of error into its inferences. For example, PLN’s probabilistic deduction embodies a heuristic independence assumption. Thus PLN contains an alternate deduction formula called the “concept geometry formula’ that is better in some contexts, based on the assumption that ConceptNodes embody concepts that are roughly spherically-shaped in attribute space. A highly advanced CogPrime system could potentially augment the independence-based and concept- geometry-based deduction formulas with additional formulas of its own derivation, optimized to minimize error in various contexts. This is a simple and straightforward example of reflexive cognition — it illustrates the power accessible to a cognitive system that has formalized and reflected upon its own inference processes, and that possesses at least some capability to modify these. In general, AGI systems can be expected to have much broader and deeper capabilities for self-modification than human beings. Ultimately it may make sense to view the AGI systems we implement as merely “initial conditions" for ongoing self-modification and selforganization. Chapter ?? discusses some of the potential technical details underlying this sort of thorough- going AGI self-modification. HOUSE_OVERSIGHT_013119