Academic discussion of bounded optimality and AI modeling of human behavior

account the cost of computation. Real agents need to modulate the amount of time they spend thinking by the effect the extra thought has on the results of a decision. If you’re trying to choose a toothbrush, you probably don’t need to consider all four thousand listings for manual toothbrushes on Amazon.com before making a purchase: You trade off the time you spend looking with the difference it makes in the quality of the outcome. This trade-off can be formalized, resulting in a model of rational behavior that artificial- intelligence researchers call “bounded optimality.” The bounded-optimal agent doesn’t focus on always choosing exactly the right action to take but rather on finding the right algorithm to follow in order to find the perfect balance between making mistakes and thinking too much. Bounded optimality bridges the gap between rationality and heuristics. By describing behavior as the result of a rational choice about how much to think, it provides a generalizable theory—that is, one that can be applied in new situations. Sometimes the simple strategies that have been identified as heuristics that people follow turn out to be bounded-optimal solutions. So, rather than condemning the heuristics that people use as irrational, we can think of them as a rational response to constraints on computation. Developing bounded optimality as a theory of human behavior is an ongoing project that my research group and others are actively pursuing. If these efforts succeed, they will provide us with the most important ingredient we need for making artificial- intelligence systems smarter when they try to interpret people’s actions, by enabling a generative model for human behavior. Taking into account the computational constraints that factor into human cognition will be particularly important as we begin to develop automated systems that aren’t subject to the same constraints. Imagine a superintelligent AI system trying to figure out what people care about. Curing cancer or confirming the Riemann hypothesis, for instance, won’t seem, to such an AI, like things that are all that important to us: If these solutions are obvious to the superintelligent system, it might wonder why we haven’t found them ourselves, and conclude that those problems don’t mean much to us. If we cared and the problems were so simple, we would have solved them already. A reasonable inference would be that we do science and math purely because we enjoy doing science and math, not because we care about the outcomes. Anybody who has young children can appreciate the problem of trying to interpret the behavior of an agent that is subject to computational constraints different from one’s own. Parents of toddlers can spend hours trying to disentangle the true motivations behind seemingly inexplicable behavior. As a father and a cognitive scientist, I found it was easier to understand the sudden rages of my two-year-old when I recognized that she was at an age where she could appreciate that different people have different desires but not that other people might not know what her own desires were. It’s easy to understand, then, why she would get annoyed when people didn’t do what she (apparently transparently) wanted. Making sense of toddlers requires building a cognitive model of the mind of a toddler. Superintelligent AI systems face the same challenge when trying to make sense of human behavior. Superintelligent AI may still be a long way off. In the short term, devising better models of people can prove extremely valuable to any company that makes money by analyzing human behavior—which at this point is pretty much every company that does business on the Web. Over the last few years, significant new commercial technologies 55 HOUSE_OVERSIGHT_016315