AI risk framed as environmental threat rather than social impact

Calibrating the Al-Risk Message While uncannily prescient, the AI-risk message from the original dissidents has a giant flaw—as does the version dominating current public discourse: Both considerably understate the magnitude of the problem as well as AI’s potential upside. The message, in other words, does not adequately convey the stakes of the game. Wiener primarily warned of the social risks—trisks stemming from careless integration of machine-generated decisions with governance processes and misuse (by humans) of such automated decision making. Likewise, the current “serious” debate about AI risks focuses mostly on things like technological unemployment or biases in machine learning. While such discussions can be valuable and address pressing short- term problems, they are also stunningly parochial. I’m reminded of Yudkowsky’s quip in a blog post: “[A]sking about the effect of machine superintelligence on the conventional human labor market is like asking how US—Chinese trade patterns would be affected by the Moon crashing into the Earth. There would indeed be effects, but you’d be missing the point.” In my view, the central point of the AI risk is that superintelligent AI is an environmental risk. Allow me to explain. In his “Parable of the Sentient Puddle,” Douglas Adams describes a puddle that wakes up in the morning and finds himself in a hole that fits him “staggeringly well.” From that observation, the puddle concludes that the world must have been made for him. Therefore, writes Adams, “the moment he disappears catches him rather by surprise.” To assume that AI risks are limited to adverse social developments is to make a similar mistake. The harsh reality is that the universe was not made for us; instead, we are fine- tuned by evolution to a very narrow range of environmental parameters. For instance, we need the atmosphere at ground level to be roughly at room temperature, at about 100 kPa pressure, and have a sufficient concentration of oxygen. Any disturbance, even temporary, of this precarious equilibrium and we die in a matter of minutes. Silicon-based intelligence does not share such concerns about the environment. That’s why it’s much cheaper to explore space using machine probes rather than “cans of meat.” Moreover, Earth’s current environment is almost certainly suboptimal for what a superintelligent AI will greatly care about: efficient computation. Hence we might find our planet suddenly going from anthropogenic global warming to machinogenic global cooling. One big challenge that AI safety research needs to deal with is how to constrain a potentially superintelligent AI—an AI with a much larger footprint than our own—from rendering our environment uninhabitable for biological life-forms. Interestingly, given that the most potent sources both of AI research and AlI-risk dismissals are under big corporate umbrellas, if you squint hard enough the “AI as an environmental risk” message looks like the chronic concern about corporations skirting their environmental responsibilities. Conversely, the worry about AI’s social effects also misses most of the upside. It’s hard to overemphasize how tiny and parochial the future of our planet is, compared with the full potential of humanity. On astronomical timescales, our planet will be gone soon (unless we tame the sun, also a distinct possibility) and almost all the resources— atoms and free energy—to sustain civilization in the long run are in deep space. Eric Drexler, the inventor of nanotechnology, has recently been popularizing the 74 HOUSE_OVERSIGHT_016877