Philosophical discussion on the Lucas‑Penrose argument and AI limits

Known Unknowns 207 a system you have and however much you extend it, the system will always be incomplete. And we really do mean; however large. Even an infinitely large formal system would be incomplete. The only way to avoid this problem is with some sort of conspiracy theory where we only come across problems our formal system can already solve. Such a theory is a determined Universe. In a determined Universe, all the mathematical problems we ever solve must be expressed by the formal systems existing in the Universe. We must never encounter a problem where we need to extend the system and break the Gédel limit because we are pre-determined not to do so. The Inconsistency Defense An argument put forward by opponents of the Lucas-Penrose position is that humans are inconsistent formal systems. Inconsistent formal systems are not subject to the incompleteness limit. Humans certainly behave inconsistently with remarkable regularity but simply making inconsistent statements is not sufficient to show the underlying formal system is, itself, inconsistent. Inconsistent beliefs can come simply from making mistakes or reading the same story in two different newspapers! We need a fundamentally inconsistent thinking mechanism inside our brains to break the constraint. The very machinery itself would have to be inconsistent. But this is exactly Penrose’s point. Constructing a machine capable of reasoning in an inconsistent but useful manner would need exotic technology, some sort of non-deterministic, rationalizing computer. The components to make it could not be computer logic as we know it today. All such logic is entirely computationally deterministic. Let me see if I can reframe the Lucas argument. Imagine IBM’s Watson computer was let loose on mathematical reasoning. Watson could scan every mathematical theorem ever written down. It would know every programming language created. It would have its enormous bank of general knowledge to call upon and it could answer many questions. It would sometimes appear inconsistent because the information it had trawled from the Internet would be wrong. But Watson would still be a consistent formal system and Gédel’s theorem says there would be truths Watson could never see. Lucas argues humans can see such truths where a machine cannot, and these truths would allow a human to discover a proof to a mathematical problem that would forever elude Watson. The Lucas argument runs into a brick wall because it asserts we see truths a machine cannot. For each alleged creative step, his opponents simply assert your brain was already sufficiently powerful to perform HOUSE_OVERSIGHT_015897