What is “reasoning” in modern AI?

Reasoning in modern AI is a complex topic, with no single definition. It is argued that reasoning can be assessed quantitatively by how well AI systems perform on tasks traditionally associated with deep thinking, like mathematical problem-solving. However, different approaches to reasoning exist, including neurosymbolic programming, where neural networks and symbolic code are combined. This approach is increasingly prevalent in AI systems, particularly in areas like code generation and theorem proving.

Defining Reasoning

• 00:03:08 Reasoning is subjective but can be defined by AI's performance on tasks like mathematical problem-solving or planning. A system demonstrating improved performance on such tasks is considered to be exhibiting reasoning.

Reasoning's How

• 00:04:31 The 'how' of reasoning is distinct from the 'what'. Multiple methods exist, including neurosymbolic programming. The how is relevant to the robustness and computational efficiency of reasoning.

Neurosymbolic Programming

• 00:12:22 Neurosymbolic programming is the use of neural networks and symbolic code in AI models. This approach is increasingly common, with examples like Alpha Proof and code-generating agents using Python interpreters, a symbolic tool.

Abstraction and Modularity

• 00:08:12 Abstraction and modularity are crucial for human reasoning, which modern language models lack. However, there might be a class of language models, potentially with different training objectives, that can achieve robustness without explicit modularity.

Reasoning and Axioms

• 00:14:07 The relationship between reasoning and axioms depends on the definition of reasoning. Mathematical reasoning relies on axioms, but physical reasoning relies on laws of physics. It's possible to imagine reasoning occurring in worlds with different laws of physics, but even those would have different axioms.

Reasoning Systems Combination

• 00:15:07 Reasoning systems can be combined, as humans do. Mathematical reasoning can be combined with empirical reasoning used in natural sciences. Future AI could combine diverse reasoning systems with various capabilities.

AI Co-Pilot for Math

• 01:05:49 The potential for an AI co-pilot for mathematical discovery by 2026 is a realistic possibility. AI can serve as a significant contributor, if not the sole author, on mathematical or computer science papers with mathematical components.

AI in Mathematical Discovery

• 01:09:21 AI can revolutionize mathematical discovery by enabling complex theorems that are too large for humans to fully understand. This could lead to new mathematical collaborations between humans and AI, resulting in increased productivity and possibly new mathematical discoveries.

AI for Code Generation

• 01:13:05 There is a trade-off between interpretability and capability in AI code generation. While a fully human-interpretable Linux kernel is possible, a more desirable approach might be to combine human-comprehensible interfaces with AI that handles low-level details. This allows for human involvement in maintenance and evolution of systems.

AI and the Halting Problem

• 01:41:23 While the halting problem is undecidable in theory, practical programs can be analyzed for termination using various methods, including inductive proofs. AI can be used to automate the search for such proofs, especially for realistic programs.

LLM-Based Reasoning

• 01:43:33 LLMs can be used for concept abstraction in AI reasoning systems, which can help improve performance on synthetic benchmarks. LLMs' ability to abstract text into higher-level concepts is valuable even when no prior knowledge is available.