Andrej Karpathy — AGI is still a decade away

Andrej Karpathy asserts that despite impressive early capabilities, the development of truly autonomous and general AI agents will take a decade due to significant cognitive deficits, rather than just a year, and advocates for a new approach to education centered on efficient knowledge transfer.

• Truly general AI agents are a decade away, requiring breakthroughs in continual learning, multimodality, and complex cognitive function.

• Current reinforcement learning methods are inefficient for intelligence tasks; new approaches are needed to avoid model gameability and collapse.

• Andrej Karpathy's Eureka project aims to revolutionize education by creating highly efficient 'ramps to knowledge' to empower humans in an AI-driven future.

Andrej Karpathy believes that the current excitement around AI agents overstates their immediate capabilities, framing the current period as the 'decade of agents' rather than the 'year of agents' because significant challenges like continual learning, multimodality, and robust problem-solving remain. He emphasizes that current models lack the complex cognitive structures of humans and animals, requiring foundational advancements beyond simple scaling. Karpathy also introduces his educational initiative, Eureka, aiming to build a 'Starfleet Academy' for frontier technology, providing highly efficient learning 'ramps' to empower humans in an AI-driven future.

Timeline for AI Agents

• 00:00:58 Karpathy states that while early AI agents like Claude and Codex are impressive, calling this the 'year of agents' is an overprediction. He believes it will take a decade to overcome current bottlenecks such as lack of continual learning, multimodality, computer use, and overall cognitive deficits that prevent AI from functioning like a true human employee or intern. His projection is based on 15 years of experience observing AI predictions and industry progress.

AI Development Shifts

• 00:04:07 The AI field has experienced seismic shifts, including the rise of deep learning with AlexNet, the misstep of focusing solely on reinforcement learning in games (Atari), and the shift towards large language models (LLMs) to build powerful representations before tackling complex agents. Karpathy's early work on agents operating web pages (Universe project) was too premature without robust language models, highlighting the need to develop foundational representation power first.

Animal vs. AI Intelligence

• 00:08:41 Karpathy is cautious about drawing analogies between animal and AI intelligence, as animals evolved with built-in hardware, unlike AI 'ghosts' or 'spirits' trained by imitating internet data. He argues that animals rely more on maturation and evolution's outer loop, with little reinforcement learning for intelligence tasks, whereas AI uses 'crappy evolution' through pre-training to build initial knowledge.

In-Context Learning vs. Pre-training

• 00:18:34 Pre-training, while providing vast knowledge, results in a 'hazy recollection' due to massive data compression, similar to long-term memory. In contrast, in-context learning, which leverages the context window and KV cache, acts as a 'working memory' that is directly accessible to the neural network. This allows for visible intelligence and adaptation during a single session, differing significantly from the model's more compressed, generalized knowledge from pre-training.

Missing Human Cognitive Parts

• 00:20:12 Current AI models, particularly large language models, lack many brain parts essential for human-like intelligence, such as a hippocampus for long-term memory distillation and processing, or an amygdala for emotions and instincts. While transformers act as a general cortical tissue and reasoning traces resemble the prefrontal cortex, many ancient brain nuclei remain unexplored and unreplicated, contributing to current models' cognitive deficits.

Challenges in RL & Model Collapse

• 00:41:36 Reinforcement learning (RL) is fundamentally flawed for complex intelligence tasks due to its 'sucking supervision through a straw' approach, broadcasting a single reward signal across entire trajectories, leading to noisy and often incorrect learning. A major bottleneck for process-based supervision is the challenge of assigning partial credit automatically without creating 'gameable' LLM judges that produce adversarial, nonsensical solutions due to out-of-sample generalization issues.

AI & Economic Growth

• 01:22:56 Karpathy views AI as a continuous extension of computing and automation, contributing to a hyper-exponential growth trajectory that has been ongoing for decades, rather than a discrete, sudden 'intelligence explosion.' He predicts AI will integrate gradually into society, similar to past technologies like computers or mobile phones, maintaining a roughly constant rate of growth as it slowly diffuses and automates tasks across the economy.

Eureka: Ramps to Knowledge

• 01:57:42 Karpathy's new initiative, Eureka, aims to build a 'Starfleet Academy'—an elite institution for technical knowledge that creates highly efficient 'ramps to knowledge.' This involves developing state-of-the-art courses, like LLM101N, that systematically break down complex topics into easily digestible steps, maximizing 'eurekas per second.' He envisions a future where AI tutors, informed by deep pedagogical understanding and human faculty, make learning any subject trivial and desirable, akin to going to the gym for self-betterment.