How Dopamine & Serotonin Shape Decisions, Motivation & Learning | Dr. Read Montague

Dopamine and serotonin dynamically shape motivation, learning, and decision-making, with dopamine encoding positive expectations and learning, and serotonin encoding negative events and active waiting.

• Dopamine is a learning signal, continuously updating expectations, not just for final rewards, but for every step towards a goal.

• Serotonin typically acts in opposition to dopamine, encoding negative events and promoting cautious waiting.

• The underlying algorithms governing dopamine and serotonin dynamics are deeply conserved across species and are being leveraged in advanced AI systems.

Most people misunderstand dopamine as solely linked to pleasure; however, Dr. Reed Montague explains that it primarily functions as a learning signal by encoding continuous updates in expectations, not just final outcomes. This temporal difference learning mechanism, observed across species and in advanced AI, drives motivation and the pursuit of goals. Serotonin acts as an opponent to dopamine, signaling negative events and promoting active waiting, with SSRIs potentially altering dopamine's rewarding properties by increasing serotonin in dopamine terminals.

Dopamine as Learning Signal

• 00:03:51 Dopamine is primarily a learning signal, controlling learning through its fluctuations. While it also influences motivation and feelings, its role in directly causing feeling states is less understood. This neuromodulator is a central player in the brain's algorithms, orchestrating how behavior adjusts and how the nervous system changes in response to ongoing experiences and updated predictions.

Temporal Difference Learning

• 00:07:15 Dopamine encodes 'temporal difference errors,' which are not just the difference between expectation and a final outcome, but continuous fluctuations between successive predictions. This sophisticated learning rule allows animals to learn over long stretches of 'nothing' and chain events, such as associating a sound with a light, which then predicts a reward, leading to robust learning even without immediate reinforcement.

AI and Biological Algorithms

• 00:11:32 The temporal difference reinforcement learning algorithm, which dopamine encodes in brains from C. elegans to humans, is the same algorithm used by DeepMind to create world-champion AI programs like AlphaGo Zero. This convergence highlights that biological learning rules have been externalized into AI, leading to breakthroughs that now supersede human capabilities in certain domains, creating an interesting recursion between biology and artificial intelligence.

Dopamine & Motivation

• 00:26:47 Dopamine's fluctuating signals encode changes in expectation, acting as crucial signals for determining motivation levels and driving forward movement, both physically and in thought. This mechanism encourages continuous engagement in 'foraging mode,' where systems are designed to constantly update expectations and push individuals to keep pursuing new goals, as stagnation would be detrimental to survival.

Dopamine and Parkinson's

• 00:32:12 Parkinson's disease, characterized by the loss of 70-75% of dopamine neurons, results in a 'flat value function' where the brain struggles to perceive differential value in things. The increased noise-to-signal ratio in dopamine signaling prevents the nervous system from accurately reading value cues, leading to an 'active freezing disease' where the individual remains put due to a lack of motivation to transition to new actions.

Dopamine, ADHD, and Focus

• 00:39:48 Dopamine-related neuromodulators play a role in stabilizing brain states and thought sequences, influencing focus. Bees, for example, exhibit an 'ADD-like' exploratory mode and a 'concentration' mode, mediated by related chemicals like octopamine. Stimulants for ADHD, which increase dopamine and norepinephrine, stabilize brain states to promote narrower, more sustained focus, suggesting a balance between exploration and exploitation is inherent in biological systems.

Serotonin's Opponent Role

• 01:03:55 Dopamine and serotonin typically operate as opponent neuromodulators; when one increases, the other decreases. While dopamine is associated with learning about positive things, serotonin is linked to active waiting and learning about negative outcomes. This opponent signaling is observed across species, with serotonin helping to encode unwanted outcomes and modulating responses to aversive events.

SSRIs and Dopamine Terminals

• 01:06:08 SSRIs, which prevent serotonin reuptake, increase serotonin levels but also cause serotonin to enter dopamine terminals via dopamine transporters. This influx of serotonin into dopamine pathways can diminish the rewarding properties of positive events and may lead to negative conditioning on things that should be pursued, potentially explaining some of the complex and sometimes adverse effects of SSRIs, including anhedonia and worsened mood.