In Deep Utopia, Nick Bostrom explores the question of what utopia could look like and what problems might be left. The book is centred around a lecture series given by a fictitious Bostrom, “The Problem of Utopia”—the problem we will face after we have solved all other problems.
Importantly, the book doesn’t try to predict what will happen, but rather explores what could happen if things go about as well as they possibly could.
Estimated time: 22 mins
Buy Deep Utopia at: Amazon (affiliate link)
Key Takeaways from Deep Utopia
- Bostrom explores what utopia could look like if we reached a state of technological maturity:
- At technological maturity, our control over nature would be close to the maximum possible. We could even have “autopotency” — the ability to modify ourselves and our own internal states.
- There would still be limits to what is possible—both physical limits and limits arising from the nature of our values. Status, for example, would remain limited.
- Getting to utopia would require overcoming a host of coordination and governance problems (but that is not the focus of this book).
- If we reach technological maturity, what problems might be left? This raises questions about what could give our lives meaning and purpose if AI can do almost everything better than humans can:
- Most discussions just focus on shallow redundancy—what happens when humans lose their jobs to AI and become unemployed.
- Bostrom goes further and looks at deep redundancy—many of our leisure activities (e.g. shopping, exercising, learning) have instrumental value. Will those still hold appeal if AI can do everything better than we do?
- Bostrom explores various possible answers:
- Hedonism alone could be a sufficient answer. While most people find ideas like wireheading or drugs unsatisfying, at technological maturity our pleasures will not have to come with bitter aftertastes.
- We could also engage in autotelic activities, like games, art, and relationships, that are valued for their own sake. This may require cultivating a true leisure culture.
- We could also create artificial purposes (e.g. sports competitions) if we want.
- There are also some tasks cannot be outsourced, perhaps because it involves our connections to other people and requires some sort of sentiment.
- Many people worry that utopia might be “boring” or lacking in diversity.
- We want lives that are good, not stories that are dramatic.
- Diversity and interestingness depend on the scale. At the largest scale, any person’s ability to contribute interestingness seems extremely small, but at smaller scales, that doesn’t matter.
- Autopotency could increase our capacity to find things interesting or beautiful.
- Moreover, utopia doesn’t need to be static but can be ever-changing like a living kaleidoscope.
Detailed Summary of Deep Utopia
What kind of utopia?
Plastic utopia
People have conceived of many different types of utopias—e.g. governance and culture utopias, or post-scarcity or post-work utopias. Bostrom explores a more radical type of “plastic utopia” in which we could achieve any preferred local configuration effortlessly.
For example, in a room full of various atoms and some source of energy, we could express a wish that the atoms be arranged in different ways and the atoms would swiftly and automatically reorganised in this way. We would also have autopotency—the ability to modify ourselves (including our internal states) however we want.
Cautionary note on modifying human nature
With autopotency, we could get rid of some unpleasant feelings like pain if we wanted.
However, unpleasant feelings may often be useful, sometimes in quite subtle ways. For example, people who suffer from congenital analgesia are born without the ability to feel physical pain. But this is a dangerous condition because pain helps us learn not to do things like stick our hands in boiling water. So if we want to get rid of pain, we’ll need some other way to alert us to dangers.
Another risk is that changes to our emotional or volitional nature can easily become permanent. For example, if you changed yourself to want nothing but the maximum number of paperclips, you’d never be able to change yourself back to wanting anything else. So, before changing human nature, we should attain greater insight and wisdom than we currently have.
A plastic utopia doesn’t mean everyone has unlimited power over their local environment and themselves, because different agents may have conflicting preferences. Plasticity describes our power over nature, not over other agents.
Technological maturity
Bostrom defines technological maturity as:
A condition in which a set of capabilities exist that afford a level of control over nature that is close to the maximum that could be achieved in the fullness of time.
This represents a state where humanity has developed technology to nearly the theoretical limits of what’s physically possible. We’d be able to rearrange matter at will and extend our lifespans indefinitely.
But there are some limits even at technological maturity. Based on our current understanding of physics (which may change), there are limits for how far into the universe we could reach. General relativity suggests that even if the universe is infinite, the volume of the universe that we could reach is finite (and decreasing).
There are also limits arising the nature of our value. For example, positional goods like status are inherently scarce, no matter how much technological progress we make. Novelty is also inherently limited—you can only take your first steps once, and experience a first kiss once. Though the number of potential “firsts” may be infinite, the number of significant and attractive firsts may not be.
Moral constraints may also limit what utopias we could reach. For example, deontological moral prohibitions against genetic engineering on humans would limit what utopia could look like. But limits could be reached even under a consequentialist view—e.g. there might be some downside risks that prevent us from going down certain paths.
Getting to utopia
Reaching utopia isn’t just about technological progress—it also requires solving fundamental coordination and governance challenges.
Without progress in the way that our civilization governs itself, increases in our material powers could easily make things worse instead of better; and even if a utopian condition were attained, it would likely be unstable and short-lived unless, at a minimum, the most serious of our global coordination problems were also solved.
A critical challenge is escaping the Malthusian trap. The Malthusian trap arises because population can grow exponentially (k^n), but economic growth cannot. Even space colonisation can produce at best polynomial growth (n^k) assuming we are limited by the speed of light.
Humans temporarily escaped the Malthusian trap during the Industrial Revolution, when economic growth outpaced population growth. However, Bostrom believes we’ll return to a Malthusian condition in the long run because natural selection will favour higher fertility, unless we can coordinate to limit population growth. This is even clearer if you apply the Malthusian trap not just to humans but to all sentient beings. For example, if you want pigeons to have above-subsistence living standards, you must find some way to cap their fertility rates. Otherwise, pigeon populations will grow faster than whatever help you can give them.
To get to utopia, we don’t just need power over nature, but also cooperation with fellow beings and wisdom. Without cooperation and wisdom, technological advances could make us worse-off. Advances could enable mischief on a larger scale, and coordination problems can become harder to solve as capabilities increase.
We can liken some coordination problems to knots, and technological progress as being akin to pulling on a string. Tugging at the ends of the string tends to stretch it out and make it reach farther. And some knots may indeed be resolved in this manner … ; but there is no guarantee that this is true for all knots. Some knots may instead require dexterous statecraft or moral finesse to straighten out. If we’re unlucky, some of these maneuvers may need to take place before technological progress pulls the knots so tight that no fingers can untie them.
Bostrom largely sets aside questions of coordination and wisdom in this book.
Purpose and meaning in a post-work world
At the heart of Deep Utopia lies the philosophical question: what gives life purpose when AI can perform all tasks better than humans?
Whenever Bostrom gives talks about the future of AI, someone invariably asks about the “purpose problem.” Since we’ve evolved in an environment with constraints and scarcity, our psyches have evolved in ways that assume such scarcity. We may therefore feel a deep need for goal-directed striving, which can come in the form of purpose.
Similarly, meaning helps us sustain motivation over long, difficult periods where mere willpower is not enough. For example, think of a parent raising young children, an athlete undergoing intense training, or an actor getting rejected over and over again. It’s meaning, not willpower, that gets us through each day. As such, a meaning crisis is more likely to affect us those who are already well-off.
Shallow Redundancy: What would we do without work?
Shallow redundancy is the traditional concern about technological unemployment—what happens when automation eliminates jobs? How will people spend their time?
Some believe that people would just use their leisure to “brawl, steal overeat, drink and sleep late” —a rather deep pessimism about human nature. However, people are different and respond differently to wealth and leisure. It may be true that some people are higher in “boredom proneness” and will get bored if they don’t have to work. Yet, for many people, work is also boring.
We can already see hints of how different groups handle abundant leisure time. Aristocrats, bohemians, retirees, students, and monastics all structure their lives quite differently when freed from economic necessity. While none of these groups are representative samples of humanity, they show that even now there are rather different models of what voluntary unemployment can look like, so we shouldn’t anchor too heavily on any single one.
A post-work society would require cultivating a true leisure culture. Our current education system prepares children to become good workers through “disciplining, civilising, sorting and certifying (and a bit of teaching).” But in utopia, we could instead prepare children to become flourishing human beings, skilled in the art of enjoying life. We could get our sense of self-worth and prestige on factors other than economic contribution, and build institutions and infrastructure that supported a broader range of enjoyable activities.
Cultural differences
Different cultures have different attitudes towards life and work. Western cultural values around work, success and striving seem to be influenced significantly by the Protestant work ethic. In contrast, Eastern religions like Buddhism, Jainism and Taoism emphasise nonattachment or even the extinction of desire. Under these viewpoints, it may seem rather perverse that, upon reaching a state of having no/few unfulfilled desires, we then start engineering some artificial, arbitrary desires just so we can continue striving.
Hinduism presents a more complex picture. On one hand, it recommends a path of spiritual liberation through non-attachment like in the other Eastern traditions. But it also presents us with the idea of Lila, or “divine play”—that the gods make sport by voluntarily imposing on themselves limitations and constraints to engage in playful activity within the mortal realm.
Deep redundancy: When even leisure loses its point
Shallow redundancy is the traditional and relatively superficial version of the purpose problem—what will we do once jobs become obsolete? But Bostrom identifies a deeper problem. Many leisure activities we consider meaningful have instrumental value, in that they help us achieve some other goal. At technological maturity, these activities could become as obsolete as work itself, leading to the problem of Deep Redundancy.
Consider these examples:
- Shopping: Part of the joy comes from finding bargains or discovering treasures. But if AI knows our preferences better than we do, old-fashioned shopping would just involve going through extra hassle for a worse result.
- Exercise: If medical nanobots, gene therapy, or safe drugs could provide health benefits instantly, would anyone still lift weights or run on treadmills?
- Learning: A key motivation for learning is acquiring knowledge or skills we lack. If you knew you’d forget everything you read overnight due to amnesia, would you still be motivated to read? But at technological maturity, we might be able to incorporate new information directly into our brains (Bostrom admits this is somewhat speculative technology and likely to be extremely challenging). Would we still want to learn the slow, difficult way?
Would parenting become redundant?
Even something like parenting, which seems like an unassailable source of meaning, faces challenges:
- First, if lifespans were extended significantly, the fraction of your life devoted to raising young children would shrink.
- Second, parenting is really a bunch of different activities, most of which are unpleasant chores. If you removed all those chores, you’d get only a small fraction of “precious moments”. Even if you doubled or tripled the frequency of those precious moments, it seems unlikely they’d be enough to fill your day with purposeful activity.
- Third, part of the satisfaction of parenting comes from the feeling that you’re benefiting your child in some way by spending time with them. But technological maturity might involve robotic caregivers with superhuman parenting skills, which could even look and sound identical to biological parents. In that case, you’d be harming your child every time you indulged in some DIY parenting. Would you still find this a rich source of purpose?
This last point illustrates the depth of the problem. When we can no longer contribute positively—when our efforts make things worse rather than better—activities lose their purpose even if we enjoy them.
Possible sources of meaning and purpose in utopia
Bostrom explores several possible answers to the purpose problem.
Hedonism (pleasure)
The first and perhaps most fundamental answer is simple: pleasure.
This answer often meets resistance. When people imagine a hedonistic utopia, they picture “mere pleasure-blobs” or drug addicts—hardly an inspiring vision. But there are two problems with this thinking:
- First, we confuse what’s interesting to contemplate with what’s good to live. A life of pure contentment might sound boring to read about, but that doesn’t mean it would be bad to experience.
- Second, our concept of “pleasure” is often used narrowly to refer to pleasant bodily sensations or thrills, which often come with bitter aftertastes—hangovers, addiction, guilt, emptiness. But Bostrom uses a broader definition of “pleasure”.
Bostrom defines “pleasure” as “the subjective quality of positive hedonic tone”—which could range from simple contentment to profound bliss that “fills our spirit with a warm affirming joy”. Our experiences don’t have to be static, either—there are many different types of positive experiences we could enjoy. And with autopotency, the states of pleasure available might extend far beyond what our current brains can even imagine.
Each day could be arranged with artistic ingenuity and turn out as little masterpieces all in themselves, while adding to an ever-rising larger structure into which they all fit together perfectly each in its unique way: like carefully carved and coordinated stones that together compose a great cathedral of life.
Pleasure alone can go a long way if we think of it this way. Perhaps it can go even further if we combine it with other values, but it wouldn’t be crazy to think pleasure is the only thing that matters. Bostrom spends considerable time exploring other values, but only because they pose more complex theoretical challenges, and not because they’re more important.
Autotelic activities
Even if pleasure alone could suffice, utopia needn’t be passive. Humans could engage in autotelic activities—activities valuable for their own sake rather than for achieving some external goal.
The challenge is that genuinely autotelic activities are surprisingly rare. As noted above, many leisure pursuits are actually instrumental. But some activities seem to retain value even without external purpose. For example, art and creativity could remain meaningful if we value the process, not just the product. After all, amateur musicians find joy in playing even though professional recordings exist.
Artificial purposes
We already have many artificial purposes in life today. For example, a football player’s goal isn’t really to get the ball in the net, else they could just carry it there. Instead, they accept artificial rules that create challenge and engagement. At technological maturity, we could similarly create artificial constraints—e.g. achieving some goal without the use of some of your enhancements. These artificial purposes could remain useful ways of allocating esteem.
Some might object that artificial purposes are not meaningful enough because they don’t have real stakes. It’s true that winning a football game sounds less grand than fixing issues of material deprivation, but most people today don’t find purpose by working on those issues anyway. If anything, parochial things like sports tend to generate more passion and engagement than matters of life and death halfway around the world.
But artificial purposes lack significance!
Some might find artificial purposes unsatisfying because we ultimately want our lives to matter. But in utopia, anything that needs doing won’t need to be done by us.
However, if this is the metric, our current human lives appear quite limited. Many people make little or no net positive difference to the world, yet their well-being doesn’t seem to suffer as a result. Even if you apply a local scale, you can think of people who, through no fault of their own, cannot contribute to society, yet they could still have great lives.
Bostrom suggests that, if anything, perhaps we should hope to become less significant. Less significance means less responsibility, less opportunity to screw up. We may already be more significant than would be optimal given our current capabilities.
If I had to guess, I would say that the average adult maybe ought to be responsible for about one year of human life. After this period, things ought to be restored to some acceptable condition if they have messed up. Maybe the most mature and worldly-wise among us could be responsible for one decade of their own life. But to be responsible for an entire human life—and some would think without even the possibility of a do-over at the end—well, that is just too much.
Connections with other people
In utopia, some tasks that have to be done by a particular person may still exist. By their nature, these tasks cannot be outsourced to others, perhaps because it requires some sort of sentiment.
For example, some people may prefer some goods and services to be manually produced, particularly if the good/service is a gift intended to express an emotion. Other examples of activities that can’t be outsourced include honouring people or traditions, or following through on commitments.
These purposes are not artificial or fake, though they will depend on your values. They have some external grounding which, Bostrom argues, should make them fully “legitimate”.
Will utopia be boring or lacking in diversity?
Good lives vs. good stories
People often fail to distinguish between boredom and boringness. Boredom is a subjective, negative mental state. At technological maturity, we could definitely avoid boredom as we’d be able to trivially generate lots of alternative psychological states like pleasure, fascination, or joy. Boringness, on the other hand, is an objective attribution to some person or thing. It seems to rely on an absence of features like novelty, relevance, significance, and worthwhile challenge.
However, “interesting times” are often horrible for those living through them. It may be worth giving up some “good stories” in order to avoid serious harms like abuse, poverty, depression, even if that makes life less meaningful.
Remember that our task is not to create a future that is good for telling stories about, but one that is good to live in.
There is a tendency, especially among nerdy intellectual types, to place too much weight on “interestingness”. Many other plausible values may remain desirable even if repeated lots of times. The enjoyment you get from reading Shakespeare might diminish after several decades of study, but maybe not the enjoyment you get from a nice cup of tea.
Diversity depends on the scale
Interestingness depends on diversity, but this may differ depending on what scale you’re looking at. At the largest scale—such as when looking at the universe—any person’s ability to contribute interestingness seems extremely small. But at smaller scales, each person’s ability to contribute interestingness increases.
Example: Changing diversity at different scales
Imagine two cities, Solburg and Lunaburg. Everyone in Solburg is a morning person (lark). Everyone in Lunaburg is a night owl.
Now imagine the two cities become connected and people start mingling. After a few generations, the two cities each have roughly 50% larks and 50% owls.
- At the scale of an individual city, diversity increased. Before, each city had just one type of person; now each has two types.
- At the scale of the region, diversity decreased. Before, the region had two types of cities; now it has one.
- But at the even larger scale of the country, diversity could have increased again if all cities were either 100% larks or 100% owls before Solburg and Lunaburg were connected. Now, there is a third type of city: cities with a 50:50 mixture.
You can keep going through this process. Paradoxically, the internally least diverse city/country could contribute the most to world diversity by being such an outlier.
Autopotency could increase our capacity for interestingness
Interestingness is really in the eye of the beholder. If we want to create a world of beauty, we don’t need necessarily more art but a greater capacity for aesthetic appreciation.
Consider a man, Bob, examining a duck’s beak. The beak itself may not be objectively interesting. But if Bob is particularly sensitive and imaginative, he might conjure up a vast series of inner phenomena while looking at a beak. The “action” is really in the mind of the observer.
With enough such capacity, the rustling of a lizard or the yellow of a duck’s beak can go a long way.
Without such capacity, it doesn’t matter how large a collection of beautiful objects we accumulate. We would be like the guard dogs patrolling the Louvre.
Utopia as a living kaleidoscope
Bostrom thinks we could go considerably longer than our current lifespans without depleting our levels of contained interestingness. He admits that novelty could be harder to achieve with vastly longer lifespans.
However, this doesn’t mean life in utopia would be static. Instead of imagining utopia as a monotonous state, we could imagine it as a “living kaleidoscope” conjuring up ever-changing patterns. And the kaleidoscope would be far more intricate and complex than our current lives—perhaps by a margin similar to that by which the richness of our lives exceeds that of a literal kaleidoscope.
[S]uch life could be like a living kaleidoscope, conjuring an ever-changing series of patterns that transform and modulate one another according to a fixed set of rules within bounded parameters. There is a sameness at a certain level, but also inexhaustible richness and novelty at other levels.
How much do you value utopia?
Towards the end of the book, Bostrom offers up the following thought experiment:
How much would you gamble for utopia?
Suppose you were given the following choice:
- Live out your current life in the normal way and die at around 80 or 90.
- Take a gamble with an x% chance of having the best possible life for you starting from today, but a (100-x)% chance of dropping dead instantly.
How big would x need to be for you to prefer the gamble?
The “best possible life” might involve you eventually developing into a super-flourishing posthuman who lives for millions of years and attaining unimaginable levels of felicity. If you really thought the best utopian life you could have would be millions of times better than your current life, you should accept a 99.9999% chance of dying immediately for the sake of a 0.0001% chance of getting this best possible life.
My Review of Deep Utopia
Deep Utopia is, in the words of another reviewer, “deeply odd”. Bostrom seems well aware that his writing isn’t for everyone, and doesn’t shy away from jargon or unreasonably long sentences. He appears to be writing more for himself and perhaps some friends (there are quite a few references that look like in-jokes). The book is extremely thorough, as Bostrom explores questions about meaning, purpose, and interestingness with a patience that surely the 99 percent of the general population will not have. I found myself skimming through many sections, as I’m pretty sympathetic to the pure hedonist view—I agree strongly with Bostrom’s point that good lives are more important than good stories.
Despite some frustrations with Bostrom’s writing, I found Deep Utopia very thought-provoking. Bostrom also demonstrates a talent for clever analogies and turns of phrases—I enjoyed his metaphor about patrol dogs at the Louvre, as well as his thought experiment with ThermoRex the heater—and some of his ideas stayed with me long after I finished the book.
Reading this book has reaffirmed for me that I really don’t care for these speculative, sci-fi visions. I’m fundamentally a satisficer when it comes to utopian thinking. My desires are reasonably modest, and the idea of using technology to increase those desires holds very little appeal. (Perhaps that’s my inner Buddhist speaking.) On a societal level, I care more about reducing suffering than than maximising well-being. But we wouldn’t need anything close to technological maturity to achieve that—we’d just need to solve all those pesky social and political problems that Bostrom doesn’t discuss.
Let me know what you think of my summary of Deep Utopia in the comments below!
Buy Deep Utopia at: Amazon <– This is an affiliate link, which means I may earn a small commission if you make a purchase through it. Thanks for supporting the site! 🙂
If you enjoyed this summary of Deep Utopia, you may also like:
2 thoughts on “Book Summary: Deep Utopia by Nick Bostrom”
Have we not already come across the problems that Bostrom mentions? Leisure time and leisure activity has certainly increased compared to prior generations. Like on the topic of deep redundancy, surely that has already happened, right? Having a strong memory, knitting, navigation… those have been lost to time in terms of their everyday utility, even if some people still keep them up as enjoyable hobbies.
Yes, Bostrom did point out that there are already groups in society with a lot of leisure time (e.g. retirees, students, monastics) but selection bias is rife. He also pointed out that most people’s lives are already not that meaningful on a large scale. I guess a difference is that, if you’re the kind of person who really wants to make a positive difference in the world, there are still many ways for you to do so now. At technological maturity, there might not be any ways to make a positive difference – perhaps not even parenting your own child! – and that could be quite challenging for some.
I disagree that the utility of having a strong memory has disappeared. I also don’t know anyone who practices their memory for fun – I know some people compete in memory competitions but they seem pretty rare. As for navigation – I suspect a lot of people do it because they still think it’s a useful skill – e.g. if you get lost in the woods without a phone. Having a strong memory and navigation skills still seem to be things that people practice because they think it strengthens their brain, which might be useful in some other way (e.g. work). Knitting is a good example of what I think is probably a true autotelic activity. I think Bostrom is right that these are harder to find than you might think, though.