Opinion

The Saturation View: some responses

​A couple of weeks ago, I published a draft of a new population axiology that I’ve been working on with Christian Tarsney. It got a lot of comments and pushback — thanks to everyone who engaged! They’ll feed into the more-polished academic-draft paper that Christian and I are working on.Here I’ll quickly respond to some of the most common or noteworthy responses. I’ll generally avoid stuff that is already covered in the draft.What’s the view? Isn’t this old hat?Very roughly, the Saturation view says that the value of a life, experience, or welfare-event depends not only on how high-welfare it is, but also on how many relevantly similar lives, experiences, or welfare-events already exist. The addition of near-duplicates has diminishing marginal impersonal value, tending toward a bound. The value of a world is a function of the total welfare in that world, and how widely distributed that welfare is across different types of lives / experiences / welfare-events. (Full draft here.)Some people suggested this is old news. And it’s totally correct that the suggestion that variety of experiences matters is not new (e.g. Yudkowsky here). The phrase “tiling the universe with hedonium” is normally used pejoratively for a reason. In fact, the idea that there’s value in variety has got a long pedigree, especially motivated by the puzzle of why God would create so many different types of being. Plato discussed the idea, as did (in different ways) Plotinus, Aquinas, Leibniz and Lovejoy.What’s new is (i) developing this thought into a specific formal population axiology (where there are many ways one could go with the basic idea; a previous version of the paper suggested “similarity-based discounting” but it had some problems I thought were serious); (ii) showing that the same basic machinery that helps with the monoculture problem can also help with the repugnant conclusion, fanaticism, and some forms of infinitarian incomparability and paralysis.Isn’t this just Scott Alexander’s UNSONG theodicy?I haven’t read UNSONG, but I have read “Answer to Job,” which I think is the same idea. At any rate, I’ll respond to the “Answer to Job”. In it, God says:IF YOU HAVE A COMPUTATIONAL THEORY OF IDENTITY, THEN TWO PEOPLE WHOSE EXPERIENCE IS ONE HUNDRED PERCENT SATURATED BY BLISS ARE JUST ONE PERSON.This idea is explored (and argued against) in Bostrom’s 2006 paper “Quantity of experience: brain-duplication and degrees of consciousness”.That is, “Answer to Job” is a particular metaphysical understanding of the numerical identity of experience (what Bostrom calls “Unification”), combined with something like the total view of population ethics. When looking to an ideal future, some of the implications of the idea are:You start off producing the experience that has the most wellbeing per unit of resources.If you create a qualitative copy of that experience, you add 0 additional value.If you create an ever-so-slightly-modified version of that experience (e.g. the colour of an object in the background is a different shade), then you add the total value of the wellbeing of that experience (i.e. essentially the same value as the first life).(Then, a difference in God’s case: he’s also making choices with zero opportunity cost. It’s a choice between net-positive Job or nothing. Whereas we’ll be choosing between a new net-positive-but-novel life or even-more-positive-and-non-novel life).This is different from the Saturation view, which is about the ethics of creating new lives. It’s not grounded in any particular metaphysics of identity of experiences. And, in contrast, its implications are:You start off producing the experience / life that has the most wellbeing per unit of resources.If you create a qualitative copy of that experience / life, you add almost as much additional value as the first life.If you create an ever-so-slightly-modified version of that experience /  life (i.e. the colour of an object in the background is a different shade), then, again, you add almost as much additional value as the first experience / life.So, the Saturation view doesn’t regard qualitatively identical copies as worthless. Two bliss-clones is a lot better than one! (Which seems right to me!) And it regards almost-indistinguishable lives as having diminishing marginal value in just the same way that actually-indistinguishable lives do. (Which also seems right to me.)I spent a bit of time when drafting trying to ground the Saturation view in “Unification”, and it’s one way to go. But its view of a best-possible future is one of a huge number of almost-identical lives, just with tiny unimportant aspects of experience (or other computation) varied. Which isn’t the anti-monoculture intuition that motivated the whole project. What if you just don’t share the starting intuition?A bunch of people just think Homogeneity really is the best possible future, and don’t get the intuition that it’s not. Totally fair! If so, the view probably won’t move you very much. But at least consider whether you’d want variety within a life. Imagine we’re post-singularity, and you get to choose the life you’ll live. You can have an extremely long life containing a wide range of wonderful experiences. Or you can identify the single best moment — the single best experience you could possibly have — and repeat it again and again and again.I think a lot of people, at least, would not choose the repetition (even putting boredom to the side).But a core utilitarian idea is that the boundaries between persons are not morally fundamental in the way common-sense morality often assumes. We should make the same choices whether we’re choosing “within a life” and “across lives”.  What seem like deep distinctions between persons are not load-bearing from the impartial point of view. I wouldn’t want to lean too heavily on the metaphysics here, but there is a poetic way of putting the thought: perhaps, morally speaking, we are all living one life. If so, then just as I would not want my own life to be one bliss-moment repeated forever, I should not want the universe to be one bliss-moment repeated forever, either.Second: think at least a bit about the problems for the Total View:  the Repugnant Conclusion and Very Repugnant Conclusion;fanaticism (when combined with plausible decision-theoretic principles);infinitarian paralysis and massive infinitarian indeterminacy;I think the Total View has a lot going for it, and I’m not claiming that the Saturation View is better. What I am saying is: if you don’t like those implications, here is another way to go — a way that addresses most of the biggest problems for the Total View in a fairly unified way.ParetoNow, Elliott Thornley’s objection: to move from Homogeneity to Variety, you have to make some particular person’s life worse, without making anyone else’s life better. You can say that the universe is now more varied, but no one benefits from that variety. You’ve violated “Pareto”. Two responses.First, once you accept the Value of Variety, you have already accepted a non-Paretian-looking thought. The claim that Variety is better than Homogeneity is not a claim that each person in Variety is better off. It is a claim about the pattern of value in the world. If that kind of claim is ruled out from the start, then the Saturation View is ruled out from the start. But so are many other pluralist views: strict Pareto also rules out the idea that there’s any meaningful non-welfarist value (e.g. of the natural environment, or art, or knowledge).Second, again think about within-life choices. Suppose I choose a life with many different wonderful experiences rather than a life consisting only of my single best moment repeated. In doing that, I am in some sense “making some temporal part of me worse off.” The me-at-time-t who gets the less-than-maximal experience might complain: why didn’t you give me the best moment instead? But that complaint misunderstands the value of the whole life. The point is not that every temporal part gets the best possible moment. The point is that my life as a whole is richer and better.Again, if we take seriously the thought that there’s no ethically meaningful distinction between persons and really We Are All One, then the Pareto objection loses its force. Here’s a high-level framing: Utilitarianism can be grounded in the dissolution of the idea of there being morally meaningful distinctions between persons. One way to go from there is what John Broome calls a “disuniting” approach: the value of every experience-moment is separable from (i.e. additively independent of) every other. That leads you, most naturally, to the Total View and its attendant issues. Alternatively, you could take a “uniting” approach: it’s like all beings are part of one collective person. If so, then what has happened in “other people’s” lives can absolutely be relevant for the value of my life, just as what has already happened at earlier moments in my life can be relevant to the value of future moments in my life.Variety of torturesSome people raised the idea that it’s implausible that a variety of tortures is worse than the same torture repeated, assuming the total amount of negative welfare is the same.My own intuitions are mixed on this. I weakly get the intuition that the diversity of tortures is worse than the same torture repeated. As I write in the draft, I think it’s an open question what the best way is to incorporate the negative domain into the Saturation view. But the Saturation View doesn’t force us to say that “Diverse Hell” is worse than “Uniform Hell.” In particular, you could adopt the variant where there are many “types” of positive experience, but only one “type” of negative experiences. (Where a “type” here is just about what sorts of experiences are meaningfully morally distinguishable; it’s not making any deep metaphysical claim. Equivalently, you could state the view as saying that every “type” of bad has a uniform and universal “footprint” over type-space.) If so, then the view wouldn’t regard a diversity of negative experiences as being worse than the same quantity of uniform experience.InequalityVitalik raised a worry about inequality: The bullet this bites is inegalitarianism, correct?Like, if you have a perfectly equal world with a billion people living medium-good lives, and you get an extra unit of resources, the Saturation View would imply it’s better not to spread those resources out equally, but to direct them all to one or a few people, because the “people living medium-good lives” archetype is saturated, but the “wealthy tycoon” archetype is completely absent.This can happen, for sure. But, on the version of the view I propose, welfare is not a part of type. So it’s not merely that by taking someone’s life and changing their wellbeing you increase variety (and can thereby make things better); you also have to be changing the nature of their lives in some meaningful way, to increase variety somehow.And holding variety fixed, inequality of wellbeing is never good. It’s either neutral or bad, depending on how we want to define the saturation function (using the equivalent of utilitarian or prioritarian aggregation of welfare.) On the prioritarian version, holding variety fixed, you always prefer a more-equal distribution. It’s possible for an unequal distribution of wellbeing to be better than an equal one. But only because that unequal one is more diverse, and the increase in diversity outweighs the increase in inequality. This is to be expected if you think there are many values at stake: as an analogy, if you think that art has meaningful intrinsic value, then it’s to be expected that you would think that an unequal world with great art can be better than an equal world with no art, keeping total wellbeing the same. Ditto with the value of variety. The Saturation View and Better Futures Finally, I’ll mention that the view has a couple of upshots for Better Futures. First, the fragility of value / the narrowness of eutopia as a target. In Value is Fragile, Yudkowsky points to diversity as an argument for the fragility of value:Consider the incredibly important human value of “boredom” – our desire not to do “the same thing” over and over and over again.  You can imagine a mind that contained almost the whole specification of human value, almost all the morals and metamorals, but left out just this one thing — and so it spent until the end of time, and until the farthest reaches of its light cone, replaying a single highly optimized experience, over and over and over again.Katja Grace responds that it doesn’t seem that hard to specify something that’s not a lot better than this.But, if the Saturation view is right (or approximately right), then eutopia really is a very narrow target. Even if we have a future that contains only lives of utter joy, and even if those lives are highly diverse, nonetheless we could still only achieve a small fraction of possible value, if the diversity is contained to only a minority of type-space. (E.g. our future contains a wide diversity of human-inspired and human-originating lives, but no inhuman, AI or alien-ish forms of life.)Second, diversity could be an example of a moral public good.  Imagine many future beings all trying to make their own lives as good as possible. Perhaps they converge on the same best life, or the same small set of best experiences. From the point of view of each individual, it might make sense to pursue that life. But collectively, it could be a huge loss of value, because the resulting world ends up too homogeneous.If those beings value variety, they might instead all prefer a coordinated arrangement where all those different groups live very different lives, explore different parts of the landscape, and realise different kinds of value. Variety would be a moral public good.Discuss ​Read More

​A couple of weeks ago, I published a draft of a new population axiology that I’ve been working on with Christian Tarsney. It got a lot of comments and pushback — thanks to everyone who engaged! They’ll feed into the more-polished academic-draft paper that Christian and I are working on.Here I’ll quickly respond to some of the most common or noteworthy responses. I’ll generally avoid stuff that is already covered in the draft.What’s the view? Isn’t this old hat?Very roughly, the Saturation view says that the value of a life, experience, or welfare-event depends not only on how high-welfare it is, but also on how many relevantly similar lives, experiences, or welfare-events already exist. The addition of near-duplicates has diminishing marginal impersonal value, tending toward a bound. The value of a world is a function of the total welfare in that world, and how widely distributed that welfare is across different types of lives / experiences / welfare-events. (Full draft here.)Some people suggested this is old news. And it’s totally correct that the suggestion that variety of experiences matters is not new (e.g. Yudkowsky here). The phrase “tiling the universe with hedonium” is normally used pejoratively for a reason. In fact, the idea that there’s value in variety has got a long pedigree, especially motivated by the puzzle of why God would create so many different types of being. Plato discussed the idea, as did (in different ways) Plotinus, Aquinas, Leibniz and Lovejoy.What’s new is (i) developing this thought into a specific formal population axiology (where there are many ways one could go with the basic idea; a previous version of the paper suggested “similarity-based discounting” but it had some problems I thought were serious); (ii) showing that the same basic machinery that helps with the monoculture problem can also help with the repugnant conclusion, fanaticism, and some forms of infinitarian incomparability and paralysis.Isn’t this just Scott Alexander’s UNSONG theodicy?I haven’t read UNSONG, but I have read “Answer to Job,” which I think is the same idea. At any rate, I’ll respond to the “Answer to Job”. In it, God says:IF YOU HAVE A COMPUTATIONAL THEORY OF IDENTITY, THEN TWO PEOPLE WHOSE EXPERIENCE IS ONE HUNDRED PERCENT SATURATED BY BLISS ARE JUST ONE PERSON.This idea is explored (and argued against) in Bostrom’s 2006 paper “Quantity of experience: brain-duplication and degrees of consciousness”.That is, “Answer to Job” is a particular metaphysical understanding of the numerical identity of experience (what Bostrom calls “Unification”), combined with something like the total view of population ethics. When looking to an ideal future, some of the implications of the idea are:You start off producing the experience that has the most wellbeing per unit of resources.If you create a qualitative copy of that experience, you add 0 additional value.If you create an ever-so-slightly-modified version of that experience (e.g. the colour of an object in the background is a different shade), then you add the total value of the wellbeing of that experience (i.e. essentially the same value as the first life).(Then, a difference in God’s case: he’s also making choices with zero opportunity cost. It’s a choice between net-positive Job or nothing. Whereas we’ll be choosing between a new net-positive-but-novel life or even-more-positive-and-non-novel life).This is different from the Saturation view, which is about the ethics of creating new lives. It’s not grounded in any particular metaphysics of identity of experiences. And, in contrast, its implications are:You start off producing the experience / life that has the most wellbeing per unit of resources.If you create a qualitative copy of that experience / life, you add almost as much additional value as the first life.If you create an ever-so-slightly-modified version of that experience /  life (i.e. the colour of an object in the background is a different shade), then, again, you add almost as much additional value as the first experience / life.So, the Saturation view doesn’t regard qualitatively identical copies as worthless. Two bliss-clones is a lot better than one! (Which seems right to me!) And it regards almost-indistinguishable lives as having diminishing marginal value in just the same way that actually-indistinguishable lives do. (Which also seems right to me.)I spent a bit of time when drafting trying to ground the Saturation view in “Unification”, and it’s one way to go. But its view of a best-possible future is one of a huge number of almost-identical lives, just with tiny unimportant aspects of experience (or other computation) varied. Which isn’t the anti-monoculture intuition that motivated the whole project. What if you just don’t share the starting intuition?A bunch of people just think Homogeneity really is the best possible future, and don’t get the intuition that it’s not. Totally fair! If so, the view probably won’t move you very much. But at least consider whether you’d want variety within a life. Imagine we’re post-singularity, and you get to choose the life you’ll live. You can have an extremely long life containing a wide range of wonderful experiences. Or you can identify the single best moment — the single best experience you could possibly have — and repeat it again and again and again.I think a lot of people, at least, would not choose the repetition (even putting boredom to the side).But a core utilitarian idea is that the boundaries between persons are not morally fundamental in the way common-sense morality often assumes. We should make the same choices whether we’re choosing “within a life” and “across lives”.  What seem like deep distinctions between persons are not load-bearing from the impartial point of view. I wouldn’t want to lean too heavily on the metaphysics here, but there is a poetic way of putting the thought: perhaps, morally speaking, we are all living one life. If so, then just as I would not want my own life to be one bliss-moment repeated forever, I should not want the universe to be one bliss-moment repeated forever, either.Second: think at least a bit about the problems for the Total View:  the Repugnant Conclusion and Very Repugnant Conclusion;fanaticism (when combined with plausible decision-theoretic principles);infinitarian paralysis and massive infinitarian indeterminacy;I think the Total View has a lot going for it, and I’m not claiming that the Saturation View is better. What I am saying is: if you don’t like those implications, here is another way to go — a way that addresses most of the biggest problems for the Total View in a fairly unified way.ParetoNow, Elliott Thornley’s objection: to move from Homogeneity to Variety, you have to make some particular person’s life worse, without making anyone else’s life better. You can say that the universe is now more varied, but no one benefits from that variety. You’ve violated “Pareto”. Two responses.First, once you accept the Value of Variety, you have already accepted a non-Paretian-looking thought. The claim that Variety is better than Homogeneity is not a claim that each person in Variety is better off. It is a claim about the pattern of value in the world. If that kind of claim is ruled out from the start, then the Saturation View is ruled out from the start. But so are many other pluralist views: strict Pareto also rules out the idea that there’s any meaningful non-welfarist value (e.g. of the natural environment, or art, or knowledge).Second, again think about within-life choices. Suppose I choose a life with many different wonderful experiences rather than a life consisting only of my single best moment repeated. In doing that, I am in some sense “making some temporal part of me worse off.” The me-at-time-t who gets the less-than-maximal experience might complain: why didn’t you give me the best moment instead? But that complaint misunderstands the value of the whole life. The point is not that every temporal part gets the best possible moment. The point is that my life as a whole is richer and better.Again, if we take seriously the thought that there’s no ethically meaningful distinction between persons and really We Are All One, then the Pareto objection loses its force. Here’s a high-level framing: Utilitarianism can be grounded in the dissolution of the idea of there being morally meaningful distinctions between persons. One way to go from there is what John Broome calls a “disuniting” approach: the value of every experience-moment is separable from (i.e. additively independent of) every other. That leads you, most naturally, to the Total View and its attendant issues. Alternatively, you could take a “uniting” approach: it’s like all beings are part of one collective person. If so, then what has happened in “other people’s” lives can absolutely be relevant for the value of my life, just as what has already happened at earlier moments in my life can be relevant to the value of future moments in my life.Variety of torturesSome people raised the idea that it’s implausible that a variety of tortures is worse than the same torture repeated, assuming the total amount of negative welfare is the same.My own intuitions are mixed on this. I weakly get the intuition that the diversity of tortures is worse than the same torture repeated. As I write in the draft, I think it’s an open question what the best way is to incorporate the negative domain into the Saturation view. But the Saturation View doesn’t force us to say that “Diverse Hell” is worse than “Uniform Hell.” In particular, you could adopt the variant where there are many “types” of positive experience, but only one “type” of negative experiences. (Where a “type” here is just about what sorts of experiences are meaningfully morally distinguishable; it’s not making any deep metaphysical claim. Equivalently, you could state the view as saying that every “type” of bad has a uniform and universal “footprint” over type-space.) If so, then the view wouldn’t regard a diversity of negative experiences as being worse than the same quantity of uniform experience.InequalityVitalik raised a worry about inequality: The bullet this bites is inegalitarianism, correct?Like, if you have a perfectly equal world with a billion people living medium-good lives, and you get an extra unit of resources, the Saturation View would imply it’s better not to spread those resources out equally, but to direct them all to one or a few people, because the “people living medium-good lives” archetype is saturated, but the “wealthy tycoon” archetype is completely absent.This can happen, for sure. But, on the version of the view I propose, welfare is not a part of type. So it’s not merely that by taking someone’s life and changing their wellbeing you increase variety (and can thereby make things better); you also have to be changing the nature of their lives in some meaningful way, to increase variety somehow.And holding variety fixed, inequality of wellbeing is never good. It’s either neutral or bad, depending on how we want to define the saturation function (using the equivalent of utilitarian or prioritarian aggregation of welfare.) On the prioritarian version, holding variety fixed, you always prefer a more-equal distribution. It’s possible for an unequal distribution of wellbeing to be better than an equal one. But only because that unequal one is more diverse, and the increase in diversity outweighs the increase in inequality. This is to be expected if you think there are many values at stake: as an analogy, if you think that art has meaningful intrinsic value, then it’s to be expected that you would think that an unequal world with great art can be better than an equal world with no art, keeping total wellbeing the same. Ditto with the value of variety. The Saturation View and Better Futures Finally, I’ll mention that the view has a couple of upshots for Better Futures. First, the fragility of value / the narrowness of eutopia as a target. In Value is Fragile, Yudkowsky points to diversity as an argument for the fragility of value:Consider the incredibly important human value of “boredom” – our desire not to do “the same thing” over and over and over again.  You can imagine a mind that contained almost the whole specification of human value, almost all the morals and metamorals, but left out just this one thing — and so it spent until the end of time, and until the farthest reaches of its light cone, replaying a single highly optimized experience, over and over and over again.Katja Grace responds that it doesn’t seem that hard to specify something that’s not a lot better than this.But, if the Saturation view is right (or approximately right), then eutopia really is a very narrow target. Even if we have a future that contains only lives of utter joy, and even if those lives are highly diverse, nonetheless we could still only achieve a small fraction of possible value, if the diversity is contained to only a minority of type-space. (E.g. our future contains a wide diversity of human-inspired and human-originating lives, but no inhuman, AI or alien-ish forms of life.)Second, diversity could be an example of a moral public good.  Imagine many future beings all trying to make their own lives as good as possible. Perhaps they converge on the same best life, or the same small set of best experiences. From the point of view of each individual, it might make sense to pursue that life. But collectively, it could be a huge loss of value, because the resulting world ends up too homogeneous.If those beings value variety, they might instead all prefer a coordinated arrangement where all those different groups live very different lives, explore different parts of the landscape, and realise different kinds of value. Variety would be a moral public good.Discuss ​Read More

Related Posts

Leave a Reply

Your email address will not be published. Required fields are marked *