One more attempt: what if collective decision-making is really a learning problem?
I have noticed that I have become unusually invested in this problem.
Partly because I believe I see a solution, and partly because I seem to keep failing to articulate what I see in a way that makes it possible for other people to see it too.
So I want to try one more time.
This time I will start from my own profession.
I work as an infectious disease modeller and epidemiologist in a national public health institute. Ultimately, my job is about reducing disease burden in populations.
Some of the interventions we work with are remarkably effective. Measles vaccination is one example. Measles is also one of the diseases for which global eradication is conceivable.
And eradication creates a peculiar decision problem.
If we eradicate measles, we don't merely reduce disease burden for the people alive today. We change the world inherited by every future generation. Eventually, vaccination against measles would no longer be necessary either.
The important point for my argument is not whether this future benefit can be assigned some particular expected value. It is that eradication changes the future decision problem itself.
To eradicate measles, we need a long sequence of connected decisions.
We need surveillance. We need models. We need vaccination. We need to observe what happens. We need to discover where our assumptions were wrong. We need to change the strategy. We need to coordinate with other countries. We need to learn from failures elsewhere.
The decision is therefore not really:
A or B?
It is a continuous process:
What do we currently believe? What should we do given that belief? What happened? What did we get wrong? What does this tell us about the world? What should we believe now? What should we do next?
This brings me back to the cluelessness problem.
We live in an evolving universe. Our models are necessarily incomplete, and the future is not simply unknown in the sense of being a hidden answer waiting to be calculated. The world itself changes.
So perhaps the central problem of decision-making is not how to make the correct decision from a fixed model.
Perhaps it is how to remain capable of correcting the model on which the next decision will depend.
And this introduces another layer.
We can learn about the problem.
But we can also learn about how we learn about the problem.
For example, better surveillance may produce better information. Better information may improve our model. A better model may lead to better decisions. The results of those decisions provide new information, which can improve the surveillance and modelling process again.
So there is a recursive possibility:
we don't only optimise the solution; we improve the process by which we discover what the solution should be.
That is the point I was trying to get at in my previous post.
But there is a second part of the problem that I had not made sufficiently explicit.
Measles eradication is not an individual decision problem.
No individual can eradicate measles.
No single institution can eradicate measles.
It requires collective action.
And now the problem becomes considerably harder.
We need a collective that can continuously adapt its understanding of a changing world.
But nobody possesses the complete model.
Different people see different things. Different institutions have different information. Different models will be wrong in different ways.
So perhaps the objective should not be to make everybody hold the same model.
Perhaps the objective is to maintain a network in which different imperfect models can continuously correct one another.
This changes what I think “alignment” means.
We cannot necessarily align people on every proposition about the world.
We cannot even guarantee that we will initially agree on the correct solution.
But perhaps we can align on the process by which we discover that we are wrong.
This is where I think something interesting happens.
The things required for this process sound, at first, like ordinary virtues:
Be honest.
Listen.
Be curious.
Say when you are uncertain.
Correct others when you believe they are wrong.
Allow yourself to be corrected.
Distinguish what you observed from what you inferred.
Help others understand information they do not have.
Forgive mistakes sufficiently that people remain willing to participate.
Remain connected even when you disagree.
But viewed from the perspective of a distributed learning system, these are not merely virtues.
They are functional properties of the system.
Honesty protects the fidelity of the information entering the network.
Listening determines whether information actually reaches another person's model.
Correction provides an error signal.
Curiosity drives exploration of uncertainty.
Forgiveness helps preserve the connections through which future information can travel.
And willingness to be corrected keeps the individual model open to updating.
The interesting thing is that this can become self-reinforcing.
Better relationships allow better information exchange.
Better information exchange allows better collective learning.
Better collective learning can increase trust in the relationships that made it possible.
And the reverse is also possible:
Fear produces concealment.
Concealment produces poorer information.
Poorer information produces worse models.
Worse models produce worse decisions.
Worse decisions produce distrust.
And distrust produces more fear and concealment.
So perhaps collective intelligence is not simply a property of how much information a group possesses.
Perhaps it is partly a property of whether the relationships between its members preserve the capacity for correction.
This also changes how I think about decentralisation.
If we don't know beforehand what the correct model of the future problem will be, we cannot simply distribute a correct solution from the centre.
Instead, we may need to distribute the capacity to learn.
Not agreement on the answer.
Agreement on the practices that allow answers to be challenged, revised and improved.
This is why I have started thinking about the “verbs of learning”.
Not what everyone should believe.
What everyone should be capable of doing.
Listening.
Questioning.
Explaining.
Correcting.
Updating.
Admitting uncertainty.
Seeking disconfirming information.
Supporting someone else's learning.
Being willing to change one's mind.
These behaviours do not automatically produce agreement. Nor do they guarantee that a collective will choose the right goal.
But they may create something more fundamental:
a collective that remains capable of discovering that it is wrong.
And perhaps that is the deeper problem I have been trying to describe.
Decision theory asks:
What should we do?
But under radical uncertainty, and especially when the decision is collective, perhaps we also need to ask:
How do we remain capable of discovering that what we are doing is wrong?
And if that capacity itself can be improved, then we arrive at a recursive problem:
We learn about the world.
We learn how to learn about the world.
And we learn together how to become better at learning together.
I suspect that this is not a new idea in the literature. Many pieces of it almost certainly exist in different disciplines.
But I increasingly suspect that putting these pieces together points toward something important:
The fundamental requirement for collective intelligence may not be agreement. It may be collective corrigibility.
Not a collective that is permanently right.
A collective that remains capable of becoming less wrong.
My original argument was aimed at P1. I argued that there is a class of decisions—decisions about how a model learns and changes—that cannot straightforwardly be reduced to comparisons of downstream consequences within that model.
The discussion has made me realise that this points to a more constructive response to DiGiovanni's challenge.
Suppose the cluelessness argument is correct. What should an impartial altruist do?
I think the answer is: we should shift part of our attention from predicting the future to preserving our capacity to correct our predictions.
This is not a rejection of expected value. EV remains useful whenever our model is sufficiently informative to support a comparison. The problem is what to do when we cannot know whether it is.
A finite intelligence faces a peculiar problem:
It can discover errors in its model, but it cannot guarantee that its mechanisms for detecting errors will detect all of its errors.
This creates a distinction between two forms of rationality.
Predictive rationality: Given my model, which action has the best expected consequences?
Corrective rationality: Given that my model may be wrong, what keeps me capable of discovering and correcting that wrongness?
The second is not simply epistemic humility. It concerns the architecture of the decision-making process itself.
And this matters because all models are necessarily limited. The relevant question is therefore not whether we can construct a model that is complete, but whether we can maintain an intelligence that remains responsive to what its model leaves out.
Consider two strategies. One attempts to maximise the apparent value of actions under a particular model. The other preserves the conditions through which that model can be challenged: listening to independent perspectives, genuinely exchanging information, allowing disagreement to remain visible, acknowledging errors, and changing one's representation when it no longer accommodates what one encounters.
The second strategy does not necessarily have higher expected value. That is precisely the point. If we cannot reliably compare long-run consequences, we need to consider properties of the intelligence doing the comparing.
This suggests a progression:
Prediction → model uncertainty → correction → meta-correction.
But correction introduces a further problem.
An intelligence can fail not only because its model is wrong, but because the way it responds to being wrong is itself inadequate. It may hear another perspective without allowing that perspective to affect its model. It may encounter disagreement without investigating it. It may acknowledge an error without examining what made the error possible. It may correct a conclusion while preserving the assumptions that generated the error.
So the problem is no longer simply:
“Is my model wrong?”
It becomes:
“Is the way I respond to being wrong itself capable of correction?”
This also gives a more precise answer to the question raised in the comments: what makes a model “open to correction” if all models are flawed?
Not the expectation that correction will make it correct.
Rather:
A model is corrigible when it preserves channels through which what it encounters can change how it represents reality.
And because those channels can themselves fail, we need to examine and change the way we listen, interpret, disagree, update, and correct.
This opens a new domain of rationality: the practice of meta-thinking. Under cluelessness, the impartial altruist should ask not only whether its model is wrong, but what it must do to remain capable of discovering that it is wrong: Whose perspective must it genuinely hear? What information must it allow to challenge its current representation? When confronted with disagreement, must it defend its model, or make the disagreement itself an object of inquiry? When an error is exposed, must it merely correct the conclusion, or examine why the error was possible in the first place? What must it be willing to change, and what must it keep open, for correction to remain possible?
These are not merely questions about acquiring better information about the future. They concern the present conditions under which an impartial altruist can justifiably move from an “is” to an “ought.” The paradox is that the actions required to maintain those conditions are themselves actions for which we need an “ought.”
The impartial altruist therefore has to reason reflexively:
To determine what it ought to do, it must also determine what it ought to preserve about the conditions that make determining what it ought to do possible.
This is what I mean by reflexive intelligence: intelligence begins to reason about the conditions of its own ability to reason.
And I think this gives us a constructive response to cluelessness that is neither a rejection of consequentialism nor an attempt to restore confidence in prediction.
Under genuine cluelessness, rationality does not end. It becomes reflexive.
I think I need to correct you, because I believe you have misunderstood the role of the counterexample.
I am not claiming that adding more information to a model necessarily produces better decisions. All models are necessarily flawed. The important point is that they are not necessarily flawed in the same domains.
A model can therefore improve by interacting with other models and treating genuinely different perspectives as information about its own blind spots. That interaction is what I mean by learning.
And I think this is the step that is missing from P1.
P1 describes the choice between A and B given a model of the world. My counterexample introduces a prior class of decisions: decisions about how the model itself learns and changes.
That learning step does not straightforwardly have an EV.
Consider the classic example of the six blind men and the elephant. Each person encounters a different part of the elephant and consequently constructs a different model: one thinks it is a snake, another a wall, another a tree, and so on.
None of the individual models is simply “the correct model”.
But the six agents can exchange information. They can recognise that their observations conflict. They can ask questions, compare perspectives, change their interpretations, and revise their models.
Through this interaction, the group can construct a representation that is closer to the elephant than any individual model.
Those decisions are not choices between A and B within a fixed model. They are decisions about how the models interact so that their blind spots can become visible to one another.
That is the step I am introducing into the premises.
And I think this is why I call the resulting principles ontological oughts. They are not rules about which outcome is better. They are rules concerning the conditions under which a finite model can remain aligned with a reality it can never completely represent.
For example, suppose I am modelling a situation and decide not to include the perspective of person Z.
That exclusion does not necessarily have an identifiable EV. I may not even be able to calculate what information I have excluded, because I have excluded it from the model through which I am doing the calculation.
But it can nevertheless matter structurally. My model may become increasingly misaligned with Z's model, while Z's model may simultaneously become increasingly misaligned with mine. We lose the possibility of correcting each other's blind spots.
The important point is therefore not that including Z necessarily produces better outcomes.
It is that excluding a potentially informative model removes a possible mechanism through which the limitations of my own model can be exposed.
This is where I think the issue goes deeper than the fact that the universe is messy or changing.
The fundamental problem is that a model is, by definition, not reality. It is a compression or representation of reality.
A model can test aspects of its own internal consistency. But it cannot, from within itself, establish that its own criteria for evaluating that consistency are sufficient to detect every way in which the model might be wrong.
In other words, the model has a blind spot concerning the adequacy of the mechanism by which it checks itself.
To resolve that blind spot, it requires another model.
But the other model has its own blind spots.
So we get:
M1↔M2
where each model can provide information about what the other cannot see from within itself.
This makes the models epistemically interdependent.
And this is the sense in which I think the learning step is prior to the EV step.
The sequence is not merely:
MODEL→A vs. B→ACTION.
There is a prior sequence:
MODEL→INTERACTION→CORRECTION→UPDATED MODEL→A vs. B.
The principles governing that interaction are therefore not themselves simply another instance of the A-vs-B problem.
They determine whether the model can continue to learn from reality at all.
I am not claiming that collaboration guarantees truth, or that every perspective should be accepted, or that more information necessarily produces better decisions.
I am claiming something more limited:
A finite model cannot fully identify its own blind spots from within itself. Therefore, if it is to improve its correspondence with reality, it requires mechanisms through which information from outside the model can challenge and modify it.
And that, I think, is the counterexample to the universality of P1.
It introduces a class of actions that P1 does not describe: actions whose object is not choosing between outcomes within the model, but maintaining and improving the model through which outcomes can subsequently be evaluated.
I think this is exactly where my argument needs to be more precise. By “defective” I do not mean “expected to produce worse consequences.” If I meant that, then I would simply be making the EV argument again.
I mean defective as a model of reality.
Suppose I have a model M and I discover X. I come to believe that X is a real feature of the world and relevant to the domain I am trying to model. I then have two options:
A: incorporate X into M;
B: knowingly maintain M while excluding X.
It is perfectly possible that, for some particular goal, B produces better outcomes than A. I am not denying that. Indeed, that possibility is exactly why I don't want to define the defect in consequentialist terms.
My claim is instead:
Model of reality + recognised relevant information → norm to remain open to that information.
If I deliberately exclude X, I may still have a useful decision-making instrument. But I can no longer say that the resulting model represents the relevant part of reality without qualification.
So there are two different questions:
Is this a good instrument for achieving my objective?
and
Is this a reliable representation of the reality on which that objective depends?
The first is an ordinary EV question.
The second is prior to it.
And this is precisely where I think P1 becomes problematic. P1 says that a justified preference between A and B requires comparing their downstream consequences. But the decision to revise the model is not itself simply a comparison between downstream consequences within a fixed model. It concerns the adequacy of the model that will subsequently be used to make those comparisons.
You ask:
“Perhaps we should omit this new finding, if the consequences of doing so are better?”
Yes — if “should” means “is the action that maximizes the relevant objective.”
But then we have already moved back to ordinary practical reasoning.
My question is one level earlier:
What makes the model a sufficiently reliable model on which that practical reasoning can be based?
If I say that I should knowingly exclude X because doing so has better consequences, I can calculate those consequences only through some model. But that model must already determine what counts as relevant information and what consequences count in the calculation.
So the model-revision question cannot be completely reduced to an EV comparison without introducing a criterion for model adequacy.
And that criterion is what I am calling an ontological ought.
On “what consequences are those?”
I think this is the most important question in your comment.
My answer is: they don't have to be consequences I have already specified.
That is precisely the point.
If I say:
“I should incorporate X because doing so will produce better consequences,”
then I have accepted the structure of P1.
But my claim is:
“I should not knowingly exclude a relevant aspect of reality merely to preserve my existing representation.”
That claim is about maintaining the relationship between the model and reality, not about predicting a particular downstream outcome.
And there is an important distinction here between ignorance and exclusion.
If I don't know X exists, then X cannot be represented. That is simply the unavoidable limitation of a finite agent.
But once I encounter X and judge it to be relevant, I have a different problem. I now have information that conflicts with the boundary of my existing model.
At that point, maintaining the old model is not merely “being unaware.” It is actively preserving a representation in the face of information that challenges it.
Thank you — I think these are exactly the right objections, and they help me clarify what I am claiming.
I think there are three separate issues here.
1. I am not challenging EV as a decision procedure
I agree that P1 is fundamentally an EV claim, and I am not arguing that EV calculations are useless. My objection is narrower.
P1 says that a justified preference for A over B requires an assessment that A's downstream consequences are better than B's. My counterexample concerns cases where A and B differ in what the agent takes to be relevant information in the first place.
Suppose I discover X, where X is a genuine feature of the world that my model previously omitted.
I now have two options:
A: incorporate X into the model;
B: knowingly continue using the model that excludes X.
My claim is not that A has higher expected value.
My claim is that the justification for A is not itself an EV comparison. A is a decision about the model through which subsequent EV comparisons are made.
Once X is incorporated, I can of course calculate expected values within the expanded model. But that does not tell me why I should have incorporated X in the first place.
That is the step I think P1 overlooks.
2. “There are countless other X's” is true — and I think it strengthens the point
Absolutely. There are presumably indefinitely many things that I do not know about.
But I don't think this creates a problem for the argument. It reveals the distinction I am trying to make.
I am not claiming that an agent can construct a complete model of reality. It cannot.
The relevant question is instead:
What should an agent do when it encounters information that it recognises as relevant but which its current model does not represent?
My answer is that the agent needs a structural norm of openness to correction.
That norm does not say “find every X.” That would obviously be impossible.
It says: when relevant information enters the agent's epistemic field, the agent should not systematically exclude it merely because incorporating it would disrupt the existing model.
This is important because P3 already grants that our models are incomplete. My argument is about what follows from that incompleteness for the maintenance of the model itself.
3. I think I may have confused “the conclusion is false” with “the conclusion does not follow”
You're right to flag this.
A successful attack on P1 does not establish that the conclusion is false. It establishes that the conclusion does not follow from the premises as stated.
So my claim should be:
If my counterexample succeeds, P1 is not universal, and therefore the argument is not deductively valid as stated.
The conclusion could nevertheless be true for other reasons.
That's actually an important distinction, and I should have made it explicit.
4. I don't think my argument requires a prediction model to “break” the universe
I agree with you that no prediction model can fully capture a messy, changing universe. But I think this is precisely why I am interested in the model-revision problem.
I am not assuming that the model can become complete.
Quite the opposite.
I am assuming:
the agent acts through a model;
the model is necessarily incomplete;
reality can therefore provide information that the model does not contain;
some of that information can be recognised as relevant;
the agent must decide what to do with that information.
At step 5, I think there is a choice that is prior to ordinary EV comparison:
Should X be incorporated into my model?
Only after answering that question can I ask:
Given my model, which action has the highest EV?
So I would distinguish:
model revision vs. action selection within a model
P1 seems to describe the latter. My counterexample concerns the former.
5. This is where I think the “idealized self” becomes interesting
You say that P1 references the idealized self and that unawareness wins if we cannot make EV calculations or know everything relevant.
I think this may actually expose the deeper issue.
If the idealized agent is defined as an agent that already has the correct model of all relevant consequences, then of course P1 becomes very difficult to challenge. But then the model-revision problem has been assumed away.
The interesting question for a finite agent is precisely how it can become better informed.
And this is where my proposed inversion comes from.
We normally write:
IS→OUGHT→ACTION.
But the IS available to an agent is itself model-dependent.
So there is a prior question:
What must I do to maintain a model capable of producing a reliable IS?
The first ought is not “I ought to choose A rather than B.”
It is something like:
I ought to maintain the conditions under which my model remains responsive to relevant information.
That includes openness to correction, willingness to update, and—when information is distributed among agents—maintaining channels through which other agents can correct my model.
This is why I think the argument has implications beyond this particular question.
I am not trying to replace EV with some alternative decision rule.
I am suggesting that EV itself operates downstream of a prior epistemic/ontological layer: the conditions under which the model on which the EV calculation operates can remain a model of reality.
Thanks for the generous response. You write that we "may have been compatible" and I'm "reacting to something you're not saying."
Here's my concern: I've come to recognize that reality operates as a dynamic network—nodes (people, institutions) whose capacity is constituted by the relationships among them. This isn't just a modeling choice; it's how cities function, how pandemics spread, how states maintain capacity. You don't work from this explicit recognition.
This creates an asymmetry. Once you see reality as a network, your Section 5 framework becomes incompatible with mine—not just incomplete, but incoherent. You explicitly frame the state as separate from people, optimizing for longtermist goals while managing preferences as constraints. But from the network perspective, this separation doesn't exist—the state's capacity just IS those relationships. You can't optimize one while managing the other.
Let me try to say this more directly: I've come to understand my own intelligence as existing not AT my neurons, but BETWEEN them—as a pattern of activation across connections. I am the edge, not the node. And I see society the same way: capacity isn't located IN institutions, it emerges FROM relationships. From this perspective, your Section 5 (state separate from people) isn't a simplification—it's treating edges as if they were nodes, which fundamentally misunderstands what state capacity is.
That's the asymmetry: your explicit framing (state separate from people) is incompatible with how I now understand reality. But if you haven't recognized the network structure, you'd just see my essay as "adding important considerations" rather than revealing a foundational incompatibility.
Thank you for engaging, and especially for the intelligence curse point—that's exactly the structural issue I'm trying to get at.
You suggest I'm arguing "we should care about some of those things intrinsically." Let me use AGI as an example to show why I don't think this is about intrinsic value at all:
What would an AGI need to persist for a million years?
Not "what targets should it optimize for" but "what maintains the AGI itself across that timespan?"
I think the answer is: diversity (multiple approaches for unforeseen challenges), error correction (detecting when models fail), adaptive capacity (sensing and learning, not just executing), and substrate maintenance (keeping the infrastructure running).
An AGI optimizing toward distant targets while destroying these properties would be destroying its own substrate for persistence. The daily maintenance—power, sensors, error detection—isn't preparation for the target. It IS what persistence consists of.
I think the same logic applies to longtermist societies. The question would shift from "how to allocate resources between present and future" to "are we maintaining or destroying the adaptive loop properties that enable any future to exist?" That changes what institutions would need to do—the essay explores some specific examples of what this might look like.
Does the AGI example help clarify the reframe I'm proposing?
I want to begin by thanking Owen Cotton-Barratt and Rose Hadshar for their thoughtful and important chapter. Their willingness to examine what longtermist societies might actually look like—moving beyond marginal analysis to whole-system thinking—opens necessary terrain. This essay is offered in that same spirit of serious engagement, not as refutation but as extension.
My response is constructive, though it may read as fundamental disagreement. I believe we share a deep concern: how do we enable human flourishing to persist? Where we differ, I think, is in our conceptual starting point, and this difference ramifies through everything that follows.
Two frameworks for thinking about persistence:
Cotton-Barratt and Hadshar work from what might be called a projection framework: existence is distributed across time, and the question is how to allocate resources between temporal slices—present people now, future people later. Within this framework, their insight is important: even extreme longtermism requires substantial investment in present welfare for instrumental reasons. People whose basic needs aren't met cannot do complex work.
My essay works from what might be called a process framework: existence is not quantity distributed across time but a continuous adaptive process. There is no "present existence" separate from "future existence"—only ongoing maintenance of adaptive capacity. The question becomes not how to optimize for distant projected outcomes, but whether we're maintaining the structures that enable any outcomes at all.
Why this difference matters:
These aren't just semantic alternatives. They lead to different institutional designs, different understandings of risk, and different responses to uncertainty.
Cotton-Barratt and Hadshar recognize that instrumental reasons require present welfare. I'm suggesting something stronger: that the process of maintaining present adaptive capacity—the sense-learn-adapt-coordinate-repair loop—isn't instrumental to distant goals but constitutive of what persistence means. The Tuesday-morning maintenance network isn't preparation for a future we're aiming toward; it is the future, continuously instantiated.
This leads to seeing different risks. The incentive gradient I describe—the structural drift toward configurations that optimize measurable proxies while degrading adaptive capacity—isn't visible from a projection framework because it looks like progress on longtermist goals right up until the system can no longer adapt to surprises.
What I hope this contributes:
Cotton-Barratt and Hadshar's analysis helps us think carefully about constraints and resource allocation. Their distinction between partial and strict longtermism, their attention to legitimacy concerns, their recognition of instrumental value—all of this is valuable.
My hope is that the process framework adds something complementary: a way to think about systemic resilience, about what makes persistence possible in the face of deep uncertainty, about why maintaining diversity, autonomy, error correction, and genuine interdependence might not be constraints on longtermism but prerequisites for anything to persist at all.
An invitation:
I'd be genuinely curious to hear how Cotton-Barratt and Hadshar see this difference. Is it a meaningful distinction? Are these frameworks reconcilable at different scales of analysis? When would we know which better serves long-term flourishing?
Perhaps the most important test is this: when unforeseen challenges arrive—as they inevitably will—which approach has preserved the adaptive capacity to sense them early, learn from evidence, coordinate responses, and iterate toward solutions?
I suspect we all want the same thing: a future where human flourishing continues. The question is how we think about—and design for—that persistence. I offer this essay as one contribution to that ongoing conversation.
A note on method: For transparency, I used Claude Sonnet 4.5 and ChatGPT-5 as thinking partners and writing tools for this essay—for structure, clarity, and articulation. The core framework, however, emerges from my hands-on work with dynamic network modeling of infectious diseases, and my training across biology, economics, and philosophy. The loop-maintenance perspective reflects years of thinking and exploration and was sparked by conceptual reflection on oak trees. The ideas are mine; the AI helped me say them clearly.
I agree. But first we need to conceptually break down this further. As AIs & humans (and anything intelligent) will become part of the same "intelligent network" where there are several inequalities which need to be addressed. This is is acknowledging the difference in maintenance of our substrate and the difference in our sensors, and a difference in our capacity to monitor and understand ourselves. Doing so will reveil - I belief the inequalities between humans, as well as between humans and AI - and will showcase also the huge differences between humans to wield power. We need to solve these all, which frankly requires a huge step-up in our democracies to become democratic and truly look after the long-term-stabilty of the full network (meaning overcoming nationalism, class divisions, sexism, racism etc.)
What I find an interesting perspective is to approach ethics from the point of view of a “network.” In our case, a network in which humans (or, more precisely, our intelligences) are the nodes, and the relationships between these intelligences are the edges.
For this network to exist, the nodes need to establish and maintain relationships. This “edge maintenance” can, in turn, be translated into what we call ethics or ethical behaviour. Whatever creates or restores these edges/relationships—and thereby enables the existence of the network—is just, correct, or virtuous. This is because, to make the intelligent nodes physically exist (to keep their substrate intact), the network itself must exist: the nodes are interdependent. One node grows wheat, another harvests it, another bakes bread, another distributes it, etc. Thus, ethics becomes about existence, which is much easier to comprehend.
Once you embrace this network between intelligent nodes, you can also start thinking about all subsequent dependencies in terms of nodes and edges/relationships. This neatly highlights the interdependences of our existence and leads me to formulate the meaning of life as: “Keep alive what keeps us/you alive.” As this becomes the internal logic of this interdependent network.
I’m curious who else finds this perspective interesting, as I believe that using the language of networks and complex systems in this context opens the door to thinking and talking more clearly about intelligence and AI alignment, (inter)national collaboration, (bio)diversity, evolution, etc
One more attempt: what if collective decision-making is really a learning problem?
I have noticed that I have become unusually invested in this problem.
Partly because I believe I see a solution, and partly because I seem to keep failing to articulate what I see in a way that makes it possible for other people to see it too.
So I want to try one more time.
This time I will start from my own profession.
I work as an infectious disease modeller and epidemiologist in a national public health institute. Ultimately, my job is about reducing disease burden in populations.
Some of the interventions we work with are remarkably effective. Measles vaccination is one example. Measles is also one of the diseases for which global eradication is conceivable.
And eradication creates a peculiar decision problem.
If we eradicate measles, we don't merely reduce disease burden for the people alive today. We change the world inherited by every future generation. Eventually, vaccination against measles would no longer be necessary either.
The important point for my argument is not whether this future benefit can be assigned some particular expected value. It is that eradication changes the future decision problem itself.
To eradicate measles, we need a long sequence of connected decisions.
We need surveillance. We need models. We need vaccination. We need to observe what happens. We need to discover where our assumptions were wrong. We need to change the strategy. We need to coordinate with other countries. We need to learn from failures elsewhere.
The decision is therefore not really:
A or B?
It is a continuous process:
What do we currently believe? What should we do given that belief? What happened? What did we get wrong? What does this tell us about the world? What should we believe now? What should we do next?
This brings me back to the cluelessness problem.
We live in an evolving universe. Our models are necessarily incomplete, and the future is not simply unknown in the sense of being a hidden answer waiting to be calculated. The world itself changes.
So perhaps the central problem of decision-making is not how to make the correct decision from a fixed model.
Perhaps it is how to remain capable of correcting the model on which the next decision will depend.
And this introduces another layer.
We can learn about the problem.
But we can also learn about how we learn about the problem.
For example, better surveillance may produce better information. Better information may improve our model. A better model may lead to better decisions. The results of those decisions provide new information, which can improve the surveillance and modelling process again.
So there is a recursive possibility:
we don't only optimise the solution; we improve the process by which we discover what the solution should be.
That is the point I was trying to get at in my previous post.
But there is a second part of the problem that I had not made sufficiently explicit.
Measles eradication is not an individual decision problem.
No individual can eradicate measles.
No single institution can eradicate measles.
It requires collective action.
And now the problem becomes considerably harder.
We need a collective that can continuously adapt its understanding of a changing world.
But nobody possesses the complete model.
Different people see different things. Different institutions have different information. Different models will be wrong in different ways.
So perhaps the objective should not be to make everybody hold the same model.
Perhaps the objective is to maintain a network in which different imperfect models can continuously correct one another.
This changes what I think “alignment” means.
We cannot necessarily align people on every proposition about the world.
We cannot even guarantee that we will initially agree on the correct solution.
But perhaps we can align on the process by which we discover that we are wrong.
This is where I think something interesting happens.
The things required for this process sound, at first, like ordinary virtues:
Be honest.
Listen.
Be curious.
Say when you are uncertain.
Correct others when you believe they are wrong.
Allow yourself to be corrected.
Distinguish what you observed from what you inferred.
Help others understand information they do not have.
Forgive mistakes sufficiently that people remain willing to participate.
Remain connected even when you disagree.
But viewed from the perspective of a distributed learning system, these are not merely virtues.
They are functional properties of the system.
Honesty protects the fidelity of the information entering the network.
Listening determines whether information actually reaches another person's model.
Correction provides an error signal.
Curiosity drives exploration of uncertainty.
Forgiveness helps preserve the connections through which future information can travel.
And willingness to be corrected keeps the individual model open to updating.
The interesting thing is that this can become self-reinforcing.
Better relationships allow better information exchange.
Better information exchange allows better collective learning.
Better collective learning can increase trust in the relationships that made it possible.
And the reverse is also possible:
Fear produces concealment.
Concealment produces poorer information.
Poorer information produces worse models.
Worse models produce worse decisions.
Worse decisions produce distrust.
And distrust produces more fear and concealment.
So perhaps collective intelligence is not simply a property of how much information a group possesses.
Perhaps it is partly a property of whether the relationships between its members preserve the capacity for correction.
This also changes how I think about decentralisation.
If we don't know beforehand what the correct model of the future problem will be, we cannot simply distribute a correct solution from the centre.
Instead, we may need to distribute the capacity to learn.
Not agreement on the answer.
Agreement on the practices that allow answers to be challenged, revised and improved.
This is why I have started thinking about the “verbs of learning”.
Not what everyone should believe.
What everyone should be capable of doing.
Listening.
Questioning.
Explaining.
Correcting.
Updating.
Admitting uncertainty.
Seeking disconfirming information.
Supporting someone else's learning.
Being willing to change one's mind.
These behaviours do not automatically produce agreement. Nor do they guarantee that a collective will choose the right goal.
But they may create something more fundamental:
a collective that remains capable of discovering that it is wrong.
And perhaps that is the deeper problem I have been trying to describe.
Decision theory asks:
What should we do?
But under radical uncertainty, and especially when the decision is collective, perhaps we also need to ask:
How do we remain capable of discovering that what we are doing is wrong?
And if that capacity itself can be improved, then we arrive at a recursive problem:
We learn about the world.
We learn how to learn about the world.
And we learn together how to become better at learning together.
I suspect that this is not a new idea in the literature. Many pieces of it almost certainly exist in different disciplines.
But I increasingly suspect that putting these pieces together points toward something important:
The fundamental requirement for collective intelligence may not be agreement. It may be collective corrigibility.
Not a collective that is permanently right.
A collective that remains capable of becoming less wrong.
My original argument was aimed at P1. I argued that there is a class of decisions—decisions about how a model learns and changes—that cannot straightforwardly be reduced to comparisons of downstream consequences within that model.
The discussion has made me realise that this points to a more constructive response to DiGiovanni's challenge.
Suppose the cluelessness argument is correct. What should an impartial altruist do?
I think the answer is: we should shift part of our attention from predicting the future to preserving our capacity to correct our predictions.
This is not a rejection of expected value. EV remains useful whenever our model is sufficiently informative to support a comparison. The problem is what to do when we cannot know whether it is.
A finite intelligence faces a peculiar problem:
This creates a distinction between two forms of rationality.
Predictive rationality:
Given my model, which action has the best expected consequences?
Corrective rationality:
Given that my model may be wrong, what keeps me capable of discovering and correcting that wrongness?
The second is not simply epistemic humility. It concerns the architecture of the decision-making process itself.
And this matters because all models are necessarily limited. The relevant question is therefore not whether we can construct a model that is complete, but whether we can maintain an intelligence that remains responsive to what its model leaves out.
Consider two strategies. One attempts to maximise the apparent value of actions under a particular model. The other preserves the conditions through which that model can be challenged: listening to independent perspectives, genuinely exchanging information, allowing disagreement to remain visible, acknowledging errors, and changing one's representation when it no longer accommodates what one encounters.
The second strategy does not necessarily have higher expected value. That is precisely the point. If we cannot reliably compare long-run consequences, we need to consider properties of the intelligence doing the comparing.
This suggests a progression:
But correction introduces a further problem.
An intelligence can fail not only because its model is wrong, but because the way it responds to being wrong is itself inadequate. It may hear another perspective without allowing that perspective to affect its model. It may encounter disagreement without investigating it. It may acknowledge an error without examining what made the error possible. It may correct a conclusion while preserving the assumptions that generated the error.
So the problem is no longer simply:
It becomes:
This also gives a more precise answer to the question raised in the comments: what makes a model “open to correction” if all models are flawed?
Not the expectation that correction will make it correct.
Rather:
And because those channels can themselves fail, we need to examine and change the way we listen, interpret, disagree, update, and correct.
This opens a new domain of rationality: the practice of meta-thinking. Under cluelessness, the impartial altruist should ask not only whether its model is wrong, but what it must do to remain capable of discovering that it is wrong: Whose perspective must it genuinely hear? What information must it allow to challenge its current representation? When confronted with disagreement, must it defend its model, or make the disagreement itself an object of inquiry? When an error is exposed, must it merely correct the conclusion, or examine why the error was possible in the first place? What must it be willing to change, and what must it keep open, for correction to remain possible?
These are not merely questions about acquiring better information about the future. They concern the present conditions under which an impartial altruist can justifiably move from an “is” to an “ought.” The paradox is that the actions required to maintain those conditions are themselves actions for which we need an “ought.”
The impartial altruist therefore has to reason reflexively:
This is what I mean by reflexive intelligence: intelligence begins to reason about the conditions of its own ability to reason.
And I think this gives us a constructive response to cluelessness that is neither a rejection of consequentialism nor an attempt to restore confidence in prediction.
Under genuine cluelessness, rationality does not end. It becomes reflexive.
I think I need to correct you, because I believe you have misunderstood the role of the counterexample.
I am not claiming that adding more information to a model necessarily produces better decisions. All models are necessarily flawed. The important point is that they are not necessarily flawed in the same domains.
A model can therefore improve by interacting with other models and treating genuinely different perspectives as information about its own blind spots. That interaction is what I mean by learning.
And I think this is the step that is missing from P1.
P1 describes the choice between A and B given a model of the world. My counterexample introduces a prior class of decisions: decisions about how the model itself learns and changes.
That learning step does not straightforwardly have an EV.
Consider the classic example of the six blind men and the elephant. Each person encounters a different part of the elephant and consequently constructs a different model: one thinks it is a snake, another a wall, another a tree, and so on.
None of the individual models is simply “the correct model”.
But the six agents can exchange information. They can recognise that their observations conflict. They can ask questions, compare perspectives, change their interpretations, and revise their models.
Through this interaction, the group can construct a representation that is closer to the elephant than any individual model.
Those decisions are not choices between A and B within a fixed model. They are decisions about how the models interact so that their blind spots can become visible to one another.
That is the step I am introducing into the premises.
And I think this is why I call the resulting principles ontological oughts. They are not rules about which outcome is better. They are rules concerning the conditions under which a finite model can remain aligned with a reality it can never completely represent.
For example, suppose I am modelling a situation and decide not to include the perspective of person Z.
That exclusion does not necessarily have an identifiable EV. I may not even be able to calculate what information I have excluded, because I have excluded it from the model through which I am doing the calculation.
But it can nevertheless matter structurally. My model may become increasingly misaligned with Z's model, while Z's model may simultaneously become increasingly misaligned with mine. We lose the possibility of correcting each other's blind spots.
The important point is therefore not that including Z necessarily produces better outcomes.
It is that excluding a potentially informative model removes a possible mechanism through which the limitations of my own model can be exposed.
This is where I think the issue goes deeper than the fact that the universe is messy or changing.
The fundamental problem is that a model is, by definition, not reality. It is a compression or representation of reality.
A model can test aspects of its own internal consistency. But it cannot, from within itself, establish that its own criteria for evaluating that consistency are sufficient to detect every way in which the model might be wrong.
In other words, the model has a blind spot concerning the adequacy of the mechanism by which it checks itself.
To resolve that blind spot, it requires another model.
But the other model has its own blind spots.
So we get:
M1↔M2
where each model can provide information about what the other cannot see from within itself.
This makes the models epistemically interdependent.
And this is the sense in which I think the learning step is prior to the EV step.
The sequence is not merely:
MODEL→A vs. B→ACTION.
There is a prior sequence:
MODEL→INTERACTION→CORRECTION→UPDATED MODEL→A vs. B.
The principles governing that interaction are therefore not themselves simply another instance of the A-vs-B problem.
They determine whether the model can continue to learn from reality at all.
I am not claiming that collaboration guarantees truth, or that every perspective should be accepted, or that more information necessarily produces better decisions.
I am claiming something more limited:
And that, I think, is the counterexample to the universality of P1.
It introduces a class of actions that P1 does not describe: actions whose object is not choosing between outcomes within the model, but maintaining and improving the model through which outcomes can subsequently be evaluated.
That is the distinction I was trying to make.
I think this is exactly where my argument needs to be more precise. By “defective” I do not mean “expected to produce worse consequences.” If I meant that, then I would simply be making the EV argument again.
I mean defective as a model of reality.
Suppose I have a model M and I discover X. I come to believe that X is a real feature of the world and relevant to the domain I am trying to model. I then have two options:
It is perfectly possible that, for some particular goal, B produces better outcomes than A. I am not denying that. Indeed, that possibility is exactly why I don't want to define the defect in consequentialist terms.
My claim is instead:
Model of reality + recognised relevant information → norm to remain open to that information.
If I deliberately exclude X, I may still have a useful decision-making instrument. But I can no longer say that the resulting model represents the relevant part of reality without qualification.
So there are two different questions:
and
The first is an ordinary EV question.
The second is prior to it.
And this is precisely where I think P1 becomes problematic. P1 says that a justified preference between A and B requires comparing their downstream consequences. But the decision to revise the model is not itself simply a comparison between downstream consequences within a fixed model. It concerns the adequacy of the model that will subsequently be used to make those comparisons.
You ask:
Yes — if “should” means “is the action that maximizes the relevant objective.”
But then we have already moved back to ordinary practical reasoning.
My question is one level earlier:
If I say that I should knowingly exclude X because doing so has better consequences, I can calculate those consequences only through some model. But that model must already determine what counts as relevant information and what consequences count in the calculation.
So the model-revision question cannot be completely reduced to an EV comparison without introducing a criterion for model adequacy.
And that criterion is what I am calling an ontological ought.
On “what consequences are those?”
I think this is the most important question in your comment.
My answer is: they don't have to be consequences I have already specified.
That is precisely the point.
If I say:
then I have accepted the structure of P1.
But my claim is:
That claim is about maintaining the relationship between the model and reality, not about predicting a particular downstream outcome.
And there is an important distinction here between ignorance and exclusion.
If I don't know X exists, then X cannot be represented. That is simply the unavoidable limitation of a finite agent.
But once I encounter X and judge it to be relevant, I have a different problem. I now have information that conflicts with the boundary of my existing model.
At that point, maintaining the old model is not merely “being unaware.” It is actively preserving a representation in the face of information that challenges it.
That is the transition I am interested in.
Thank you — I think these are exactly the right objections, and they help me clarify what I am claiming.
I think there are three separate issues here.
1. I am not challenging EV as a decision procedure
I agree that P1 is fundamentally an EV claim, and I am not arguing that EV calculations are useless. My objection is narrower.
P1 says that a justified preference for A over B requires an assessment that A's downstream consequences are better than B's. My counterexample concerns cases where A and B differ in what the agent takes to be relevant information in the first place.
Suppose I discover X, where X is a genuine feature of the world that my model previously omitted.
I now have two options:
My claim is not that A has higher expected value.
My claim is that the justification for A is not itself an EV comparison. A is a decision about the model through which subsequent EV comparisons are made.
Once X is incorporated, I can of course calculate expected values within the expanded model. But that does not tell me why I should have incorporated X in the first place.
That is the step I think P1 overlooks.
2. “There are countless other X's” is true — and I think it strengthens the point
Absolutely. There are presumably indefinitely many things that I do not know about.
But I don't think this creates a problem for the argument. It reveals the distinction I am trying to make.
I am not claiming that an agent can construct a complete model of reality. It cannot.
The relevant question is instead:
My answer is that the agent needs a structural norm of openness to correction.
That norm does not say “find every X.” That would obviously be impossible.
It says: when relevant information enters the agent's epistemic field, the agent should not systematically exclude it merely because incorporating it would disrupt the existing model.
This is important because P3 already grants that our models are incomplete. My argument is about what follows from that incompleteness for the maintenance of the model itself.
3. I think I may have confused “the conclusion is false” with “the conclusion does not follow”
You're right to flag this.
A successful attack on P1 does not establish that the conclusion is false. It establishes that the conclusion does not follow from the premises as stated.
So my claim should be:
The conclusion could nevertheless be true for other reasons.
That's actually an important distinction, and I should have made it explicit.
4. I don't think my argument requires a prediction model to “break” the universe
I agree with you that no prediction model can fully capture a messy, changing universe. But I think this is precisely why I am interested in the model-revision problem.
I am not assuming that the model can become complete.
Quite the opposite.
I am assuming:
At step 5, I think there is a choice that is prior to ordinary EV comparison:
Should X be incorporated into my model?
Only after answering that question can I ask:
Given my model, which action has the highest EV?
So I would distinguish:
model revision vs. action selection within a model
P1 seems to describe the latter. My counterexample concerns the former.
5. This is where I think the “idealized self” becomes interesting
You say that P1 references the idealized self and that unawareness wins if we cannot make EV calculations or know everything relevant.
I think this may actually expose the deeper issue.
If the idealized agent is defined as an agent that already has the correct model of all relevant consequences, then of course P1 becomes very difficult to challenge. But then the model-revision problem has been assumed away.
The interesting question for a finite agent is precisely how it can become better informed.
And this is where my proposed inversion comes from.
We normally write:
IS→OUGHT→ACTION.
But the IS available to an agent is itself model-dependent.
So there is a prior question:
What must I do to maintain a model capable of producing a reliable IS?
That gives something like:
ONTOLOGICAL OUGHT→RELIABLE IS→PRACTICAL OUGHT→ACTION
The first ought is not “I ought to choose A rather than B.”
It is something like:
That includes openness to correction, willingness to update, and—when information is distributed among agents—maintaining channels through which other agents can correct my model.
This is why I think the argument has implications beyond this particular question.
I am not trying to replace EV with some alternative decision rule.
I am suggesting that EV itself operates downstream of a prior epistemic/ontological layer: the conditions under which the model on which the EV calculation operates can remain a model of reality.
And that is the part I am interested in.
Thanks for the generous response. You write that we "may have been compatible" and I'm "reacting to something you're not saying."
Here's my concern: I've come to recognize that reality operates as a dynamic network—nodes (people, institutions) whose capacity is constituted by the relationships among them. This isn't just a modeling choice; it's how cities function, how pandemics spread, how states maintain capacity. You don't work from this explicit recognition.
This creates an asymmetry. Once you see reality as a network, your Section 5 framework becomes incompatible with mine—not just incomplete, but incoherent. You explicitly frame the state as separate from people, optimizing for longtermist goals while managing preferences as constraints. But from the network perspective, this separation doesn't exist—the state's capacity just IS those relationships. You can't optimize one while managing the other.
Let me try to say this more directly: I've come to understand my own intelligence as existing not AT my neurons, but BETWEEN them—as a pattern of activation across connections. I am the edge, not the node. And I see society the same way: capacity isn't located IN institutions, it emerges FROM relationships. From this perspective, your Section 5 (state separate from people) isn't a simplification—it's treating edges as if they were nodes, which fundamentally misunderstands what state capacity is.
That's the asymmetry: your explicit framing (state separate from people) is incompatible with how I now understand reality. But if you haven't recognized the network structure, you'd just see my essay as "adding important considerations" rather than revealing a foundational incompatibility.
Does this help clarify where I'm coming from?
Thank you for engaging, and especially for the intelligence curse point—that's exactly the structural issue I'm trying to get at.
You suggest I'm arguing "we should care about some of those things intrinsically." Let me use AGI as an example to show why I don't think this is about intrinsic value at all:
What would an AGI need to persist for a million years?
Not "what targets should it optimize for" but "what maintains the AGI itself across that timespan?"
I think the answer is: diversity (multiple approaches for unforeseen challenges), error correction (detecting when models fail), adaptive capacity (sensing and learning, not just executing), and substrate maintenance (keeping the infrastructure running).
An AGI optimizing toward distant targets while destroying these properties would be destroying its own substrate for persistence. The daily maintenance—power, sensors, error detection—isn't preparation for the target. It IS what persistence consists of.
I think the same logic applies to longtermist societies. The question would shift from "how to allocate resources between present and future" to "are we maintaining or destroying the adaptive loop properties that enable any future to exist?" That changes what institutions would need to do—the essay explores some specific examples of what this might look like.
Does the AGI example help clarify the reframe I'm proposing?
Afterword: A Note of Appreciation and Reflection
I want to begin by thanking Owen Cotton-Barratt and Rose Hadshar for their thoughtful and important chapter. Their willingness to examine what longtermist societies might actually look like—moving beyond marginal analysis to whole-system thinking—opens necessary terrain. This essay is offered in that same spirit of serious engagement, not as refutation but as extension.
My response is constructive, though it may read as fundamental disagreement. I believe we share a deep concern: how do we enable human flourishing to persist? Where we differ, I think, is in our conceptual starting point, and this difference ramifies through everything that follows.
Two frameworks for thinking about persistence:
Cotton-Barratt and Hadshar work from what might be called a projection framework: existence is distributed across time, and the question is how to allocate resources between temporal slices—present people now, future people later. Within this framework, their insight is important: even extreme longtermism requires substantial investment in present welfare for instrumental reasons. People whose basic needs aren't met cannot do complex work.
My essay works from what might be called a process framework: existence is not quantity distributed across time but a continuous adaptive process. There is no "present existence" separate from "future existence"—only ongoing maintenance of adaptive capacity. The question becomes not how to optimize for distant projected outcomes, but whether we're maintaining the structures that enable any outcomes at all.
Why this difference matters:
These aren't just semantic alternatives. They lead to different institutional designs, different understandings of risk, and different responses to uncertainty.
Cotton-Barratt and Hadshar recognize that instrumental reasons require present welfare. I'm suggesting something stronger: that the process of maintaining present adaptive capacity—the sense-learn-adapt-coordinate-repair loop—isn't instrumental to distant goals but constitutive of what persistence means. The Tuesday-morning maintenance network isn't preparation for a future we're aiming toward; it is the future, continuously instantiated.
This leads to seeing different risks. The incentive gradient I describe—the structural drift toward configurations that optimize measurable proxies while degrading adaptive capacity—isn't visible from a projection framework because it looks like progress on longtermist goals right up until the system can no longer adapt to surprises.
What I hope this contributes:
Cotton-Barratt and Hadshar's analysis helps us think carefully about constraints and resource allocation. Their distinction between partial and strict longtermism, their attention to legitimacy concerns, their recognition of instrumental value—all of this is valuable.
My hope is that the process framework adds something complementary: a way to think about systemic resilience, about what makes persistence possible in the face of deep uncertainty, about why maintaining diversity, autonomy, error correction, and genuine interdependence might not be constraints on longtermism but prerequisites for anything to persist at all.
An invitation:
I'd be genuinely curious to hear how Cotton-Barratt and Hadshar see this difference. Is it a meaningful distinction? Are these frameworks reconcilable at different scales of analysis? When would we know which better serves long-term flourishing?
Perhaps the most important test is this: when unforeseen challenges arrive—as they inevitably will—which approach has preserved the adaptive capacity to sense them early, learn from evidence, coordinate responses, and iterate toward solutions?
I suspect we all want the same thing: a future where human flourishing continues. The question is how we think about—and design for—that persistence. I offer this essay as one contribution to that ongoing conversation.
A note on method: For transparency, I used Claude Sonnet 4.5 and ChatGPT-5 as thinking partners and writing tools for this essay—for structure, clarity, and articulation. The core framework, however, emerges from my hands-on work with dynamic network modeling of infectious diseases, and my training across biology, economics, and philosophy. The loop-maintenance perspective reflects years of thinking and exploration and was sparked by conceptual reflection on oak trees. The ideas are mine; the AI helped me say them clearly.
I agree. But first we need to conceptually break down this further. As AIs & humans (and anything intelligent) will become part of the same "intelligent network" where there are several inequalities which need to be addressed. This is is acknowledging the difference in maintenance of our substrate and the difference in our sensors, and a difference in our capacity to monitor and understand ourselves. Doing so will reveil - I belief the inequalities between humans, as well as between humans and AI - and will showcase also the huge differences between humans to wield power. We need to solve these all, which frankly requires a huge step-up in our democracies to become democratic and truly look after the long-term-stabilty of the full network (meaning overcoming nationalism, class divisions, sexism, racism etc.)
Hi all,
What I find an interesting perspective is to approach ethics from the point of view of a “network.” In our case, a network in which humans (or, more precisely, our intelligences) are the nodes, and the relationships between these intelligences are the edges.
For this network to exist, the nodes need to establish and maintain relationships. This “edge maintenance” can, in turn, be translated into what we call ethics or ethical behaviour. Whatever creates or restores these edges/relationships—and thereby enables the existence of the network—is just, correct, or virtuous. This is because, to make the intelligent nodes physically exist (to keep their substrate intact), the network itself must exist: the nodes are interdependent. One node grows wheat, another harvests it, another bakes bread, another distributes it, etc. Thus, ethics becomes about existence, which is much easier to comprehend.
Once you embrace this network between intelligent nodes, you can also start thinking about all subsequent dependencies in terms of nodes and edges/relationships. This neatly highlights the interdependences of our existence and leads me to formulate the meaning of life as: “Keep alive what keeps us/you alive.” As this becomes the internal logic of this interdependent network.
I’m curious who else finds this perspective interesting, as I believe that using the language of networks and complex systems in this context opens the door to thinking and talking more clearly about intelligence and AI alignment, (inter)national collaboration, (bio)diversity, evolution, etc