TLDR; CTMCs are a mathematically accurate way of modelling X-Risks. Skip to section X-Risk Model using CTMCs to see the results of a model I performed using Toby Ord's estimates in The Precipice.

Purpose

Continuous-Time Markov Chains (CTMCs) are a popular tool used throughout industry (e.g. insurance, epidemiology) and academia to model multi-state systems.

In this post, I discuss how CTMCs have desirable features that make them well-suited to model Existential Risk (X-Risk) scenarios. I then produce a model based off of Toby Ord's probability estimates (The Precipice) for demonstration purposes.

By specifying a system of X-Risks and probability assumptions such as Ord's, I show how mathematical solutions can be derived to understand the implications of the system of assumptions.

• The Total X-Risk Probability
• Marginal Reduction in Total X-Risk from eliminating one risk
• Expected time until extinction
• Likeliest X-Risk at any given moment in time

Two applications of CTMCs I discuss in this post are:

1. Informing cause prioritisation decisions
2. A method to sense check and revise probability estimates

I am able to derive some results, and provide my suggestions on interpretation and application.

I also discuss limitations and weaknesses of CTMCs, and provide a brief comparison to alternative modelling approaches.

Mathematical Pre-requisites

This is a very maths-heavy forum post, but please don't get intimidated by the technical mathematical derivations which are not important to understanding the final results.

My hope is that most readers interested in X-Risk models are able to takeaway the main points about the capabilities of CTMC modelling, only requiring an understanding of:

• probability concepts (e.g. dependent vs independent probabilities)
• X-Risk scenarios

But to readers wishing to conduct CTMC modelling themselves and follow the mathematical derivations, you will need to have an understanding of

• probability distributions
• ordinary differential equations
• matrix exponentiation
• markov chains

Readers with actuarial, biostatistics, or similar backgrounds may meet many of these pre-requisites, and indeed CTMCs are applied in these fields. I personally have an insurance/actuarial background, and was inspired by those techniques used in industry.

For such readers, I found this to be an excellent resource on CTMCs: https://mpaldridge.github.io/math2750/S18-forward-backward.html

Continuous-Time Markov Chains Introduction

CTMCs are used to model a system with multiple states, where transitions can take place at any point in time from one state to another.

To familiarise readers to the concept of CTMCs, here I introduce the Health-Sickness-Death (HSD) model, and talk through some core CTMC concepts. This is a popular model used to model hospital patients.

In the above example, there is one absorbing state (Death) and two non-absorbing states (Healthy and Sick). The patient can move back and forth between the non-absorbing states indefinitely. Once in the absorbing state (Dead) however, there can be no further transitions. This is indicated in the diagram by the absence of arrows pointing outwards from the Death state.

The notation used for the probability of transitioning from one state to another (e.g. Healthy to Dead) is:
$P($State at time $x+t$ =Dead| State at time $x$ = Healthy$)=p_x^{HD}(t)$.

The Greek letters (e.g. ${\mu }_x$) are transition intensities, which I like to think of as "instantaneous probabilities" that a patient moves from one state to another. They're not real probabilities as they sum to zero/ can be negative/ can exceed 1, but that hasn't stopped me from thinking of them as such. Transition intensities are a core concept in CTMCs, and later I explain how they help modellers think about risks more clearly.

Their formal definition is: ${\mu }_x=\underset{h\to 0}{lim}\frac{p_x^{HD}\left(h\right)}{h}$.

The subscript $x$ here refers to the age of the patient. And so these transition intensities in this HSD model themselves can be functions of age (or time). Such Markov Chains are described as being time-inhomogeneous. This is important to account for as we know older patients have higher death rates (e.g. ${\mu }_{70}>{\mu }_{30}$) than younger, and hence it would be inaccurate to use constant (time-homogeneous) intensities.

The intensities can be collected together and written into matrix form:

$A\left(x\right)=\left[\begin{array}{ccc}-({\sigma }_x+{\mu }_x)& {\sigma }_x& {\mu }_x\\ {\rho }_x& -({\rho }_x+{\nu }_x)& {\nu }_x\\ 0& 0& 0\end{array}\right]$ 
Note that the diagonals $a_{ii}=-\sum _{j\ne i}{a}_{ij}$ so that each row sums to zero.

By generalising for any two states ($m,n$), and taking the limit, we can arrive at a set of differential equations known as the Kolmogorov forward equations:

$\frac{d}{dt}{p}_x^{mn}\left(t\right)=\underset{h\to 0}{lim}\frac{p_x^{mn}(t+h)-p_x^{mn}\left(t\right)}{h}=\sum _j{p}_x^{mj}\left(t\right){\sigma }_{x+t}^{jn}$ , where $\sigma$ are our transition intensities.
This can also be expressed in matrix form:$\frac{d}{dt}P\left(t\right)=P\left(t\right)A\left(t\right)$

This a system of ordinary differential equations that can be solved simultaneously. The solution is $P\left(t\right)=e^{{\int }_x^{t}A\left(t\right)dt}$, where "e" is the matrix exponential operator.

Some exact results are shown below:

$p_x^{HH}\left(t\right)=\frac{-{\lambda }_1+{\lambda }_2+h}{2h}{e}^{-\frac{1}{2}{\int }_x^t({\lambda }_1+{\lambda }_2-h)dt}+\frac{{\lambda }_1-{\lambda }_2+h}{2h}{e}^{-\frac{1}{2}{\int }_x^t({\lambda }_1+{\lambda }_2+h)dt}$

$p_x^{HS}\left(t\right)=\frac{(-{\lambda }_1+{\lambda }_2+h)({\lambda }_1-{\lambda }_2+h)}{4h{\rho }_{x+t}}(e^{-\frac{1}{2}{\int }_x^t({\lambda }_1+{\lambda }_2-h)dt}-e^{-\frac{1}{2}{\int }_x^t({\lambda }_1+{\lambda }_2+h)dt})$

$p_x^{HD}\left(t\right)=1-p_x^{HH}\left(t\right)-p_x^{HS}\left(t\right)$

where $h=\sqrt{({\lambda }_1-{\lambda }_2{)}^2+4{\sigma }_{x+t}{\rho }_{x+t}}$ and ${\lambda }_i=-a_{ii}$

X-Risk Model using CTMCs

Similar to the Health Sickness Death Model, we can create a multi-state model for X-Risk scenarios. The special feature of X-Risk systems is that there are multiple absorbing states due to the multiple X-Risks.

To demonstrate, here I apply CTMCs to Toby Ord's estimates in The Precipice.

First I reverse-engineer Ord's probability estimates into transition intensities.
I then assign an assumption (e.g. constant, exponential) about how each risk changes with time (time-inhomogeneity effects). Generally speaking, natural risks have constant intensities as the risks are independent of time. Anthropogenic risks however tend to vary with time owing to heightened periods of political tension, changing climate effects, and advances in technology etc.

The time-inhomogeneity assumptions I assign are my own and I don't wish them to be taken as serious recommendations. Properly informed assumptions can definitely be made by consulting with relevant experts for each X-Risk (e.g. Climate change researchers).

We can also calculate the dependent probabilities for each X-Risk using this result:

$p_x^i\left(t\right)=\frac{{\mu }_x^i}{\sum _j{\mu }_x^j}-\frac{{\mu }_{x+t}^i}{\sum _j{\mu }_{x+t}^j}{e}^{-\underset{0}{\overset{t}{\int }}\sum _j{\mu }_{x+t}^{j}dt}$

Note this equation is for the case when there is only 1 non-absorbing state. If you have multiple non-absorbing states you will need to re-solve the matrix exponential equation: $P\left(t\right)=e^{{\int }_x^{t}A\left(t\right)dt}$ with the appropriate transition matrix $A\left(t\right)$. The HSD model for example has 2 non-absorbing states and hence a different equation which I quote in the HSD section.

• Dependent probabilities (p)
  This is the probability of the particular X-Risk occurring with consideration of all other X-Risks. A full description of the space of all possible X-Risks is required to calculate these estimates.
• Independent probabilities (q)
  In contrast, independent probabilities only require judgement of the particular X-Risk in isolation, without consideration of other existential risks and how they may compete and interact. As such these probabilities are easier to assign and work with.

And here are the results.

Transition intensities and probability density

What we can see is how X-Risks evolve with time and compete with each other over the next century.

Notice that the probability density distributions are similar to the intensities but angled slightly downwards, (most visible in Nuclear War). What's happening is that the CTMC maths is taking into account the small but growing probability that an X-Risk event has already taken place.

Cumulative probability

Notice that Ord's estimate for Climate and Nuclear in 100 years is the same, i.e. $q_0^{nuclear}\left(100\right)=q_0^{climate}\left(100\right)=0.001$. However, this chart shows that nuclear is slightly higher at 100 years. What's happening is that Ord's estimates are "independent" (q) estimates that treat each X-Risk in isolation. But with CTMCs, we can calculate the "dependent" probabilities (p), and since the time assumptions I gave nuclear give it a higher chances of happening in the earlier part of the century, it ultimately has a slightly higher value by year 100.

This demonstrates a very powerful application of CTMCs in cause prioritisation. For example grant-makers are often deciding to allocate resources now, and may consider many measures in their decision-making:

• reducing total X-Risk
• reducing X-Risks over the next 100 years
• maximising Expected-Life Years

A grant-maker may decide Nuclear is more important to reduce because it provides both long-term and immediate benefits.

Total X-Risk

We can calculate the Total X-Risk probability as such:$p_x^{Total}\left(t\right)=\sum _i{p}_x^i\left(t\right)=1-e^{-{\int }_0^t\sum _j{\mu }_{x+t^{\ast }}^{j}d{t}^{\ast }}=1-\prod _j(1-q_x^j(t\left)\right)$
This result is not unique to CTMCs and is quoted in Ord's book The Precipice. But it does show that CTMCs are consistent with conventional probability calculations.

We can calculate the Expected Time until an X-Risk event:

$E\left(T\right)=\underset{0}{\overset{\infty }{\int }}1-p_0^{Total}\left(t\right)dt$

Just be careful that the time assumptions continue to be sensible beyond 100 years. For example I assumed that the Climate and AI risks (ie transition intensities) respectively became zero and constant after 100 years, as they were exponential and would have approached inifity with time.

This was my result: 194.31 years.

We can also calculate Expected Life-Years lived by human beings using this result:

$E\left(LY\right)=\underset{0}{\overset{\infty }{\int }}\left(\frac{d}{dt}{p}_0^{Total}\right(t\left)\right)\ast WorldPopulation\left(t\right)dt$

Population projections can be used from sources like the UN (albeit not to infinity). I have not performed such a calculation but am confident it is doable.

We can also calculate the net reduction in Total X-Risk from eliminating or mitigating one X-Risk.

NetReduction in Total X-Risk Probability$\left(t\right)=p_x^{Total}(t{)}_{originalmodel}-p_x^{Total}(t{)}_{newmodel}$

This was my result, if Nuclear War is eliminated as an X-Risk:
NetRedcution (100 years) = 0.1784552 - 0.1776329 = 0.0008223671

Note that this value is slightly less than the probability of a nuclear war happening, $p_0^{Nuclear}\left(100\right)=0.000948905$. This is because when one X-Risk disappears, the other X-Risks step in to "compete" for the freed up territory.

Similarly, we can also calculate:
Net Improvement in Years Saved = $E(T{)}_{originalmodel}-E(T{)}_{newmodel}$

Further Advanced Models

Here I speculate about possible further advanced models.

Elevated States

We can add further non-absorbing states to our models, to incorporate "elevated states" if we think it is important to model states which exacerbate the chance of X-Risk, whilst not being X-Risks themselves. For example, we may suppose that 3°C warming increases the likelihood of global conflict and nuclear war. These elevated states play a similar role to the Sickness state, in the HSD model. They may align the model closer to reality.

Multivariate transition intensities

I also wonder if it is possible to model transition intensities as multivariate functions. For example I would love to model the climate and nuclear intensities as functions of time and CO2 levels. Then it would be possible to rely on an explicit CO2 projections which would be superior to the example I gave using "elevated states". Unfortunately I cannot find any literature on such models.

Discrete effects

It is also possible to introduce "discrete effects" into the model for X-Risk events that may only take place at a known fixed point in time (e.g. date of a gender reveal party with universe-ending potential).

Incorporating uncertainty

It should also be possible to embed uncertainty into these models allowing models to show the degree of confidence/uncertainty, including the usage of confidence intervals/ prediction intervals. Here I suggest two ways.

We can assign a distribution (Bayesian credence) to the initial probability estimates (or transition intensities) and use monte-carlo simulations. 

I also wonder if it is possible to model transition intensities as random processes (e.g. random walk/ brownian motion). We can then make it so our certainty in risks decreases with time, reflecting greater uncertainty in the far future.

Applications

Cause Prioritisation

As discussed in the results section, CTMCs can produce many results that would be of interest to cause prioritisation research.

• Total X-Risk
• How X-Risks emerge and compete over time
• Expected Life-Years

These are highly relevant statistics to prioritisers. Prioritisers for example would be interested in estimating the effect of a potential intervention. Using CTMCs, they can estimate the marginal reduction in Total X-Risk, or Expected Life-Years, of different interventions.

They could even produce figures like Expected "Life-Years" saved per dollar, a quantity that is analogous to QALYs/DALYs per dollar used by charity evaluators like GiveWell. 

It's important not boil everything down into a single number (expected value/mean), and consider a wide variety of statistics (median, 95th percentile etc) and the distributions themselves. CTMCs can provide these.

CTMCs can even help cause prioritisers decide when to target their intervention, by targeting peak periods of risk when the intervention would be most effective.

There are lots of possibilities in this area.

Sense-checking probability estimates

CTMC models use probability estimates such as Ord's as inputs. But, by reversing this process, CTMCs are also useful to sense-check if these estimates were reasonable in the first place. If our CTMC model produces silly results, it's usually because of flawed assumptions.  

Transition intensities, a core concept in CTMCs, can also help modellers think about X-Risks in a clarified way because of three properties:

• Transition intensities are independent
  Unlike probability estimates which come in dependent and independent forms, intensities are always independent.
• Transition intensities are local
  Probabilities (density or cumulative), are non-local. They are calculated over a window of time $(x,t)$.
• Transition intensities are linear
  Unlike probabilities, intensities are linear. If we have two X-Risks with intensities ${\mu }^1$ & ${\mu }^2$, and we suppose that one is twice the other (i.e. ${\mu }^2=2{\mu }^1$), then we are supposing that in a small window of time (e.g. a day), X-Risk #2 is twice as likely to take place. This is also true for the same X-Risk at different points of time (e.g. AI today, vs AI in 10 years). Using a small window of time allows for us to think about risks linearly. This is helpful since over a long timeframe, non-linear effects arise. Thus it is often advantageous to think about risks first in terms of transition intensities.

In the weakenesses section on time-inhomogeneity, I also explain that transition intensities can help disagreeing X-Risk modellers understand the precise source of their disagreement better, and improve the quality of debate.

Advantages and Weaknesses of CTMCs

Advantages

CTMCs have many desirable features ready-made, that address important needs in X-risk modelling.

They capture important probability concepts in a mathematically accurate way that is often missed by other approaches.

• Marginally decreasing Total X-Risk probability due to X-Risks competing with each other
• Independent vs dependent probabilities
• Marginally diminishing X-Risk probability density over time, as an X-Risk may have already occurred

In my opinion, CTMCs can reveal unintuitive results accurately that often missed by less sophisticated models.

CTMCs are well-studied in academic literature and established industry methods in insurance and biostatistics. There are well documented guides and lecture notes on CTMCs. There are maintained pacakges in R, Python & Julia for CTMC modelling, as well as the underlying mathematical techniques (integration, matrix exponentiation etc).

CTMCs are also able to produce analytical (exact) mathematical solutions, instead of numerical approximations. This is more of a nice-to have feature. It allows us to present results, and analyse results with calculus techniques.

Weaknesses and Limitations

Time-inhomogeneity assumptions

Assigning time-inhomogeneity assumptions to transition intensities is half art and half science. It is very similar in this sense to assigning probability estimates to X-Risks. Assumptions are contentious and indeed political. Different experts/modellers will have different views and different intuitions.

In my opinion, all X-Risks models are making an assumption on how X-Risks evolve with time, it's just that it's often implicitly. CTMCs simply make the assumptions explicitly and obvious. I do think the transition intensities ultimately help modellers understand exactly what their intuition is saying (e.g. Risk doubles every n years, natural X-Risks are constant), and so it should help improve the debate amongst disagreeing modellers as they can pinpoint their precise difference. 

Memorylessness

A core assumption of Markov models is that they are memoryless (the Markov property). This means that the future (i.e. the transition intensities) only depends on the current state, and not the history of previously occupied states. Assuming the markov property greatly simplifies modelling but will be a severe flaw in the model if non-memoryless effects are material. This is a known flaw in the Health-Sickness-Death Model since patients that occupy the Sick state for prolonged periods of time are likelier to die.

Every X-Risk that I can think of is memoryless and so I do not think this is a problem. I tried to think of what memory would look like and the best I could think of is this: possibly a period of prolonged occupation in an "elevated state" (3C warming) may increase increases X-Risks more than fleeting occupation of the "elevated state". Even in this scenario, I feel that the memory effects are very minor. But I encourage readers to think about possible memory effects.

From a quick glance, there may be generalisations of CTMCs which include memory effects. They look to me as though they use something similar "elevated" states, and this may be a good enough workaround.

Impact, tractability, neglectedness

CTMCs do not naturally incorporate EA concepts like impact, tractability, neglectedness used in cause prioritisation.

These concepts need to be overlaid onto CTMCs, perhaps with other models/frameworks.

Comparison of CTMCs to alternate approaches

Discrete Markov Chain Models

Discrete Markov Chains use discrete, equally-spaced time units instead of continuous results. CTMCs are their natural analogue/ generalistion. It is common practice to approximate continuous-time effects by using many discrete small time-intervals. I fully expect discrete models to produce similar results (albeit slighlty off). This can be a good way to check two models are correctly working.

An EA user Arepo (Sasha), has employed Discrete-Time Markov Chains to model a similar phenomenon to X-Risks (civilisational collapse), and we have collaborated slightly. https://forum.effectivealtruism.org/posts/YnBwoNNqe6knBJH8p/modelling-civilisation-after-a-catastrophe

Exponential distribution model

Another approach to model X-Risks is to assume there is a constant probability of an X-Risks event happening in any year (Geometric distribution) or moment in time (Exponential distribution).

These models are simple and straightforward, and perfectly adequate for certain use cases. But they are not able to capture time-inhomogeneity effects.

This approach is actually a special case of CTMC approach (where we assume constant risk), and the exact same results are arrived at (an exponential distribution of $T\sim Exp\left(\sum _j{\mu }^j\right)$).

EA researcher Sam Nolan is currently using such a model for cause prioritisation: https://observablehq.com/d/bd8db447f3026662

Fixed time period models

Toby Ord's model in The Precipice uses a fixed time period of 100 years. With this model and his arrived at estimates, he is able to produce a calculation of Total X-Risk probability using probability theory, and discuss dependent vs independent probabilities and how X-Risks compete with eachother.

The main drawback of these models is that they are inflexible if wishing to use a different time period. They are effectively a discrete markov chain model of 1 time step.

I point out in this post that Ord's Total X-Risk probability calculation is reproduced with CTMCs, $p_x^{Total}\left(t\right)=1-\prod _j(1-q_x^j(t\left)\right)$. This gives me confidence that CTMCs are indeed a generalisation this approach.

Acknowledgements

Special thanks to Emrik Garden, Arepo (Sasha Cooper), Kramer Thompson, Sam Nolan, Rumi Salazar, Riley Harris, Bradley Tjandra for their assistance with this post.