New replication: I find that the results in Moretti (AER 2021) are caused by coding errors.
The paper studies agglomeration effects for innovation, but the results supporting a causal interpretation don't hold up.

https://twitter.com/michael_wiebe/status/1749462957132759489

Tweet-thread promoting Rotblat Day on Aug. 31, to commemorate the spirit of questioning whether a dangerous project should be continued.

What AI safety work should altruists do? For example, AI companies are self-interestedly working on RLHF, so there's no need for altruists to work on it. (And even stronger, working on RLHF is actively harmful because it advances capabilities.)

So far, the effective altruist strategy for global poverty has followed a high-certainty, low-reward approach. GiveWell only looks at charities with a strong evidence base, such as bednets and cash transfers. But there's also a low certainty, high reward approach: promote catch-up economic growth. Poverty is strongly correlated with economic development (urbanization, industrialization, etc), so encouraging development would have large effects on poverty. Whereas cash transfers have a large probability of a small effect, economic growth is a small probability of a large effect. (In general, we should diversify across high- and low-risk strategies.) In short, can we do “hits-based development”?

How can we affect growth? Tractability is the main problem for hits-based development, since GDP growth rates are notoriously difficult to change. However, there are a few promising options. One specific mechanism is to train developing-country economists, who can then work in developing-country governments and influence policy. Lant Pritchett gives the example of a think tank in India that influenced its liberalizing reforms, which preceded a large growth episode. This translates into a concrete goal: get X economists working in government in every developing country (where X might be proxied by the number in developed countries). Note that local experts are more likely than foreign World Bank advisors to positively affect growth, since they have local knowledge of culture, politics, law, etc.

I will focus on two instruments for achieving this goal: funding scholarships for developing-country scholars to get PhDs in economics, and funding think tanks and universities in developing countries. First, there are several funding sources within economics for developing-country students, such as Econometric Society scholarships, CEGA programs, and fee waivers at conferences. I will map out this funding space, contacting departments and conference organizers, and determine if more money could be used profitably. For example, are conference fees a bottleneck for developing-country researchers? Would earmarked scholarships make economics PhD programs accept more developing-country students? (We have to be careful in designing the funding mechanism, so that recipients don’t simply reduce funding elsewhere.) Next, I will organize fundraisers, so that donors have a ‘one-click’ opportunity to give money to hits-based development. (This might take the form of small recurring donations, or larger funding drives, or an endowment.) Then I will advertise these donation opportunities to effective altruists and others who want to promote hits-based development. (One potential large funder is the EA Global Health and Development Fund.)

My second approach is based on funding developing-country think tanks. Recently, IDRC led the Think Tank Initiative (TTI), which funded over 40 think tanks in 20 countries over 2009-2019. This program has not been renewed. My first step here would be to analyze the effectiveness of the TTI, and figure out whether it deserves to be renewed. While causal effects are hard to estimate, it seems reasonable to measure the number of think tanks, their progress under the program, and their effects on policy. To do this I will interview think tank employees, development experts, and the TTI organizers. Next I will determine what funding exists for renewing the program, as well as investigate whether a decentralized funding approach would work.

Should you "trust literatures, not papers"?
I replicated the literature on meritocratic promotion in China, and found that the evidence is not robust.

https://twitter.com/michael_wiebe/status/1750572525439062384

Has anyone looked at the effect of air pollution on animal welfare (farmed or wild)?

What does longtermism add beyond the importance-tractability-crowdedness framework? According to the ITC framework, we allocate resources to interventions with the highest expected value, given current funding levels. (More precisely, allocate the next dollar to the intervention with the highest marginal utility per dollar.) If those interventions turn out to be aimed at helping future generations, so what?

How much do non-nuclear countries exert control over nuclear weapons? How would the US-Soviet arms race have been different if, say, African countries were all as rich as the US, and could lobby against reckless accumulation of nuclear weapons?

What is the definition of longtermism, if it now includes traditional global health interventions like reducing lead exposure?

Will MacAskill says (bold added):

Well, it’s because there’s more of a rational market now, or something like an efficient market of giving — where the marginal stuff that could or could not be funded in AI safety is like, the best stuff’s been funded, and so the marginal stuff is much less clear. Whereas something in this broad longtermist area — like reducing people’s exposure to lead, improving brain and other health development — especially if it’s like, “We’re actually making real concrete progress on this, on really quite a small budget as well,” that just looks really good.

Do vaccinated children have higher income as adults?
I replicate a paper on the 1963 measles vaccine, and find that it is unable to answer the question.

https://twitter.com/michael_wiebe/status/1750197740603367689

Will says:

in order to assess the value (or normative status) of a particular action we can in the first instance just look at the long-run effects of that action (that is, those after 1000 years), and then look at the short-run effects just to decide among those actions whose long-run effects are among the very best.

Is this not laughable? How could anyone think that "looking at the 1000+ year effects of an action" is workable?

Longtermism is the view that positively influencing the longterm future is a key moral priority of our time.

Longtermism is a conclusion we arrive at by applying the EA framework of importance-tractability-crowdedness (where 'importance' is defined to include valuing future lives). Hence, EA is primary, and longtermism is secondary. EA tells us how to manage tradeoffs between benefitting the far future and doing good in the near term, and how to change our behavior as longtermist interventions hit diminishing returns.

Strong waterism: dying of thirst is very bad, because it prevents all of the positive contributions you could make in your life. Therefore, the most important feature of our actions today is their impact on the stockpile of potable water.

Crowdedness by itself is uninformative. A cause could be uncrowded because it is improperly overlooked, or because it is intractable. Merely knowing that a cause is uncrowded shouldn't lead you to make any updates.

Does longtermism vs neartermism boil down to cases of tiny probabilities of x-risk? 

When P(x-risk) is high, then both longtermists and neartermists max out their budgets on it. We have convergence.

When P(x-risk) is low, then the expected value is low for neartermists (since they only care about the next ~few generations) and high for longtermists (since they care about all future generations). Here, longtermists will focus on x-risks, while neartermists won't.

We need to drop the term "neglected". Neglectedness is crowdedness relative to importance, and the everyday meaning is "improperly overlooked". So it's more precise to refer to crowdedness ($ spent) and importance separately. Moreover, saying that a cause is uncrowded has a different connotation than saying that a cause is neglected. A cause could be uncrowded because it is overlooked, or because it is intractable; if the latter, it doesn't warrant more attention. But a neglected cause warrants more attention by definition.

More: https://forum.effectivealtruism.org/posts/fR55cjoph2wwiSk8R/formalizing-the-cause-prioritization-framework

I've written up my replication of Cook (2014) on racial violence and patenting by Black inventors.

Bottom line: I believe the conclusions, but I don't trust the results.

https://twitter.com/michael_wiebe/status/1749831822262018476

Longtermism is defined as holding that "what most matters about our actions is their very long term effects". What does this mean, formally? Below I set up a model of a social planner maximizing social welfare over all generations. With this model, we can give a precise definition of longtermism.

A model of a longtermist social planner

Consider an infinitely-lived representative agent with population size $N_t$. In each period there is a risk of extinction via an extinction rate ${\delta }_t$.

The basic idea is that economic growth is a double-edged sword: it increases our wealth, but also increases the risk of extinction. In particular, 'consumption research' develops new technologies $A_t$, and these technologies increase both consumption and extinction risk.

Here are the production functions for consumption and consumption technologies:

$c_t=A_t^{\alpha }\cdot \left(\text{number of consumption workers}\right)$

$\frac{d{A}_t}{dt}=A_t^{\varphi }\cdot \left(\text{number of consumption scientists}\right)$

However, we can also develop safety technologies to reduce extinction risk. Safety research produces new safety technologies $B_t$, which are used to produce 'safety goods' $h_t$.

Specifically,

$h_t=B_t^{\alpha }\cdot \left(\text{number of safety workers}\right)$

$\frac{d{B}_t}{dt}=B_t^{\varphi }\cdot \left(\text{number of safety scientists}\right)$

The extinction rate is ${\delta }_t=h_t^{-\beta }{A}_t^{\eta }$, where the number $A_t$ of consumption technologies directly increases risk, and the number $h_t$ of safety goods directly reduces it.

Let $M_t=exp(-{\int }_0^t{\delta }_{s}ds)=P\left(\text{being alive at time}t\right)$.

Now we can set up the social planner problem: choose the number of scientists (vs workers), the number of safety scientists (vs consumption scientists), and the number of safety workers (vs consumption workers) to maximize social welfare. That is, the planner is choosing an allocation of workers for all generations:

$L=\{\text{C scientists, C workers, S scientists, S workers}{\}}_{t=0}^{\infty }$

The social welfare function is:

$U={\int }_0^{\infty }{M}_t{N}_{t}u\left(c_t\right)e^{-\rho t}dt$

The planner maximizes utility over all generations ($t=0\text{to}\infty$), weighting by population size $N_t$, and accounting for extinction risk via $M_t$. The optimal allocation $L^{\ast }$ is the allocation that maximizes social welfare.

The planner discounts using $\rho =\zeta +\gamma g$ (the Ramsey equation), where we have the discount rate $\rho$, the exogenous extinction risk $\zeta$, risk-aversion $\gamma$ (i.e., diminishing marginal utility), and the growth rate $g$.  (Note that $\rho$ could be time-varying.)

Here there is no pure time preference; the planner values all generations equally. Weighting by population size means that this is a total utilitarian planner.

Defining longtermism

With the model set up, now we can define longtermism formally. Recall the informal definition that "what most matters about our actions is their very long term effects". Here are two ways that I think longtermism can be formalized in the model:

(1) The optimal allocation in our generation, $L_0^{\ast }$, should be focused on safety work: the majority (or at least a sizeable fraction) of workers should be in safety research of production, and only a minority in consumption research or production. (Or, $L_t$ for small values of $t$ (say $t<5$) to capture that the next few generations need to work on safety.) This is saying that our time has high hingeyness due to existential risks. It's also saying that safety work is currently uncrowded and tractable.

(2) Small deviations from $L_0^{\ast }$ (the optimal allocation in our generation) will produce large decreases in total social welfare $U$, driven by generations $t>100$ (or some large number). In other words, our actions today have very large effects on the long-term future. We could plot $U$ against $t$ for $L_0^{\ast }$ and some suboptimal alternative $\overline{L_0}$, and show that $U\left(\overline{L_0}\right)$ is much smaller than $U\left(L_0^{\ast }\right)$ in the tail.

While longtermism has an intuitive foundation (being intergenerationally neutral or having zero pure time preference), the commonly-used definition makes strong assumptions about tractability and hingeyness.

FTX Future Fund says they support "ambitious projects to improve humanity's long-term prospects". Does it seem weird that they're unanimously funding neartermist global health interventions like lead elimination?

Will MacAskill:

Lead Exposure Elimination Project. [...] So I saw the talk, I made sure that Clare was applying to  [FTX] Future Fund. And I was like, “OK, we’ve got to fund this.” And because the focus [at FTX] is longtermist giving, I was thinking maybe it’s going to be a bit of a fight internally. Then it came up in the Slack, and everyone was like, “Oh yeah, we’ve got to fund this.” So it was just easy. No brainer. Everyone was just totally on board.

How to make the long-term future go well: get every generation to follow the rule "leave the world better off than it was under the previous generation".

asteroid detection [...] approximately 300,000 additional lives in expectation for each $100 spent. [...]

Preventing future pandemics [...] 200 million extra lives in expectation for each $100 spent. [...]

the best available near-term-focused interventions save approximately 0.025 lives per $100 spent 
(source)

We should have a dashboard that tracks expected value per dollar for each cause area. This could be measured in lives saved, QALYs, marginal utility, etc, and could be measured per $1, $100, $1M, etc. We'd also want an estimate of diminishing returns: how much smaller would the expected value be for the next funding increment? And we should represent uncertainty by using a probability distribution instead of point estimates.

The argument for longtermism in a nutshell:

First, future people matter. [...] Second, the future could be vast. [...] Third, our actions may predictably influence how well this long-term future goes.

Here, whether longtermism ("positively influencing the long-term future is a key moral priority of our time") is true or false depends on whether our actions can predictably influence the far future. But it's bad to collapse such a rich topic down to a binary true/false. (Imagine having a website IsInfluencingTheLongTermFutureAKeyMoralPriority.com to tell you whether longtermism is true for your time period.)

I would instead use the ITC framework combined with the premises that future people matter and that the future could be vast. Then, we evaluate specific interventions according to cost-effectiveness (or more precisely, marginal utility per dollar, MU/$). If interventions aimed at the long-term future turn out to have the highest MU/$, then funding is directed to them. If these interventions become fully funded and hit diminishing returns (i.e., tractability worsens), then funding is directed to other causes. 

On this framing, we evaluate interventions on a case-by-case basis, and keep our focus on the ultimate goal: allocating resources across causes to maximize aggregate social welfare over all generations.

What are the comparative statics for how uncertainty affects decisionmaking? How does a decisionmaker's behavior differ under some uncertainty compared to no uncertainty?

Consider a social planner problem where we make transfers to maximize total utility, given idiosyncratic shocks to endowments. There are two agents, $A\text{and}B$, with endowments  $e_A=5$ (with probability 1) and $e_B=0\sim p,10\sim 1-p.$ So $B$ either gets nothing or twice as much as $A$.

We choose a transfer $T$ to solve:
$ma{x}_{T}u(5-T)+p\cdot u(0+T)+(1-p)\cdot u(10+T)\text{s.t.}0\le T\le 5$

For a baseline, consider $p=0.5$ and $u=ln$. Then we get an optimal transfer of $T^{\ast }=1.8$. Intuitively, as $p\to 0$, $T^{\ast }\to 0$ (if B gets 10 for sure, don't make any transfer from A to B), and as $p\to 1,T^{\ast }\to 2.5$ (if B gets 0 for sure, split A's endowment equally).

So that's a scenario with risk (known probabilities), but not uncertainty (unknown probabilities). What if we're uncertain about the value of $p$?

Suppose we think $p\sim F$, for some distribution $F$ over $[0,1]$. If we maximize expected utility, the problem becomes:
$ma{x}_{T}E\left[u\right(5-T)+p\cdot u(0+T)+(1-p)\cdot u(10+T\left)\right]\text{s.t.}0\le T\le 5$

Since the objective function is linear in probabilities, we end up with the same problem as before, except with $E\left[p\right]$ instead of $p$. If we know the mean of $F$, we plug it in and solve as before.

So it turns out that this form of uncertainty doesn't change the problem very much.

Questions:
- if we don't know the mean of $F$, is the problem simply intractable? Should we resort to maxmin utility?
- what if we have a hyperprior over the mean of $F$? Do we just take another level of expectations, and end up with the same solution?
- how does a stochastic dominance decision theory work here?

'Longtermism' is the view that positively influencing the long-term future is a key moral priority of our time. [from here]

It seems weird to make an 'ism' out of a currently highly cost-effective cause area. On the ITC framework, we expect these interventions to become less cost-effective as funding is directed to them and they hit diminishing returns and become less tractable. That is, if the EA community is functioning properly, the marginal dollar allocated to each cause will have the same effectiveness (otherwise, we could reallocate funding and do more good).

We can view longtermism as trying to achieve this reallocation: historically, x-risk interventions have been badly underfunded, and we need a movement to generate support for getting these interventions to their optimal funding levels. That seems fine, but it's important to maintain the context that the overall goal is to optimally allocate a budget across many causes.

Why don't models of intelligence explosion assume diminishing marginal returns? In the model below, what are the arguments for assuming a constant $\omega$, rather than diminishing marginal returns (eg, ${\omega }_t\to -\infty$). With diminishing returns, an AI can only improve itself at a dimishing rate, so we don't get a singularity.

https://www.nber.org/papers/w23928.pdf