MichaelDickens
Bio
Participation2
I do independent research on EA topics. I write about whatever seems important, tractable, and interesting (to me).
I have a website: https://mdickens.me/ Much of the content on my website gets cross-posted to the EA Forum, but I also write about some non-EA stuff over there.
I used to work as a software developer at Affirm.
Posts 90
Comments1042
RE the first con, my guess is that people will still hate AI even if they stand to make money off it via a SWF. As a basic test, you could look at AI favorability among people who do vs. don't have 401(k)s / retirement funds (which almost always include some investments in AI), although this test would probably be confounded by wealth and job stability, so there's probably a better test.
Pros:
- Reduces company incentive to dangerously race to build AI (also increases government incentive to do that, but governments don't really respond to incentives in the same way)
- Alleviates economic disenfranchisement due to AI displacing human workers (I'm more concerned about x-risk but this still matters more than zero)
Cons:
- Right now the general public hates AI. Putting everyone in a position to make money off AI might change that, and make it harder to get the safety regulations we actually need (this argument sounds right to me, but it also sounds suspiciously like accelerationism)
- Something about government overreach stifling capitalism (this is a real concern but it's dwarfed by x-risk concerns)
Neither-pro-nor-con:
- Has basically nothing to do with x-risk
But I think expecting returns something like 50-100% p.a. would be reasonable, based on historic performance of these funds.
You can't have it both ways. Either you can extrapolate from historical performance, in which case you should use long-run market returns of ~5%. Or you have a specific view on what's going to outperform, in which case it's a question of what your expectations are. You can't just take a fund that performed extremely well over the last 3–5 years and then say that level of performance will continue. If some thesis did really well in the past, then everyone else can also see that it did well, and you should assume it's now priced in unless you have some marginal thesis for why it's not priced in.
In fact, in the long run, stocks have exhibited 3–5 year reversals—that is, stocks that perform particularly well over the last 3–5 years tend to underperform the market going forward.[1] Mutual funds haven't exhibited reversals, but funds that outperform tend to regress to average performance.[2] At minimum, your thesis needs to have some explanation of why that's not going to happen this time.
I'm not saying you're wrong. AI stocks may well be a good investment. But I am pretty unimpressed by most arguments I've seen. "The last 5 years saw 50–100% return, therefore that same level of return will continue" is not a good argument—by (weak) default, it's evidence that future returns will be lower than the market, not higher. (I wrote more on my thoughts in a recent post, especially this section.)
[1] Fama, E. F., & French, K. R. (1988). Permanent and Temporary Components of Stock Prices. Journal of Political Economy, 96(2), 246–273. http://www.jstor.org/stable/1833108
[2] Berk, J. B., & Green, R. C. (2004). Mutual Fund Flows and Performance in Rational Markets. Journal of Political Economy, 112(6), 1269–1295. https://doi.org/10.1086/424739
I basically agree, although "dangerous approaches are tamped down" is doing most of the work here IMO. By default (i.e. no tamping-down), I expect the situation with a weakly-superhuman Scientist AI to be:
- a small number of sane people ask "are we doomed if we go with the following alignment strategy", and when it says yes, they don't do it
- a lot of people don't bother to ask at all, they just ask the Scientist AI how to build ASI
- a lot of people say "we have to build ASI before the reckless people in group 2", they build ASI using their best-guess alignment strategy that has an 88.5% chance of failing, and we die with 88.5% probability
(I think Bengio would agree that this is a concern, and would agree that we need global coordination on AI safety to make this work.)
It annoys me that they cite the Verge article as the "origin", rather than your article that the Verge article was based on.