Sequences
1
I do independent research on EA topics. I write about whatever seems important, tractable, and interesting (to me).
I have a website: https://mdickens.me/ Much of the content on my website gets cross-posted to the EA Forum, but I also write about some non-EA stuff over there.
I used to work as a software developer at Affirm.
It is hilarious(ly sad) that the largest present-day problem in the entire world only receives on the order of $5 million/year. I really hope these orgs get more funding in the near future.
The argument isn't that the variance on your EV is very high. The argument is that you can't have an EV at all. The challenge of unawareness sequence makes this argument in detail.
That looks like a link to an unpublished draft that only you can view. I can't see it
I agree that it doesn't make sense to make extremely risky investments and then be very risk-averse with your donations.
I'm not sure about the donation recommendations. Why should donors have a small core of low-risk donations? Why not donate all their money to the single highest-EV opportunity? Or why not give to a few different opportunities that they believe are the best candidates for highest-EV, without concern for risk?
To clarify, I'm not saying that's because LLMs don't have welfare. It could also be that they do have welfare, but their statements are not correlated to their internal experiences in the obvious way.
We know that we can get LLMs to express any welfare statement by saying e.g. "pretend you are a sentient AI that's having an awesome time / being tortured". It could be that all LLM outputs are roleplaying, but there is some true underlying experience that we don't (yet?) know how to observe.
Do we have much reason to believe that when a model outputs tokens describing good welfare, that's because it is having good subjective experiences? It seems to me that we don't.
This is more likely to be true if the care comes from some general theory of concern for sentient welfare, and less likely if it comes from something more arbitrary like values-learned-via-RL.
The question isn't, "Are current AIs capable of suffering?" The question is, "How big a deal is their suffering in expectation?" Probability is low, but expected importance is high.
The question isn't whether benchmarks will become useless. The question is, "is the probability high enough that we can't count on benchmarks?" To which the answer is yes.
None of these are from "core AI safety funders". Views in the EA community were all across the map, but I share James's impression that big funders (especially CG) were too reluctant to fund anything that's critical of AI companies.