I've been orienting to the field myself intensely during this year, after following it in the background for years.
Some takes:
Research that has impact is difficult.
AI twitter and AI safety papers on Arxiv surface a high number of papers every week, many of which are using shoddy statistics, bad experiment design, or other basic problems. Many good AIS orgs seem to aim for a high quality standard, and this may well be worth the cost.
Research output is also heavy-tailed. If a small fraction of people produce most of the useful work, a high bar can be correct.
Many organizations are growing as fast as they can. But due to research being difficult, and avoiding organizational drift, keeping quality high, that fast is still not that fast.
From outside, it's hard to tell someone moving toward good research apart from someone treading water. So the bar is partly about legible evidence, not raw quality. "We can't tell" is a common reason for rejection, distinct from "we can tell and the answer is no."
This is one reason “skill up first” is common advice: it’s a way to generate signal. Some things that can work: serious attempts at open challenges, replications or extensions of safety papers, benchmark work, public writeups, or programs like MATS, ARENA, SPAR, AI Safety Camp, etc., where the output of the program can itself become a credential. A nice thing about multiple of these routes is that they don’t require permission to start.
In any case, since you've thought of this for 300 hours, I definitely recommend getting some outside feedback! 80000 hours career advising can be one good option, since they have focused on AI Safety related advising recently.
As someone trying to broadly develop views on what kind of neglected work there is available in the AI Safety space, Richard Ngo's giving shortform was a very good read. Thanks for linking it!
I was quite worried when I saw the tweets going around, as well. I think the implications of the possibility of LLM sentience are already ethically quite large and biological computing intuitively has a much larger change of inducing sentience in the "computers".
But I'm broadly not sold on the moratorium concept.
I mean, a global moratorium would definitely be the ethically careful choice to do here. But I think if even a couple of countries allow the building of these datacenters, and other countries allow the purchase of biological computing, then it would be important to act on this in other ways than moratoriums as well.
Something like mandating x amount of research into the ethical implications of this per y amount of spending on biological computing (e.g. a pigouvian tax that is earmarked to solving the ethical problems present) would be what I would primarily advocate, at least from my European point of view.
In the regions where the data centers are being built campaigning for a moratorium or slowdown and generally raising public awareness sounds like something that should be attempted and it should be possibly to sell people on the ethical implications of brain-based computing...
(In general, I suspect there's insufficient social pressure on people to increase our donations to good causes
I think in general if we agree to a ballpark of "10% donations is enough to satisfice some goodness thresholds", and also to "It would be good for social pressure to exist for everyone to do at least threshold amount of good", I think it raises various considerations.
10% makes sense to me as a schelling point (and I think the tables that scale by income bracket are also sensible).
But if the threshold amount of good would be "Donate 10%, aim for an impactful career, become vegan" (which is what I feel the social pressure inside EA is pointing towards), I think that is already a significant ask for many people.
I think it is also important to note that some people are more motivated by trying to maximize impact and offset harm, and some people more motivated by minimizing harm and satisficing for impact. (Of course a standard total utilitarian model would output that whatever maximizes your net impact is best, but human value systems aren't perfectly utilitarian.)
How do "donate 10%, become vegan, aim for an impactful", and "donate 30%", and "donate 20%, aim for an impactful career" compare in effectiveness as norms? I think this is pretty hard to estimate.
What kind of social pressure are you pointing here? Is it more in the direction of "donate 30%" or "donate as much as you can and aim for an impactful career?" Or do you mean social pressure in the wider society, and not within the EA community?
(Fwiw I think people underestimate the value of effective marginal spending on themselves, when considering areas of spending where there is space for significant extra value (Like purchasing more free time.). People plausibly overestimate the value on some other things, especially if one doesn't do spending introspectiont.)
I find it intuitive that there could be a small set of objective moral facts but much smaller than is generally believed for moral realist positions + i do not think this can be justified rationally to a large degree. I think there can be contextual moral facts (as in "rational agents in a society would agree to cooperate on problem X" or "rational agents would agree to behave in a certain way on a moral problem, given following constraints"), but I do not think these are enough to justify an objective moral realism position.
I think the set of sensible moral views and positions is large, and thus think that morality is mostly not objective.
I don't know if this will be a useful comment but putting it here anyway. Personally, and most other people who feel like they are being judged too much or have too much performance pressure in EA, often do it themselves. I think there is a causation in how EA material and dynamics can facilitate people being more self-critical than is healthy, and to me that seems like a much more common problem than being judged by other EAs. (Didn't notice a survey question that would measure the thing I'm trying to point at here.)
The role of guilt and perfectionism in EA, and how EA as an environment focused on efficiency and doing the very best we can can lead to difficult mental hangups, and be more demanding than more traditional ways of doing good. (Traditional ways of doing good are often more focused in feeling good and feeling altruistic, which is useful for the good-doers wellbeing but suboptimal for actual amount of good done.)
Positivity focused ethics! The inbalance between negativity-biased vs. positively biased versions of utilitarianism, and implications of this on evaluating policy ideas and the medium-term future.
Any hints / info on what to look for in a mentor / how to find one? (Specifically for community building.)
I'm starting as a national group director in september, and among my focus topics for EAG London are group-focused things like "figuring out pointers / out of the box ideas / well-working ideas we haven't tried yet for our future strategy", but also trying to find a mentor.
These were some thoughts I came up with when thinking about this yesterday: - I'm not looking for accountability or day to day support. I get that from inside our local group. - I am looking for someone that can take a description of the higher level situation and see different things than I can. Either due to perspective differences or being more experienced and skilled. - Also someone who can give me useful input on what skills to focus on building in the medium term. - Someone whose skills and experience I trust, and when they say "plan looks good" it gives me confidence, when I'm trying to do something that feels to me like a long shot / weird / difficult plan and I specifically need validation that it makes sense.
On a concrete level I'm looking for someone to have ~monthly 1-1 calls with and some asynchronous communication. Not about common day to day stuff but larger calls.
I've been orienting to the field myself intensely during this year, after following it in the background for years.
Some takes:
This is one reason “skill up first” is common advice: it’s a way to generate signal. Some things that can work: serious attempts at open challenges, replications or extensions of safety papers, benchmark work, public writeups, or programs like MATS, ARENA, SPAR, AI Safety Camp, etc., where the output of the program can itself become a credential. A nice thing about multiple of these routes is that they don’t require permission to start.
In any case, since you've thought of this for 300 hours, I definitely recommend getting some outside feedback! 80000 hours career advising can be one good option, since they have focused on AI Safety related advising recently.
As someone trying to broadly develop views on what kind of neglected work there is available in the AI Safety space, Richard Ngo's giving shortform was a very good read. Thanks for linking it!
I was quite worried when I saw the tweets going around, as well. I think the implications of the possibility of LLM sentience are already ethically quite large and biological computing intuitively has a much larger change of inducing sentience in the "computers".
But I'm broadly not sold on the moratorium concept.
I mean, a global moratorium would definitely be the ethically careful choice to do here. But I think if even a couple of countries allow the building of these datacenters, and other countries allow the purchase of biological computing, then it would be important to act on this in other ways than moratoriums as well.
Something like mandating x amount of research into the ethical implications of this per y amount of spending on biological computing (e.g. a pigouvian tax that is earmarked to solving the ethical problems present) would be what I would primarily advocate, at least from my European point of view.
In the regions where the data centers are being built campaigning for a moratorium or slowdown and generally raising public awareness sounds like something that should be attempted and it should be possibly to sell people on the ethical implications of brain-based computing...
I think in general if we agree to a ballpark of "10% donations is enough to satisfice some goodness thresholds", and also to "It would be good for social pressure to exist for everyone to do at least threshold amount of good", I think it raises various considerations.
10% makes sense to me as a schelling point (and I think the tables that scale by income bracket are also sensible).
But if the threshold amount of good would be "Donate 10%, aim for an impactful career, become vegan" (which is what I feel the social pressure inside EA is pointing towards), I think that is already a significant ask for many people.
I think it is also important to note that some people are more motivated by trying to maximize impact and offset harm, and some people more motivated by minimizing harm and satisficing for impact. (Of course a standard total utilitarian model would output that whatever maximizes your net impact is best, but human value systems aren't perfectly utilitarian.)
How do "donate 10%, become vegan, aim for an impactful", and "donate 30%", and "donate 20%, aim for an impactful career" compare in effectiveness as norms? I think this is pretty hard to estimate.
What kind of social pressure are you pointing here? Is it more in the direction of "donate 30%" or "donate as much as you can and aim for an impactful career?" Or do you mean social pressure in the wider society, and not within the EA community?
(Fwiw I think people underestimate the value of effective marginal spending on themselves, when considering areas of spending where there is space for significant extra value (Like purchasing more free time.). People plausibly overestimate the value on some other things, especially if one doesn't do spending introspectiont.)
I find it intuitive that there could be a small set of objective moral facts but much smaller than is generally believed for moral realist positions + i do not think this can be justified rationally to a large degree. I think there can be contextual moral facts (as in "rational agents in a society would agree to cooperate on problem X" or "rational agents would agree to behave in a certain way on a moral problem, given following constraints"), but I do not think these are enough to justify an objective moral realism position.
I think the set of sensible moral views and positions is large, and thus think that morality is mostly not objective.
I don't know if this will be a useful comment but putting it here anyway. Personally, and most other people who feel like they are being judged too much or have too much performance pressure in EA, often do it themselves. I think there is a causation in how EA material and dynamics can facilitate people being more self-critical than is healthy, and to me that seems like a much more common problem than being judged by other EAs. (Didn't notice a survey question that would measure the thing I'm trying to point at here.)
The role of guilt and perfectionism in EA, and how EA as an environment focused on efficiency and doing the very best we can can lead to difficult mental hangups, and be more demanding than more traditional ways of doing good. (Traditional ways of doing good are often more focused in feeling good and feeling altruistic, which is useful for the good-doers wellbeing but suboptimal for actual amount of good done.)
Positivity focused ethics! The inbalance between negativity-biased vs. positively biased versions of utilitarianism, and implications of this on evaluating policy ideas and the medium-term future.
Any hints / info on what to look for in a mentor / how to find one? (Specifically for community building.)
I'm starting as a national group director in september, and among my focus topics for EAG London are group-focused things like "figuring out pointers / out of the box ideas / well-working ideas we haven't tried yet for our future strategy", but also trying to find a mentor.
These were some thoughts I came up with when thinking about this yesterday:
- I'm not looking for accountability or day to day support. I get that from inside our local group.
- I am looking for someone that can take a description of the higher level situation and see different things than I can. Either due to perspective differences or being more experienced and skilled.
- Also someone who can give me useful input on what skills to focus on building in the medium term.
- Someone whose skills and experience I trust, and when they say "plan looks good" it gives me confidence, when I'm trying to do something that feels to me like a long shot / weird / difficult plan and I specifically need validation that it makes sense.
On a concrete level I'm looking for someone to have ~monthly 1-1 calls with and some asynchronous communication. Not about common day to day stuff but larger calls.
Same, I only had ~800 mana free but wouldn't have realized to donate it otherwise, and it only took a minute.