Comments
1) What careers that directly contribute to AI alignment should someone consider who is not likely suited as a researcher in fields like math or decision theory? 2) Which first steps are recommendable to end up in such a career?
More from the author
Curated and popular this week
63
AI Use Note: Main body text entirely human written. Claude (Opus 4.8) helped develop models of animal life histories in the appendix.
Cross-posted from Good Structures.
Executive Summary
* Animal advocates sometimes make claims like “there are X of this animal...
“How long have you been v*g*n?”
This is one of the most common icebreakers at animal protection events. It’s a baseline assumption, and it mostly holds true: if you’re out advocating for animals not to be tortured or abused, realistically these days you are v**n, or close. And it makes for good conversation. It seems fairly safe to assume when you meet strangers.
But this assumption is hurting the movement in a way which we don’t always notice: someone new comes into the sp...
Summary
Back in November 2023 I posted here to launch Spiro and raise our first $198k. Two and a half years later this is an update and a fundraiser for the next step.
The short version: we've now reached over-5,900 people with TB preventive medicine, including over 3,000 children under five years old. Our early results have held up well an...
Recent opportunities to take action
Here are some questions of mine. I haven't done a ton to follow discussions of AI safety, which means my questions will either be embarrassingly naive or will offer a critical outside perspective. Please don't use any that fit the former case :)
It seems like there's a decent chance that whole brain emulations will come before de novo AI. Is there any "friendly WBE" work it makes sense to do to prepare for this case, analogous to "friendly AI" work?
Around the time AGI comes in to existence, it's important how cheap and fast the hardware available to run it on is. If hardware is relatively expensive and slow, we can anticipate a slower (and presumably more graceful) transition. Is there anything we can do to nudge the hardware industry away from developing ever-faster chips, so hardware will be relatively expensive and slow at the time of the transition? For example, Musk could try to hire away researchers at semiconductor firms to work on batteries or rocket ships, but this could only be a temporary solution: the wages for such researchers would rise in response to the shortage, likely leading to more students going in to semiconductor research. (Hiring away professors that teach semiconductor research might be a better idea, assuming American companies are bad at training employees.)
In this essay, I wrote: "At some point our AGI will be just as smart as the world's AI researchers, but we can hardly expect to start seeing super-fast AI progress at that point, because the world's AI researchers haven't produced super-fast AI progress." I still haven't seen a persuasive refutation of this position (though I haven't looked very hard). So: Given that human AI researchers haven't produced a FOOM, is there any reason to expect that an AI equal to the level of the human AI research community would produce a FOOM? (EDIT: A better framing might be whether or not we can expect chunky, important AI insights to be discovered in the future, or whether AGI will come out of many small cumulative developments. I suppose the idea that the brain implements a single cortical algorithm should push us in the direction of believing there is at least one chunky undiscovered insight?)