Forethought[1] is a new AI macrostrategy research group cofounded by Max Dalton, Will MacAskill, Tom Davidson, and Amrit Sidhu-Brar.
We are trying to figure out how to navigate the (potentially rapid) transition to a world with superintelligent AI systems. We aim to tackle the most important questions we can find, unrestricted by the current Overton window.
More details on our website.
Why we exist
We think that AGI might come soon (say, modal timelines to mostly-automated AI R&D in the next 2-8 years), and might significantly accelerate technological progress, leading to many different challenges. We don’t yet have a good understanding of what this change might look like or how to navigate it. Society is not prepared.
Moreover, we want the world to not just avoid catastrophe: we want to reach a really great future. We think about what this might be like (incorporating moral uncertainty), and what we can do, now, to build towards a good future.
Like all projects, this started out with a plethora of Google docs. We ran a series of seminars to explore the ideas further, and that cascaded into an organization.
This area of work feels to us like the early days of EA: we’re exploring unusual, neglected ideas, and finding research progress surprisingly tractable. And while we start out with (literally) galaxy-brained schemes, they often ground out into fairly specific and concrete ideas about what should happen next. Of course, we’re bringing principles like scope sensitivity, impartiality, etc to our thinking, and we think that these issues urgently need more morally dedicated and thoughtful people working on them.
Research
Research agendas
We are currently pursuing the following perspectives:
* Preparing for the intelligence explosion: If AI drives explosive growth there will be an enormous number of challenges we have to face. In addition to misalignment risk and biorisk, this potentially includes: how to govern the development of new weapons of mass destr
With another EAG nearby, I thought now would be a good time to push out this draft-y note. I'm sure I'm missing a mountain of nuance, but I stand by the main messages:
"Keep Talking"
I think there are two things EAs could be doing more of, on the margin. They are cheap, easy, and have the potential to unlock value in unsuspecting ways.
Talk to more people
I say this 15 times a week. It's the most no-brainer thing I can think of, with a ridiculously low barrier to entry; it's usually net-positive for one while often only drawing on unproductive hours of the other. Almost nobody would be where they were without the conversations they had. Some anecdotes:
- A conversation led both parties discovering a good mentor-mentee fit, leading to one dropping out of a PhD, being mentored on a project, and becoming an alignment researcher.
- A first conversation led to more conversations which led to more conversations, one of which illuminated a new route to impact which this person was a tremendously good fit for. They're now working as a congressional staffer.
- A chat with a former employee gave an applicant insight about a company they were interviewing with and helped them land the job (many, many such cases).
- A group that is running a valuable fellowship programme germinated from a conversation between three folks who previously were unacquainted (the founders) (again, many such cases).
Make more introductions to others (or at least suggest who they should reach out to)
By hoarding our social capital we might leave ungodly amounts of value on the table. Develop your instincts and learn to trust them! Put people you speak with in touch with other people who they should speak with -- especially if they're earlier in their discovery of using evidence and reason to do more good in the world. (By all means, be protective of those whose time is 2 OOMs more precious; but within +/- 1, let's get more people connected: exchanging ideas, improving our thinking, illuminating truth, building trust.)
At EAG, at the very least, point people to others they should be talking to. The effort in doing so is so, so low, and the benefits could be massive.
Edit: Punctuation
One habit to make that second piece of advice stick even more that I often recommend: introduce people to other people as soon as you think of it (i.e. pause the conversation and send them an email address or list of names or open a thread between the two people).
I often pause 1:1s to find links or send someone a message because I'm prone to forgetting to do follow-up actions unless I immediately do it (or write it down).