Comments
Funding for work that builds capacity to address risks from transformative AI
6 min read · Aug 13, 2024
40
Post authors: Eli Rose, Asya Bergal
Posting in our capacities as members of Open Philanthropy’s Global Catastrophic Risks Capacity Building team.
This program, together with our separate program for programs and events on global catastrophic risk, effective altruism, and other topics, has replaced our 2021 request for proposals for outreach projects. If you have a project which was in-scope for that program but isn't for either of these, you can apply to our team's general application instead.
We think it's possible that the coming decades will see "transformative" progress in artificial intelligence, i.e., progress that leads to changes in human civilization at least as large as those brought on by the Agricultural and Industrial Revolutions. It is currently unclear to us whether these changes will go well or poorly, and we think that people today can do meaningful work to increase the likelihood of positive outcomes.
To that end, we're interested in funding projects that:
We refer to this category of work as “capacity-building”, in the sense of “building society’s capacity” to navigate these risks. Types of work we’ve historically funded include training and mentorship programs, events, groups, and resources (e.g., blog posts, magazines, podcasts, videos), but we are interested in receiving applications for any capacity-building projects aimed at risks from advanced AI. This includes applications from both organizations and individuals, and includes both full-time and part-time projects.
Apply for funding here. Applications are open until further notice and will be assessed on a rolling basis.
We’re interested in funding work to build capacity in a number of different fields which we think may be important for navigating transformative AI, including (but very much not limited to) technical alignment research, model evaluation and forecasting, AI governance and policy, information security, and research on the economics of AI.
This program is not primarily intended to fund direct work in these fields, though we expect many grants to have both direct and capacity-building components — see below for more discussion.
These are programs that teach relevant skills or understanding, offer mentorship or professional connections for those new to a field, or provide career opportunities. These could take the form of fellowships, internships, residencies, visitor programs, online or in-person courses, bootcamps, etc.
Some examples of training and mentorship programs we’ve funded in the past:
We've previously funded a number of such programs in technical alignment research, and we’re excited to fund new programs in this area. But we think other relevant areas may be relatively neglected — for instance, programs focusing on compute governance or on information security for frontier AI models.
For illustration, here are some (hypothetical) examples of programs we could be interested in funding:
We’re interested in funding a wide variety of different events related to risks from advanced AI, including:
Historically we haven’t funded many events of this type, but one example is the 2023 NOLA Alignment Workshop hosted by our grantee FAR.AI.
We’re especially interested in supporting events that:
For illustration, here are some (hypothetical) examples of events we could be interested in funding:
We’re interested in funding regular meetups that bring people together based on an interest in helping society navigate transformative AI and reduce its risks. These could be in-person or online meetup groups, bringing people together based on a location (e.g., AI Safety Tokyo), a profession, or another shared affinity. Historically, we’ve seen groups run discussions, put on events, and embark on learning and professional development projects.
For groups at universities, please apply through this page instead.
Some (hypothetical) examples:
This category includes written content (e.g., books, magazines, newsletters, websites, online explainers), video (e.g., documentaries, YouTube videos) or audio (e.g., podcasts). We’re interested in work that informs and provides thoughtful evidence-based discussion of transformative AI, its risks, or related topics. This work might be for people with a specific educational goal in mind (e.g. tutorials, curricula, or online guides), or it might be lighter and more casual.
We're interested in supporting both projects that cover new ground or go in-depth on topics with shallow existing coverage, and those that present well-covered subjects to new audiences or in more accessible ways.
Some (hypothetical) examples:
In addition to the categories above, we're interested in receiving funding applications for almost any type of project that builds capacity for advanced AI risks — in the sense of increasing the number of careers devoted to these problems, supporting people doing this work, and sharing knowledge related to this work.
This includes:
We're interested in funding a wide range of proposals, ranging from small grants to individuals, to multi-year grants to organizations. If you're unsure whether your proposal is in-scope for this program, we encourage you to err on the side of applying.
This program is focused on capacity-building work rather than “object-level” work, e.g., technical research. For example, we’re unlikely to provide generalized funding for an academic lab’s research through this program, though we may fund training or mentorship programs, events, or other capacity-building work hosted by the lab. However, the line can be fuzzy: if you’re in doubt about where your application falls, you can contact us at [email protected] or just submit it anyway.
Some applications may be in-scope for both this RFP and another at Open Philanthropy (in particular our AI governance and policy RFP). If your proposal would be in-scope for another team’s RFP, we prefer that you apply to the other RFP rather than this one. And in general if we think your application is a better fit for another team’s funding program, we may redirect your application to them (we’ll let you know if we do this).
Apply for funding here.
The application form asks for information about you, your project/organization (if relevant), and the activities you're requesting funding for.
We expect to make most funding decisions within 8 weeks of receiving an application (assuming prompt responses to any follow-up questions we may have). You can indicate on our form if you'd like a more timely decision, though we may or may not be able to accommodate your request.
Applications are open until further notice and will be assessed on a rolling basis.
Executive summary: Open Philanthropy is offering funding for projects that build capacity to address risks from transformative AI, including training programs, events, groups, and resources aimed at developing talent and discourse in this field.
Key points:
This comment was auto-generated by the EA Forum Team. Feel free to point out issues with this summary by replying to the comment, and contact us if you have feedback.