AI governance fellow @ Pivotal Research
2228 karmaJoined Working (0-5 years)Oxford, UK



I'm currently researching how to govern AI-driven explosive technological growth via a summer research fellowship with Pivotal Research.

Previously, I worked on executive support and events at the Centre for Effective Altruism (CEA). I also scaled up the EA Opportunity Board, interned at Global Challenges Project, and founded the EA student group at University of Wisconsin–Madison, where I studied Neurobiology and Psychology. 

If you think we share an interest (we probably do!), don't hesitate to reach out :)


Meta Coordination Forum 2023


Topic contributions

Just an FYI that most non-profits are often legally constrained on doing many sorts of political advocacy work, I think. 

National securitisation privileges extraordinary measures to defend the nation, often centred around military force and logics of deterrence/balance of power and defence. Humanity macrosecuritization suggests the object of security is to defend all of humanity, not just the nation, and often invokes logics of collaboration, mutual restraint and constraints on sovereignty.

I found this distinction really helpful. 

It reminds me of Holden Karnofsky's piece on How to make the best of the most important century (2021), in which he presents two contrasting frames: 

  • The "Caution" frame. In this frame, many of the worst outcomes come from developing something like PASTA in a way that is too fast, rushed, or reckless. We may need to achieve (possibly global) coordination in order to mitigate pressures to race, and take appropriate care. (Caution)
  • The "Competition" frame. This frame focuses not on how and when PASTA is developed, but who (which governments, which companies, etc.) is first in line to benefit from the resulting productivity explosion. (Competition)
  • People who take the "caution" frame and people who take the "competition" frame often favor very different, even contradictory actions. Actions that look important to people in one frame often look actively harmful to people in the other.
    • I worry that the "competition" frame will be overrated by default, and discuss why below. (More)

FYI, I was making a difficult career decision a few months ago and found this post helpful. Thanks for writing it!

Love it.

Your discussion of how ‘Green’ vibes can fall short of really looking consequences in the face reminds of a quote:

“All that blood was never once beautiful, it was always just red.” - Kait Rokowski

Thanks for sharing this! I've really been liking your forum posts recently :) 

Oh sorry, I missed this! I should have read that more closely before commenting.

Fair enough. Interesting to see people's different intuitions on this. 

Registering that this line of questioning (and volume of questions) strikes me as a bit off-putting/ too intense. 

If someone asked about what "What were the key concerns here, and how were they discussed?" [...] "what questions did you ask, and what were the key considerations/evidence?" about interactions I had years ago, I would feel like they're holding me to an unrealistic standard of memory or documentation.  

(Although I do acknowledge the mood that these were some really important interactions. Scrutiny is an appropriate reaction, but I still find this off-putting.) 

the EA community is not on track to learn the relevant lessons from its relationship with FTX. 

In case it helps, here’s some data from Meta Coordination Forum attendees on how much they think the FTX collapse should influence their work-related actions and how much it has influenced their work-related actions:

  • On average, attendees thought the average MCF attendee should moderately change their work-related actions because of the FTX collapse (Mean of 4.0 where 1 = no change and 7 = very significant change; n = 39 and SD = 1.5)
  • On average, attendees reported that the FTX collapse had moderately influenced their work related actions (Mean of 4.2 where 1 = no change and 7 = very significant change; n = 39 and SD = 1.7)

My interpretation of this is that MCF attendees have changed their professional behavior a reasonable amount (according to MCF attendees), although maybe this doesn’t address broader questions of reform (eg., ecosystem-wide work that requires substantial coordination). 

And here’s a summary of responses to the question “what lessons do we need to learn from the past year”, asked directly after the above question:  

  1. Improve governance and organizational structures (mentioned by 7 respondents):
    • Shore up governance, diversify funding sources, build more robust whistleblower systems, and have more decentralized systems in order to be less reliant on key organizations/people.
  2. Build crisis response capabilities (mentioned by 6 respondents):
    • Create crisis response teams, do crisis scenario planning, have playbooks for crisis communication, and empower leaders to coordinate crisis response.
  3. Improve vetting and oversight of leaders (mentioned by 5 respondents):
    • Better vet risks from funders/leaders, have lower tolerance for bad behavior, and remove people responsible for the crisis from leadership roles.
  4. Diversify and bolster communication capacities (mentioned by 5 respondents):
    • Invest more in communications for crises, improve early warning/information sharing, and platform diverse voices as EA representatives.
  5. Increase skepticism and diligence about potential harms (mentioned by 4 respondents):
    • Adopt lower default trust, consult experts sooner, and avoid groupthink and overconfidence in leaders.
  6. Learn about human factors in crises (mentioned by 3 respondents):
    • Recognize the effect of stress on behavior, and be aware of problems with unilateral action and the tendency not to solve collective action problems.
  7. Adopt more resilient mindsets and principles (mentioned by 3 respondents):
    • Value integrity and humility, promote diverse virtues rather than specific people, and update strongly against naive consequentialism.

I thought that this might be relevant when discussing how much or little has been done post-FTX.

My personal guess is that public discussions on the Forum under-represent changes to org policies, institutional norms, and fuzzy updates about who to trust how much.

(I work at CEA, so I could be very biased.)

Thanks! It's something very similar to the 'responsibility assignment matrix' (RACI) popularized by consultants, I think. But in this case BIRD is more about decisions (rather than tasks) and stands for Bound (set guidelines), Input (give advice), Responsible (do the bulk of thinking decision through and laying out reasoning), and Decider (make decision). 

Load more