JamesN's Quick takes

We do a monthly public forecast at the Swift Centre (swiftcentre.org) - just a chance for us to make predictions and insights on some topical events.

As it’s EAG London month I thought I’d ask what topics/questions are on people’s mind so we can potentially throw them into the mix.

Comment below!

Mental health support for those working on AI risks and policy?

During the numerous projects I work on relating to AI risks, policies, and future threats/scenarios, I speak to a lot of people who bring exposed to issues of catastrophic and existential nature for the first time (or grappling with them for the first time in detail). This combined with the likelihood that things will get worse before they better, makes me frequently wonder: are we doing enough around mental health support?

Things that I don’t know exist but feel they should. Some may sound OTT but I expect you could fund all of these for c.$300k, which relative to the amount being spent in the sector as a whole, is tiny in exchange for resilience of the talent we’re building. 

• Structured proactive therapy or mental health resilience sessions tied into Fellowship programs.
• Regular in-built mental health support within organisations dealing with AI risks etc., particularly around helping to anti-catastrophise (there are several threat models which are only catastrophically bad if many, many, many things go down and happen counter to regular incentives - but are very worrying to people (especially new entrants to the field) - it feels support to help prioritise the risks would help both outcomes and mental health.
• Free to use services for mental health support for those in the field.

I'm considering my future donation options, either directly to charities or through a fund. I know EA Funds is still somewhat cash constrained but I'm also a little concerned with the natural variance in grant quality. 

I'd be interested in why others have or have not chosen to donate to EA Funds, and if so, would they again in the future?

I respect people may prefer to answer this by DM, so please do feel free to drop me a message there if posting here feels uncomfortable.

Much of the community’s focus is rightly on technical alignment and governance. However, there seems to be a significant blind spot regarding societal adaptation, specifically, how we raise and educate the next generation.

Our current education model is predicated on a learn skills to provide economic value loop. When transformative AI disrupts this model, we risk creating a generation that is not only economically displaced but fundamentally disenfranchised and without a clear sense of purpose. Historically, large populations of disenfranchised young people have been a primary driver of societal collapse and political volatility.

If the transition to a post-AGI world is chaotic due to human unrest, our ability to manage technical safety drops significantly. Is anyone seriously funding or working on how education/raising children needs to change to fit with an AGI era? It seems like ensuring the next generation is psychologically and philosophically prepared for a world of transformative AI is a necessary prerequisite for a stable transition.

Do EA funding organisations have contingency plans (I use the term loosely) to manage downside risks to cash-flow in the eventuality of a recession? With the possibility of a recession in the coming 6-18 months seemingly increasing, it would be prudent to model the scenarios and think through how to smooth the funding profile if needed. Given cash is already constrained post the collapse of "that will shall not be named", it feels a recession (and thus implications to donations/value of investments etc.) could compound existing funding issues. I don't expect to get a financial plan from any of the big funders, but I thought worth raising the question in the low probability scenario it isn't already part of their modelling.

What are some of the best (relatively) small podcasts on AI and/or policy that people would recommend? I know of all the big ones, but keen to see if there are any nascent ones that are worth sharing with others.

I'm interesting in chatting to any civil servants, ideally in the UK, who are keen on improving decision making in their teams/area - potentially through forecasting techniques and similar methods. If you'd be interested in chatting, please DM me!

As my toddler continues to grow, my wife and I are reaching the point that all parents do (if not already done) where we have to decide what to do about his education. Obviously education and life aren't fixed so there is always a necessary amount of flexibility to any decision. 

My issue (UK focus): Broadly there are three stages in the UK - 1) nursery, 2) primary school, 3) secondary/high school. With the world becoming increasingly uncertain and complex, I have very little confidence that any current institution (in stage 2 and 3) is set up and run well enough to prepare a child for the future - beyond the basic reading and writing skills. As a small example for instance, there is a shocking lack of investment in both resources and capability on key topics such as computer science, maths, engineering, rationality, and political science. However, given this is a quick post I won't go into the depth of my reasoning of why I feel there is a structural problem with the education system that would be almost impractical (or at least, too politically costly) to resolve. To add some potential credence, this isn't just wandering thoughts but they have been generated through the experiences of multiple family (including my wife) and friends who work/worked in primary or secondary schools.

My question: What are others' thoughts on this (both parents and non-parents)? Do you have similar concerns? Do you not? If you do, what are your intended actions to mitigate?

I'll be in D.C. on the 12th February and morning of the 13th, before heading to EAG. If people are around and want to meet, feel free to drop me a DM!

With a number of charity evaluators recommendations coming out over the last few days/weeks, has there been any further development on AI safety/GCR evaluator(s)? A need that was raised in the post below (I don't know if best EA forum practice is to ressurrect an old thread or not, so I apologies if it's better I just comment in there).

https://forum.effectivealtruism.org/posts/QahDdRzpPuat8ujx2/why-isn-t-there-a-charity-evaluator-for-longtermist-projects