Concerns about AI safety career change

(context: I've recently started as a research engineer (RE) on DeepMind's alignment team. All opinions are my own)

Hi, first off, it's really amazing that you are looking into changing your career to help reduce x-risks from AI.

I'll give my perspective on your questions.

1.

a. All of this is on a spectrum but.. There is front-end engineering, which to my knowledge is mostly used to build human-feedback-interfaces or general dialogue chats like ChatGPT.

Then there's research engineering, which I'd roughly sort into two categories. One is more low-level machine learning engineering, like ensuring that you can train, query, serve (large) models, making your ML code more efficient or making certain types of analyses feasible in the first place. This one seems pretty crucial to a lot of orgs and is in especially high demand afaict. The second is more research-y, analysing existing models, or training/fine-tuning models in order to testing a hypothesis that's somehow related to safety. Again those exist on a spectrum.

c. In my work, I need to use quite a bit of ML/DL knowledge regularly and it was expected and tested for in the application process (I doubt that that would be the case for front-end engineering roles though). ML theory is basically never used in my work. I think this is similar across orgs who are working on "prosaic alignment", i.e. directly working with deep learning models, although I could be missing some cases here.

2.

... why are there so many different organizations?

Different beliefs about theory of change and threat models, but I'd say also more money and interested researchers than management capacity. Let's say there are 100 qualified people who would like to work on team X, chances are team X can only absorb a handful per year without imploding. What are those 100 people going to do? Either start their own org, work independently or try to find some other team.

a. That's a super valid point. All these organizations state to aim to reduce AI x-risk. As I see it they mainly differ along the axes "threat/development model" (how will dangerous AI be built and how is it causing x-risk?) and "theory of change" (how can we reduce the x-risk?). Of course there is still substantial overlap between orgs.

b. i. MIRI has chosen to follow a "not-publishing-by-default" policy, explaining some of the lack of publications. Afaict, MIRI still operates and can best be modeled as a bunch of semi-independent researchers (although I have especially little knowledge here)

c. For a slightly outdated overview you could checkout Larks' review from 2021

d. This seems like an accurate description, not sure what exactly you would like more clarity on. The field is in fact quite fragmented in that regard. Regarding the private companies (in particular OpenAI, DeepMind, Anthropic): They were founded with more or less focus on safety but all of them did have the safety of AI systems as an important part of their founding DNA.

3. I don't feel qualified to say much here. My impression is that frontend comes mostly into play when gathering human feedback/rating which is important for some alignment schemes but not others.

4. I'm not aware about groups in Asia. Regarding Europe there is DeepMind (with some work on technical safety and some work on AI governance) and Conjecture, both based in London. I think there are academic groups working on alignment as well, most notably David Krueger's group at Cambridge and Jan Kulveit's group in Prague. I'm probably forgetting someone here.

The reason everything is very confusing is because it is a fast growing field, with lots of people doing lots of things. As someone already pointed out, different orgs often have the same end en goal (i.e. reducing X-risk form AI) but different ideas for how to do this. 

But this is not even the reason there are several orgs. Orgs are just legal entities to hire people to do the work. One reason to have several orgs is that there are researchers in more than one country and it's easier to organise this under different orgs. Another reason is that different orgs have different funding models, or leadership styles, etc. 

But also, most orgs don't grow very fast, probably for good reasons, but I don't know this is just an imperial observation. This means there are lots of researchers wanting to help, and some of them get funding, and some of them decide to start new orgs.

So we end up with this mess of lots of orgs doing their own thing, and no one really knows everything that is going on. This has some cost, e.g. there are probably people doing almost the same research with out knowing of each other. And as you say, it is confusing, especially when you are new. But I rather have this mess than a well ordered centrally controlled research ecosystem. Central coordination might seem good in theory, but in practice it not worth it. A centralised system is slow and can't spot its own blind spots. 

So what to do? How to navigate this mess?

There are some people who are creating information resources which might be helpful. 

There's AI Safety Supports lots of links page, which is both too long and to abbreviated, because making a good list is hard. 
AI Safety Support - Lots of Links

Alignment Ecosystem are working on some better resources, but it's all still under construction.
Other resources · Alignment Ecosystem Development (coda.io)

Currently I think the best way to get oriented is to talk so someone who is more acquainted with the AI Safety career landscape than you. Either someone you know, or book a call with AI Safety Support.
AI Safety Support - Career Coaching

Given the similarities, why are there so many different organizations? How is an outsider >supposed to know what makes each of them unique? ... What makes these different from each other?

You are not the first person to ask questions like this. And I feel like I often hear people trying to make databases of researchers and orgs or give overviews of the field etc. etc. But the situation you describe just seems...normal, to me. As in, most industries and areas, even pretty niche things will have many different organizations with slightly different outlooks and no central way to navigate them and learn about them. And most research fields in academia are like this; there isn't a great way to get to know everyone in the field, where they work, what they work on etc, it just takes years of talking to people, reading things, meeting them at conferences etc. and you just slowly build up the picture .

I don't think it's silly or bad to ask what you're asking, that's not what I'm saying/I may well be wrong anyway, but to my mind the situation seems like it wouldn't necessarily have a good 'explanation' that someone can just hand to you, and the usual way to find out the sorts of things you want to find out would be to just continue immersing yourself in the field (from what I hear from most people who start figuring out this field, ~15 hours is not very long).

Context: I work as an alignment researcher. Mostly with language models.

I consider myself very risk-averse, but I also personally struggled (and still do) with the instability of alignment. There’s just so many things I feel like I’m sacrificing to work in the field and being an independent researcher right now feels so shaky. That said, I’ve weighed the pros and cons and still feel like it’s worth it for me. This was only something I truly felt in my bones a few months after taking the leap. It was in the back of my mind for ~5 years (and two 80k coaching calls) before I decided to try it out.

With respect to your engineering skills, I’m going to start to work on tools that are explicitly designed for alignment researchers (https://www.lesswrong.com/posts/a2io2mcxTWS4mxodF/results-from-a-survey-on-tool-use-and-workflows-in-alignment) and having designers and programmers (web devs) would probably be highly beneficial. Unfortunately, I only have funding for myself for the time being. But it would be great to have some people who want to contribute. I’d consider doing AI Safety mentorship as a work trade.

I honestly feel like software devs could probably still keep their high-paying jobs and just donate a bit of time and expertise to help independent researchers if they want to start contributing to AI Safety.

Great questions. 
On question 4, I don't personally know of any groups based in Asia, but feel free to check out this database of AI Safety relevant communities, and join any of them. 

After looking into other resources, it seems like this is the best overview of what everyone in the field is doing: 

https://www.alignmentforum.org/posts/QBAjndPuFbhEXKcCr/my-understanding-of-what-everyone-in-technical-alignment-is

Both OpenAI and DeepMind are hiring for software engineers right now, as per their website!

Hello all,

There might currently be a security breach on EAforum. Please do not share your contact information or that of anyone you know until the situation is resolved, tomorrow at the earliest.