Hide table of contents

Long TL;DR: You’re an engineer, you want to work on AI Safety, you’re not sure which org to apply to, so you’re going to apply to all of them. But - oh no - some of these orgs may actively be causing harm, and you don’t want to do that. What’s your alternative? Study AI Safety for 2 years before you apply? In this post I suggest you can collect the info you want quickly by going over specific posts

Why do I think some orgs might be [not helping] or [actively causing harm]?

Example link. (Help me out in the comments with more?)

My suggestion:

1. Open the tag of the org you want on lesswrong

How: Search for a post related to that org. You’ll have tags on top of the post. Click the tag with the org name.

2. Sort by “newest first״

3. Open 2-3 posts

(Don’t read the post yet!)

4. In each post, look at the top 2-3 most upvoted comments

What I expect you’ll find sometimes

A post by the org, with comments trying to politely say “this is not safe”, heavily upvoted.

Bonus: Read the comments

Or even crazier: Read the post! [David Johnson thinks this is a must!]

Ok, less jokingly, this seems to me like a friendly way to start to see the main arguments without having to read too much background material (unless you find, for example, a term you don’t know).

Extra crazy optimization: Interview before research

TL;DR: First apply to lots of orgs, and then, when you know which orgs are interested[1], then do your[2] research only those orgs.

Am I saying this idea for vetting AI Safety orgs is perfect?

No, I am saying it is better than the alternative of “apply to all of them (and do no research)”, assuming you resonate with my premise of “there’s a lot of variance in effectiveness of orgs” and “that matters”.

I also hope that by posting my idea, someone will comment with something even better.

  1. ^

    However you choose to define "interested". Maybe research the orgs that didn't reject your CV? Maybe only research the ones that accepted you? Your call

  2. ^

    Consider sharing your thoughts with the org. Just remember, whoever is talking to you was chosen as a person that convinces candidates to join. They will, of course, think their org is great. Beware of reasons like "the people saying we are causing harm are wrong, but we didn't explain publicly why". The whole point is letting the community help you with this complicated question.

17

0
0

Reactions

0
0
Comments21


Sorted by Click to highlight new comments since:

I don’t endorse judging AI safety organisations by less wrong consensus alone - I think you should at least read the posts!

Thanks for the push back!

Added this to the post

I think this is fairly bad advice - LessWrong commenters are wrong about a lot of things. I think this is an acceptable way to get a vibe for the what the LessWrong bubble thinks though. But idk, for most of these questions the hard part is figuring out which bubble to believe. Most orgs will have some groups think they're useless, some think they're great, and probably some who think they're net negative. Finding one bubble who believes one of these three doesn't tell you much!

Thanks for the pushback!

Do you have an alternative suggestion?

I personally interpret Neel's comment as saying this is ~not better (perhaps worse) than going in blindly. So I just wanted to highlight that a better alternative is not needed for the sake of arguing this (even if it's a good idea to have one for the sake of future AI researchers).

Do you think that going to do capabilities work at DeepMind or OpenAI is just as impactful as going to whatever the lesswrong community recommends (as presented by their comments and upvotes) ?

Possibly. As we've discussed privately, I think some AI safety groups which are usually lauded are actually net negative 🙃

But I was trying to interpret Neel and not give my own opinion.

My meta-opinion is that it would be better to see what others think about working on capabilities in top labs, compared to going there without even considering the downsides. What do you think? (A)

And also that before working at "AI safety groups which are usually lauded [but] are actually net negative", it would be better to read comments of people like you. What do you think? (B)

I somewhat disagree with both statements.

(A) Sure, it'd be good to have opinions from relevant people, but on the other hand it's non-trivial to figure out who "relevant people" are, and "the general opinion on LW" is probably not the right category. I'd look more at what (1) people actually working in the field, and (2) the broad ML community, think about an org. So maybe the Alignment Forum.

(B) I can only answer on my specific views. My opinion on [MIRI] probably wouldn't really help individuals seeking to work there, since they probably know everything I know and have their own opinions. My opinions are more suitable for discussions on the general AI safety community culture.

By the way, I personally resonate with your advice on forming an inside view and am taking that path, but it doesn't fit everyone. Some people don't want all that homework, they want to get in a company and write code, and, to be clear, it is common for them to apply to all orgs that [they see their names in EA spaces] or something like that (very wide, many orgs). This is the target audience I'm trying to help.

I would just probably tell people to work in another field than explicitly encouraging goodharting their way to trying to having positive impact in an area with extreme variance.

Thinking about where to work seems reasonable, listening to others' thoughts on where to work seems reasonable, this post advises both.

This post also pretty strongly suggests that lesswrong comments are the best choice of others' thoughts, and I would like to see that claim made explicit and then argued for rather than slipped in. As a couple of other comments have noted, lesswrong is far from a perfect signal of the alignment space.

Thanks (also) for the pushback part!

Do you have an alternative to lesswrong comments that you'd suggest?

Seems like there is a big gap between "Study AI Safety for 2 years before you apply" and reading posts, rather than just the most up-voted comments.

Other feedback: I don't understand why you call some of your suggestions "crazy"/"crazier". Also when you wrote "less joklingly" I had missed the joke. Perhaps your suggestions could be rewritten to be more clear without these words.

The joke is supposed to be that "reading the post" isn't actually that crazy. I see this wasn't understood (oops!). I'm going to start by trying to get the content right (since lots of people pushed back on it) and then try fixing this too

I didn't understand, could you say this in other words please?

Or did you mean it like this:

Seems like there is a big gap between ["Study AI Safety for 2 years before you apply" and reading posts], rather than [just the most up-voted comments].

?

I meant it like that, yes. Seems like there is a big gap between ["Study AI Safety for 2 years before you apply" and reading posts].

So what would you suggest? Reading a few posts about that org?

Seems worth asking in interviews "I'm concerned about advancing capabilities and shortening timelines, what actions is your organization taking to prevent that", with the caveat that you will be BSed.

Bonus: You can turn down roles explicitly because they're doing capabilities work, which if it becomes a pattern may incentivize them to change their plan.

I agree, see foot note 2

Curated and popular this week
 ·  · 22m read
 · 
The cause prioritization landscape in EA is changing. Prominent groups have shut down, others have been founded, and everyone’s trying to figure out how to prepare for AI. This is the third in a series of posts critically examining the state of cause prioritization and strategies for moving forward. Executive Summary * An increasingly common argument is that we should prioritize work in AI over work in other cause areas (e.g. farmed animal welfare, reducing nuclear risks) because the impending AI revolution undermines the value of working in those other areas. * We consider three versions of the argument: * Aligned superintelligent AI will solve many of the problems that we currently face in other cause areas. * Misaligned AI will be so disastrous that none of the existing problems will matter because we’ll all be dead or worse. * AI will be so disruptive that our current theories of change will all be obsolete, so the best thing to do is wait, build resources, and reformulate plans until after the AI revolution. * We identify some key cruxes of these arguments, and present reasons to be skeptical of them. A more direct case needs to be made for these cruxes before we rely on them in making important cause prioritization decisions. * Even on short timelines, the AI transition may be a protracted and patchy process, leaving many opportunities to act on longer timelines. * Work in other cause areas will often make essential contributions to the AI transition going well. * Projects that require cultural, social, and legal changes for success, and projects where opposing sides will both benefit from AI, will be more resistant to being solved by AI. * Many of the reasons why AI might undermine projects in other cause areas (e.g. its unpredictable and destabilizing effects) would seem to undermine lots of work on AI as well. * While an impending AI revolution should affect how we approach and prioritize non-AI (and AI) projects, doing this wisel
 ·  · 9m read
 · 
This is Part 1 of a multi-part series, shared as part of Career Conversations Week. The views expressed here are my own and don't reflect those of my employer. TL;DR: Building an EA-aligned career starting from an LMIC comes with specific challenges that shaped how I think about career planning, especially around constraints: * Everyone has their own "passport"—some structural limitation that affects their career more than their abilities. The key is recognizing these constraints exist for everyone, just in different forms. Reframing these from "unfair barriers" to "data about my specific career path" has helped me a lot. * When pursuing an ideal career path, it's easy to fixate on what should be possible rather than what actually is. But those idealized paths often require circumstances you don't have—whether personal (e.g., visa status, financial safety net) or external (e.g., your dream org hiring, or a stable funding landscape). It might be helpful to view the paths that work within your actual constraints as your only real options, at least for now. * Adversity Quotient matters. When you're working on problems that may take years to show real progress, the ability to stick around when the work is tedious becomes a comparative advantage. Introduction Hi, I'm Rika. I was born and raised in the Philippines and now work on hiring and recruiting at the Centre for Effective Altruism in the UK. This post might be helpful for anyone navigating the gap between ambition and constraint—whether facing visa barriers, repeated setbacks, or a lack of role models from similar backgrounds. Hearing stories from people facing similar constraints helped me feel less alone during difficult times. I hope this does the same for someone else, and that you'll find lessons relevant to your own situation. It's also for those curious about EA career paths from low- and middle-income countries—stories that I feel are rarely shared. I can only speak to my own experience, but I hop
 ·  · 8m read
 · 
And other ways to make event content more valuable.   I organise and attend a lot of conferences, so the below is correct and need not be caveated based on my experience, but I could be missing some angles here. Also on my substack. When you imagine a session at an event going wrong, you’re probably thinking of the hapless, unlucky speaker. Maybe their slides broke, they forgot their lines, or they tripped on a cable and took the whole stage backdrop down. This happens sometimes, but event organizers usually remember to invest the effort required to prevent this from happening (e.g., checking that the slides work, not leaving cables lying on the stage). But there’s another big way that sessions go wrong that is sorely neglected: wasting everyone’s time, often without people noticing. Let’s give talks a break. They often suck, but event organizers are mostly doing the right things to make them not suck. I’m going to pick on two event formats that (often) suck, why they suck, and how to run more useful content instead. Panels Panels. (very often). suck. Reid Hoffman (and others) have already explained why, but this message has not yet reached a wide enough audience: Because panelists know they'll only have limited time to speak, they tend to focus on clear and simple messages that will resonate with the broadest number of people. The result is that you get one person giving you an overly simplistic take on the subject at hand. And then the process repeats itself multiple times! Instead of going deeper or providing more nuance, the panel format ensures shallowness. Even worse, this shallow discourse manifests as polite groupthink. After all, panelists attend conferences for the same reasons that attendees do – they want to make connections and build relationships. So panels end up heavy on positivity and agreement, and light on the sort of discourse which, through contrasting opinions and debate, could potentially be more illuminating. The worst form of shal