Criticism of the 80k job board listing strategy

Yonatan Cale

Criticism of the 80k job board listing strategy

Comments 50

Sorted by

New & upvoted

kush_kan

Hey Yonatan, thanks very much for posting this! It’s useful for 80k to know this stuff so we can improve what we do and how we communicate about it!

Posting below a few concrete things we're planning to do (hopefully soon) to help solve some of the issues outlined:

Better distinguishing more impactful roles

1. We plan to visually distinguish between orgs on our top recommended list (which we think are the most promising places to work in each problem area) and other orgs we list. Note: we plan to make this available as a filter & in our public API

Clearly communicating what we think (esp on how we think about career capital and direct impact)

2. We’ve updated our FAQs to outline as clearly as we can our reasons for listing roles

3. We’re updating the tagline for the job board (our current one can be improved IMO!). I’ll reply to Guy’s comment now with some more of our thinking on this..

Encouraging pushback / users to see more info on orgs

4. We plan to add a link to each org’s EA Forum page on our listings. This is the easiest way we can think of for our users to see more discussion of the orgs

5. We’re adding the new feedback form to the homepage. Also note: (1) the anonymous feedback form goes to a slack channel that we check regularly and is visible to everyone at 80k, (2) we regularly include or de-list roles based on expert views in each problem area

Yonatan Cale

0. Communicating your plans: I endorse this!

"We plan to visually distinguish between orgs on our top recommended list": Yay!
"We’ve updated our FAQs": Please note I don't think people read your FAQ. We can make another Twitter poll to check this, or (what seems to me like best practice) - watch users browse the site without nudging them to any direction.
"We’re updating the tagline for the job board" - same as point 2: I don't think this will solve the problem, and I'm afraid that hoping otherwise is what lead to this situation originally.
[see below]

"We plan to add a link to each org’s EA Forum page on our listings"

I endorse this, but it doesn't solve the problem.

There is no clear place to say "I think this org isn't impactful / is harmful", and no clear place to check if anyone said that.

What do you think?

Also see this comment.

As I already said, I'm really happy you're working to improve the situation

TW123

I think this is putting too much on 80k. They have hundreds of jobs in many different areas listed on their website, and it's a very daunting task to evaluate each one of them when the evaluators are generalists who often have to rely on (conflicting) opinions from people in the community. On top of that, what is career capital for one person could plausibly be direct impact for some other person, so it doesn't really seem one size fits all.

If somebody can't evaluate jobs on the job board for themselves, I'm not that confident that they'll take a good path regardless. People have a tendency to try to offload thinking to places like 80k, and I actually think it could be bad if 80k made it easier to do that on extremely fine grained topics like individual jobs.

I do like the idea of having comments on particular jobs. And it also would be good for 80k to be more clear they don't expect all these jobs to necessarily have direct impact.

I don't know whether it's the case that many people on the internet are looking at the job board and deciding which jobs to apply to when they don't have a strong engagement with EA ideas, and that these sorts of people are the types who would actually get the jobs. If that's the case (80k would know better than me), then I think it maybe does make sense to restrict to jobs that aren't going to be bad if such a person gets them. That seems like an empirical question.

Yonatan Cale

My vision (agreeing but elaborating):

If each job^[1] will have a clear place to discuss it, where someone can comment "I think this is harmful because..." and someone else can reply "I think this is wrong because...", and even the org itself (or 80k) can reply with their take - I would personally say this is a good situation.

Extra bonuses from this solution:

If I want an impactful job, I have a clear place to check if anyone pushed back on this job and why (and also: seeing no pushback is a useful data point)
We get a healthy [in my opinion] conversation about impact within the community
As you said, not "putting too much on 80k", which I also endorse

^{^}
Or each org, I'm not sure

Guy Raveh

How would you plan this exactly? Comments per job posting or per organisation?

On the one hand job postings come and go, so if you think anyone working at [Effective Consultancy] is doing a bad thing, you don't want to have to keep looking for their postings and re-comment.

On the other hand, maybe you think their [Consultancy Safety] team is currently doing good work, and you want to endorse it temporarily while not endorsing the rest of EC.

Spencer R. Ericson

(I see that this is old) Etsy's product reviews for every product in the seller's shop are all on every product page. There might be some fancy sorting bringing the more relevant reviews to the top, I'm not sure. So with a model like this, you could review one job posting, and people could see that job posting review on all the company's other job postings.

Kirsten

I disagree - I think it's okay if 80,000 Hours occasionally makes mistakes, but I'd prefer it if their policy was to only add jobs to their job board that they believe would make a positive difference to the world. I don't think that's too much to ask.

Lukas_Gloor

I'd worry that this leads to a false sense of security. Just like jobs that people take purely for career capital require some active thinking on the part of the person about when it's enough and when to pivot, one could make a case that most highly impactful jobs wouldn't be exceptionally impactful without "active thinking" of a similar kind.

For instance, any sort of policy work has more or less impact depending on what specific policies you advocate for, not just how well one does it.

Unfortunately, I think it's somewhat rare that for-profit organizations (especially outside of EA) or governments have streamlined missions and the type of culture that encourages "having impact" as a natural part of one's job description. Hospitals are the main counter-example I could think of, since your job description as a doctor or nurse or even as almost any hospital staff is literally about saving lives and may include instructions for working under triage conditions. By contrast, the way I envision work in policy (you obviously know more about this than I do) or things like biosecurity research, I'd imagine it depends a lot on the specific program / group and that people can make a big difference if they have personal initiative – which are things that require paying close attention to one's path to impact (on top of excelling at one's immediate job description).

What IMO could be quite useful is if 80k would say how much of a given job's impact comes from "following the job description and doing well in a conventional sense" vs. "introducing particular ideas or policies to this organization based on EA principles."

Kirsten

I kind of don't see the point of the job board if it's just "these are jobs, some are good and some are bad, good luck"? Why would I use their job board at all? I'd prefer a job board where the rationale for posting each role was something like"80k thinks you should consider whether you'd have significant impact in this role based on your individual circumstances."

80k doesn't even need to change how they write about their jobs, if that's your concern, and I agree people should still use critical thinking - but I don't think they should list jobs that are likely negative impact or only useful for career capital.

(As a data point I currently don't use their job board, but I'd be more likely to if they at least aimed to know list jobs that would have a significant positive impact on the world.)

JKM

Personally I’m super grateful 80,000 Hours posts roles that are good for career capital, even if they are at organisations that have questionable overall impact, and even if the roles themselves may actively do harm. This is because I’m still fairly young and am interested in building career capital!

It’s plausible a software engineering role at Facebook might do harm, but I still think many EAs would rightly jump at this opportunity, and I’d rather have an EA in the role than a non-EA.

Also, I feel like it’s pretty easy for me to know which roles are directly impactful and which are for career capital (or which are both). For example, roles that aren’t clearly related to one of 80K’s top problems are usually going to be there for career capital reasons - these are also usually at very well known orgs. 80K actually gives some examples of organisations some people might think are harmful but which they still recommend roles for in their guidance - these being Amazon, Facebook and the US military. It seems pretty obvious to me as someone who has read the rest of 80K’s guidance how to judge the roles for myself and I’m just grateful 80K has listed them for me.

So I think it would be a big loss if 80,000 Hours stopped posting these roles. Should they make it clearer which roles are immediately directly impactful and which aren’t? Maybe. This would be catering for people who can’t figure it out for themselves, but that isn’t necessarily a bad thing…

Kirsten

Are you, as a software developer, not aware of roles at Facebook unless you see 80,000 Hours advertise them? I'm struggling to see the value add of advertising a role like that

JKM

You might not be aware of them if you haven't signed up for Facebook careers alerts or look at the Facebook careers website regularly.

Of course you might say "just sign up for the careers alerts then". But you'd then want to do this for all of the impressive organisations that you would potentially want to work for, of which there may be quite a few. Two possible downsides of this are:

You might miss a few good options accidentally. Maybe the places to work as a software engineer are pretty obvious, but this won't always be the case. For example, maybe someone looking to build career capital in policy won't be aware of all of the good options available, including individual think tanks or other organisations that do impactful policy work. I work at the Confederation of British Industry (CBI) which surprisingly many people aren't aware of (I met Will MacAskill one time and he hadn't heard of it!) - but it's got a pretty good reputation and soon after I joined the CBI someone left to become Executive Director of the Centre for Data Ethics & Innovation, a pretty high impact and EA-relevant role. Furthermore I don't think the role she came from at the CBI was a directly high impact role (by EA lights), so the role she came from would have been excluded from 80K's job board under your preference. In short, I doubt everyone is automatically aware of all good career capital roles.
It might be annoying to have loads of career alert emails when you could see them all in one place. I quite like getting the email from 80,000 Hours reminding me to look at the job board and then seeing everything in one place. Makes life kind of easy! I don't 100% rely on the 80K job board, but if the 80K job board covers all bases then one could rely on it guilt-free, and it might make life easier for them.

Yonatan Cale

Hey,

Regarding jobs for building career capital

This is a much simpler problem that I'm happy to help with [1] [2] [3].

The problem of finding impactful roles is much harder

And that's where I need (and I think others too) help from an org like 80k.

Facebook

This is not an example of a company that worries me.

"roles that aren’t clearly related to one of 80K’s top problems are usually going to be there for career capital reasons"

I think:

Some of these are potentially overly damaging
Roles that are related to 80k's top causes might also be there for career capital

"I’m just grateful 80K has listed them [Amazon, Facebook and the US military] for me"

Again, I think it would be very easy to list roles like this without help from 80k experts. If it seems to be a pain point, perhaps we could talk about it.

"This [saying which roles are impactful and which not] would be catering for people who can’t figure it out for themselves"

I think this is useful in a similar way to how Givewell are useful. They do the analysis so we don't all need to do it individually. Not a perfect example, but points to something I mean.

I can also tell you in practice that many developers (dozens?) expect this from 80k, and also see the Twitter poll at the top of the post, which I think hints this is a problem for many people

Guy Raveh

This is a much simpler problem that I'm happy to help with

Just flagging that you're referring to the problem of "getting career capital as a SW engineer" and not "getting career capital", which is in general much harder.

Though the problem you are talking about is in my opinion somewhat more complex than you think.

Yonatan Cale

I'm talking to guy one on one about what complexity I'm missing and he may share anything I say

JKM

This is a much simpler problem that I'm happy to help with

I'd rather all roles be summarised in one place for simplicity. If people are concerned about not knowing which roles are for career capital vs direct impact then 80K can signpost that - which I think I am in favour of. I'm not sure why removing the career capital roles would be the better approach - I think it would be a loss of value.

Some of these are potentially overly damaging

Can you give some examples? I'm interested.

"I'm just grateful 80K has listed them for me"

When I said this I was referring to all roles they list not just the career capital ones by the way.

Yonatan Cale

Examples: Discussing one here: https://forum.effectivealtruism.org/posts/YCMgg6x6zWJmran5L/criticism-of-the-80k-job-board-listing-strategy?commentId=zZHtmpFRXGg8SK43b

JKM

Are you sure you linked to the right place? I don’t see an example of a role you think is very harmful

Yonatan Cale

I'm not against keeping career capital roles in if they're clearly marked

Yonatan Cale

Big upvote to:

What IMO could be quite useful is if 80k would say how much of a given job's impact comes from "following the job description and doing well in a conventional sense" vs. "introducing particular ideas or policies to this organization based on EA principles."

This would not solve everything, but it would improve a lot.

calebp

If somebody can't evaluate jobs on the job board for themselves, I'm not that confident that they'll take a good path regardless. People have a tendency to try to offload thinking to places like 80k, and I actually think it could be bad if 80k made it easier to do that on extremely fine grained topics like individual jobs.

I often agree with something in this direction in specific cases, when there is some skill that is present in both the job choosing and job performing endeavour. I think job choosing is often largely about 'having good judgement and caring a lot' whereas doing well in a job often does not rely on 'having good judgment'.

I think there are many examples of solid software engineers, operations staff, marketers etc. where having good judgement does not seem to be particularly important for their role (although that's not to say good judgement and these roles are anticorrelated or that good judgment isn't ever required here).

TW123

I think you're right and it's worth thinking about these cases. That being said, I think tail impact is going to come from people who have good judgement including on how to develop their careers, open new opportunities, and select future jobs across their career. It's unclear to me which group the 80k job board should be catering to, but plausibly those most extremely self-motivated people don't need a job board to show them their options.

Yonatan Cale

People who don't need a job board don't need a job board

Lukas_Gloor

If somebody can't evaluate jobs on the job board for themselves, I'm not that confident that they'll take a good path regardless.

That was also my instinctive reaction to this post. At least in the sense of "if someone can't distinguish what's mostly for career capital vs. where a specific role ends up saving lives or improving the world, that's a bit strange."

That said, I agree with the post that the communication around the job board can probably be improved!

Yonatan Cale

If you'd know that there are many developers (including senior ones) who don't want to do an effectiveness analysis and mainly want to go work somewhere useful, and expect this to be the 80k job board, would that change your mind?

Or would you say something like "we don't want to hire those people"?

Yonatan Cale

This is extra true for complicated calculations like "does this AI company do more safety or more capabilities?"

[example link for why this matters]

Lukas_Gloor

Thanks, those are good examples and I think you're changing my mind a bit! If the board just lists all kinds of jobs at a particular org and that org also hires for developers (or some other role that requires comparatively little involvement with organizational strategy, perhaps operations in some cases – though note that operations people often take on various responsibility that shape the direction of an organization), that could be quite misleading. This would be a problem even if we don't expect 80k to directly recommend to developers to take developer jobs at an org that they don't think has positive impact.

"does this AI company do more safety or more capabilities?"

That's yet another challenge, yeah. Especially because there may not even always be a consensus among thoughtful EAs on how much safety work (and what sort of org structure) is enough.

Yonatan Cale

Thanks for saying!

Regarding

there may not even always be a consensus among thoughtful EAs on how much safety work (and what sort of org structure) is enough.

My current best suggestion is "let there be a place for the community to discuss this, and then job seekers can at least see the discussion, at least see what the main arguments are of if they even exist". What do you think?

Guy Raveh

I'd be happy for some explanation on why even a binary tag - "impact" vs. "career capital" - is infeasible. What's the selection process that leads to this being unclear?

JP Addison🔸

Imagine a junior level role in gov't where you're working on writing documents about emerging technology that no one important will read. Some things that would seem to my quick thoughts to make this hard:

This job will initially not have much direct impact, but will ramp up if you're good.
To be clear, even in the world where you're good, a major source of impact of this role is career capital.
It's rude to the partners with whom 80k is working to say "your org is not impactful, we're sending people to you with the understanding that they'll build career capital and leave".

Yonatan Cale

I have lots of thoughts and some suggested solutions for this, but my main pushback is that sending someone to that kind of role for that goal without giving them this context is problematic.

Yonatan Cale

My secondary pushbacks are

Saying "we think this org is impactful but not our top in terms of [our complicated calculation]" isn't something that I think a non-EA org would be insulted about.
Saying "you'll learn a lot there that can help you in the rest of your career" might even be considered positive by such organizations.

(I'm not jumping to my actual proposed solutions because (A) I'm trying to keep the post focused, and (B) I'd like to wait for 80k's comment, which I expect they'll write soon, and might address what you (JP) wrote)

Yonatan Cale

JP, regarding your point 3:

It's rude to the partners with whom 80k is working to say "your org is not impactful, we're sending people to you with the understanding that they'll build career capital and leave".

See 80k's response, which I interpret as "they are willing to be 'rude' and say that an org isn't top recommended"

We plan to visually distinguish between orgs on our top recommended list (which we think are the most promising places to work in each problem area) and other orgs we list.

https://forum.effectivealtruism.org/posts/YCMgg6x6zWJmran5L/criticism-of-the-80k-job-board-listing-strategy?commentId=zZHtmpFRXGg8SK43b

I endorse their approach here

ChristianKleineidam

Building career capital doesn't automatically mean leaving. If we take the example of the junior-level role in government, building career capital means being able to access senior roles in government in the future instead of leaving.

kush_kan

Hey, (Kush here, new product manager for the 80k job board). This is a good question!

So we basically post roles for a combination of (1) they are an opportunity for people to have impact in the role, (2) they are an opportunity for the person to build career capital to have an impact in the future.

In practice very few roles fall squarely in each of these categories and separating out the mix of each factor ex ante is really hard. (All of the roles we think are the most impactful are also great career capital opportunities, and even in a mostly career building role a great person can have a lot of impact).

I think we haven’t communicated this super well (we're planning on changing our landing page taglines to reflect, and also have updated our FAQs). I’m also hoping that some of the changes in my other comment help on this but we’re also open to further suggestions

JKM

I think I’d be in favour of you guys indicating for each role if it is:

Mainly there for direct impact
Mainly there for career capital
There for both impact and career capital (I.e. neither one clearly dominates the other)

Obviously whoever is putting the role on the board is doing so for a reason, so should know the answer to this question.

Miguel Lima Medín 🔸

Hi,

I was quite afraid when I read that there are jobs listed for career capital. I'm glad to read that actually there is a combination of impact and capital. I hope all jobs posted have a significant positive impact.

In my mind "High Impact" should be the first priority. If we have to add other considerations I would also go for “EA Community Expansion and Diversity” as more important than "Career Capital".

Let's assume we have already listed all the High Impact opportunities we can find. As the next step I would go for listing the second job position in China rather than the 301^st in San Francisco, even if the second one is better for career capital.

Note: I edited my original post to make it more clear.

Yonatan Cale

Update:

I love that the 80k job board team added this filter:

as well as in the "area":

And the tags (and even title!) in some of the postings:

(and maybe more things I didn't notice yet?)

This seems both well communicated (I won't take a job that I mistakenly think is rated by 80k as high impact) and it's easy to configure based on what I'm actually looking for.

I really like it, and I'll edit the post to indicate that the original criticism I had is mostly resolved.

Kudos from me to @kush_kan and the rest of the team

Yonatan Cale

Tiny suggestion:

In the "Career development: Technical tag"

add alt text that appears when my mouse is over it, similarly to what you did here:

(which looks really good and clear to me, I love it)

David M

A problem for 80,000 Hours' jobs board and marketing departments (from my outside perspective): the overall effect we should expect, if 80K have powerful marketing that brings new people into the website, and if the jobs board doesn't distinguish career capital roles from impact roles, is that 80K funnels people with little exposure to EA ideas straight into career capital roles (which those people mistake for impactful roles), and 80K loses contact with them thereafter; and the overall effect of that would be to have ~no direct impact, and possibly prevent a more-engaged person from accessing the career capital.

Ezra Newman

See also https://www.effectivejobsboard.org/, courtesy of Nathan Young and Superlinear.

Yonatan Cale

I didn't write about them because, as opposed to 80k:

They don't vet and don't claim to. Anyone can add a job.
They are not (sorry for being extra subjective here, feel free to push back) a corner stone of EA that many people know and trust and have certain expectations from.

Miranda_Zhang

I find myself slightly confused - does 80k ever promote jobs they consider harmful (but ultimately worth it if the person goes on to leverage that career capital)?

My impression was that all career-capital building jobs were ~neutral or mildly positive. My stance on the 80k job board—that the set up is largely fine, though the perception of it needs shifting—would change significantly if 80k were listing jobs they thought were net negative if they didn't expect the person to later take an even higher-impact role because of the net negative job.

Linch

One obvious example is working in AI companies, particularly companies directly aimed at building AGI. The jobs are default harmful, but it might be good for EAs to be the ones to work there (especially if they are careful/have good judgement and substantial moral courage). But the sign of the direct impact case is at best unclear. The career capital case is comparatively stronger however.

Yonatan Cale

I agree AGI companies are a good example for where this question matters, and where the impact may not be ~neutral or mildly positive, and this is an important example.

I agree it might sometimes be worth working there for career capital, which isn't negligible.

I don't agree that it's ok to send people there without telling them about this plan.

A "naive" example is an engineer working at a great company and building career capital, wanting to move to a company that has high impact, and unknowingly picking an AI org from the 80k job board that does harm, where 80k put the org there for career capital reasons.

Linch

Thanks. I did not mean to imply that working in AGI companies to build career capital is clearly good or even something that will come out to be net positive after (more informed) reflection. I find myself pretty confused about the sign, and know people who have strong arguments or beliefs in both directions.

calebp

I think it is kind of odd to say that the setup of the jobs board is fine but the perspective needs shifiting, as 80k are by far best positioned to change the perspective people have of the jobs board.

I am not confident that 80k are making a bad trade off here, the current setup may well be close to optimal given the tradeoffs (including the time tradeoff of bothering to optimise these things). But, I am a bit averse to attitudes of 'it's not this one orgs problem, everyone else needs to change' when it seems more efficient to address issues at the source.

Miranda_Zhang

To clarify, I agree that 80k is the main actor who could + should change people's perceptions of the job board!

Criticism of the 80k job board listing strategy

"We plan to add a link to each org’s EA Forum page on our listings"

My vision (agreeing but elaborating):

Extra bonuses from this solution:

Regarding jobs for building career capital

The problem of finding impactful roles is much harder

Facebook

"roles that aren’t clearly related to one of 80K’s top problems are usually going to be there for career capital reasons"

"I’m just grateful 80K has listed them [Amazon, Facebook and the US military] for me"

"This [saying which roles are impactful and which not] would be catering for people who can’t figure it out for themselves"

Criticism of the 80k job board listing strategy

"We plan to add a link to each org’s EA Forum page on our listings"

My vision (agreeing but elaborating):

Extra bonuses from this solution:

Regarding jobs for building career capital

The problem of finding impactful roles is much harder

Facebook

"roles that aren’t clearly related to one of 80K’s top problems are usually going to be there for career capital reasons"

"I’m just grateful 80K has listed them [Amazon, Facebook and the US military] for me"

"This [saying which roles are impactful and which not] would be catering for people who can’t figure it out for themselves"

I’ll describe the poll in words, mainly for listeners of the Nonlinear Podcast:

Why I think this is important

Read more

Solutions

80k’s response

In the job board page:

80k’s website, on promoting some roles at potentially harmful organizations

Criticism: “Improving the org from the inside”:

80k’s article on taking a harmful job

Criticism on closed door push backs:

80k’s response on the forum:

Kush (the new Head of Job Board at 80k (best job in the world!)) explains Niel’s answer:

I’m discussing and collaborating with Kush (head of job board)