NN

Neel Nanda

4580 karmaJoined neelnanda.io

Bio

I lead the DeepMind mechanistic interpretability team

Comments
347

Thanks for clarifying! I somewhat disagree with your premises, but agree this is a reasonable position given your premises

Thanks for the post, it seems like you're doing valuable work!

I'm curious how you'd compare One Acre Fund's work to the baseline of just directly giving the farmers cash to spend as they see fit? And if you did this, do you expect they would spend it on the kind of things One Acre Fund is providing?

Based on this post, possible arguments I see:

  • You believe that loans make the approach more efficient as money is often paid back
  • You can provide expertise and teaching which is hard to purchase, or people may not value correctly

I do have a lot of respect for the Open Phil team I just think they are making some critical mistakes, which is fully compatible with respectability

Sorry, my intention wasn't to imply that you didn't respect them, I agree that it is consistent to both respect and disagree.

Re the rest of your comment, my understanding of what you meant is as follows:

You think the most effective strategies for reducing AI x risk are explicitly black listed by OpenPhil. Therefore OpenPhil funding an org is strong evidence they don't follow those strategies. This doesn't necessarily mean that the orgs work is neutral or negative impact, but it's evidence against being one of your top things. Further, this is a heuristic rather than a confident rule, and you made the time for a shallow investigation into some orgs funded by OpenPhil anyway, at which point heuristics are screened off and can be ignored anyway.

Is this a correct summary?

As a rule of thumb, I don't want to fund anything Open Philanthropy has funded. Not because it means they don't have room for more funding, but because I believe (credence: 80%) that Open Philanthropy has bad judgment on AI policy (as explained in this comment by Oliver Habryka and reply by Akash—I have similar beliefs, but they explain it better than I do).

This seems like an bizarre position to me. Sure, maybe you disagree with them (I personally have a fair amount of respect for the OpenPhil team and their judgement, but whatever, I can see valid reasons to criticise), but to consider their judgement not just irrelevant, but actively such strong negative evidence as to make an org not worth donating to, seems kinda wild. Why do you believe this? Reversed stupidity is not intelligence. Is the implicit model that all of x risk focused AI policy is pushing on some 1D spectrum such that EVERY org in the two camps is actively working against the other camp? That doesn't seem true to me.

I would have a lot more sympathy with an argument that eg other kinds of policy work is comparatively neglected, so OpenPhil funding it is a sign that it's less neglected.

In fairness, I wrote my post because I saw lots of people making arguments for a far stronger claim than necessary, and was annoyed by this

Community seems the right categorisation to me - the main reason to care about this is understanding the existing funding landscape in AI safety, and how much to defer to them/trust their decisions. And I would consider basically all the large funders in AI Safety to also be in the EA space, even if they wouldn't technically identify as EA.

More abstractly, a post about conflicts of interest and other personal factors, in a specific community of interest, seems to fit this category

Being categorised as community doesn't mean the post is bad, of course!

Personally, I find the idea somewhat odd/uncomfortable, but also vaguely buy the impact case, so I've only added it on LinkedIn, as that's the social network where I feel like the norm is shameless signalling and where I tie it least to my identity - I may as well virtue signal rather than just bragging!

This seems a question of what the policy is, not of judgement re how to apply it, in my opinion.

The three examples you gave obviously are in the category of "controversial community drama that will draw a lot of attention and strong feelings", and I trust the mod's ability to notice this. The question is whether the default policy is to make such things personal blog posts. I personally think this would be a good policy, and that anything in this category is difficult to discuss rationally. I do also consider the community pane a weaker form of low visibility, so there's something here already, but I would advocate for a stronger policy.

Another category is "anything about partisan US politics", which I don't think is that hard to identify, is clearly hard to discuss rationally, and in my opinion is reasonable to have a policy of lowering the visibility of.

I don't trust karma as a mechanism, because if the post is something that people have strong feelings about, and many of those feelings are positive (or at least, righteous anger style feelings), then posts often get high karma. Eg I think the Nonlinear posts got a ton of attention, in my opinion were quite unproductive and distracting, got very high karma, and if they had been less visible I think this would have been good

Load more