That makes sense, thanks for sharing. In your post on your HPMOR campaign, you describe asking questions like how much recipients liked the book and how much it influenced them, and I'm guessing you're asking similar questions here. Since you asked, I think it could be valuable to also ask the recipient to briefly describe if reading the book caused them to take some action they would otherwise have not taken, whether that's taking a particular job or something smaller. If you do end up publishing a write-up of this IABIED campaign once it's concluded, I'd be interested to read it!
That makes sense, thanks for responding. I can see where you're coming from.Â
I think early spacefaring will probably be a quite competitive space, since there seems to be a lot of diverse commercial and government interest right now. Because of that, it seems like it may be difficult to create any sort of durable norms around something like food sourcing unless there's a clear winner on efficiency, which I presume would be arrived at independently of whether it involves animals. This might be weak argument if you expect early spacefaring to be dominated by an individual entity like NASA and if you expect government contracts for things like food in space to not be very competitive.Â
Maybe something that would make this more concrete is looking at whether there are examples of the lock-in of practices like this in the past, and whether we have reason to believe that additional dynamics would make early spacefaring harder to draw historical comparison to. What do you think?
Thanks for sharing that post, I thought it provided good insight into how you're thinking about reputational risk. I did come away from the post wondering why the proposed reputational risk management method doesn't appear more widespread across competitive firms if it has an edge on the traditional approach.Â
I don't know if I understand/agree with your comparison of CEA to Yudkowsky. CEA, as the steward of much of the EA community's infrastructure, has a lot of authority and responsibility for this community and its people. In order to maintain peoples' trust and confidence to do that well, I suspect it's necessary for it to observe a quite conservative form of message discipline that's more consistent with traditional PR than the other strategy that the post you shared mentions. I can think of a couple examples where I don't agree with a communications decision I've heard of them making, but I don't think Yudkowsky's approach that you gesture to would be the right direction to move. Maybe looking to the ways organizations like universities and churches succeed and fail with communications would provide more useful comparisons.Â
For individuals or organizations that don't have responsibility for community stewardship, I agree with the praise you give of Yudkowsky's approach.Â
Maybe there are other signals that aren’t vulnerable to the same kind of faking, like evidence of integrity and discretion paired with a track record of impact of the type you’re looking for. Seems harder to easily evaluate, but also seems like a more common way to evaluate alignment with a cause’s mission.Â
Separately, while I’m vegan myself, I think using veganism as a signal is less reliable than it may appear because it appears not to meet the same effectiveness bar as other popular interventions, and there’s honest disagreement within EA on whether EAs should generally follow a vegan diet. I think I’d have the same concerns with using other marginally good but costly lifestyle choices, like donating a kidney, as a signal as well, though I haven’t thought about a lot of individual cases.Â
Thank you for posting this! I appreciated both learning how others feel about this and your thoughtful commentary.Â
Your last paragraph especially reminded me of The Plague by Albert Camus. Beyond its political allegory (and topical subject matter), I read it as an absurdist case for persistent altruism and an approach to everyday life that centers present experiences over hope and despair that map onto the narrative of a broader cause. Both ideas resonated with me later when I read posts in the EA handbook on altruism and scope sensitivity like Nate Soares’ On Caring.Â
I haven’t spent a lot of time thinking about this, but I suspect a couple reasons to continue pursuing this contract beyond the present revenue include (1) retaining relationships and a reputation that provides option value for (especially defense-related) future contracts and (2) increasing the likelihood that safer models are used in high-stakes settings, especially ones that could carry some non-negligible AI-related risks. While those are plausible (and plausibly right) lines of reasoning, I’m writing them without taking a stance on specific details that have central importance to their truth (e.g. are Anthropic’s models “safer” than competitors’? Seems quite likely based on reputation, but I’m not well-informed enough to make that claim confidently). If you’re right and the alternative is the use of another lab’s models for the same jobs, and if the article is right that Anthropic’s models are the only ones being used on classified networks, then I don’t think there are good reasons for Anthropic to intentionally cede that space to competitors.Â
That makes sense, thanks for sharing. In your post on your HPMOR campaign, you describe asking questions like how much recipients liked the book and how much it influenced them, and I'm guessing you're asking similar questions here. Since you asked, I think it could be valuable to also ask the recipient to briefly describe if reading the book caused them to take some action they would otherwise have not taken, whether that's taking a particular job or something smaller. If you do end up publishing a write-up of this IABIED campaign once it's concluded, I'd be interested to read it!
That makes sense, thanks for responding. I can see where you're coming from.Â
I think early spacefaring will probably be a quite competitive space, since there seems to be a lot of diverse commercial and government interest right now. Because of that, it seems like it may be difficult to create any sort of durable norms around something like food sourcing unless there's a clear winner on efficiency, which I presume would be arrived at independently of whether it involves animals. This might be weak argument if you expect early spacefaring to be dominated by an individual entity like NASA and if you expect government contracts for things like food in space to not be very competitive.Â
Maybe something that would make this more concrete is looking at whether there are examples of the lock-in of practices like this in the past, and whether we have reason to believe that additional dynamics would make early spacefaring harder to draw historical comparison to. What do you think?
Thanks for sharing that post, I thought it provided good insight into how you're thinking about reputational risk. I did come away from the post wondering why the proposed reputational risk management method doesn't appear more widespread across competitive firms if it has an edge on the traditional approach.Â
I don't know if I understand/agree with your comparison of CEA to Yudkowsky. CEA, as the steward of much of the EA community's infrastructure, has a lot of authority and responsibility for this community and its people. In order to maintain peoples' trust and confidence to do that well, I suspect it's necessary for it to observe a quite conservative form of message discipline that's more consistent with traditional PR than the other strategy that the post you shared mentions. I can think of a couple examples where I don't agree with a communications decision I've heard of them making, but I don't think Yudkowsky's approach that you gesture to would be the right direction to move. Maybe looking to the ways organizations like universities and churches succeed and fail with communications would provide more useful comparisons.Â
For individuals or organizations that don't have responsibility for community stewardship, I agree with the praise you give of Yudkowsky's approach.Â
There are some good write-ups on her profile on EA in the Lakes, like this one.Â
Could you talk more about how you think the norm might persist?Â
Maybe there are other signals that aren’t vulnerable to the same kind of faking, like evidence of integrity and discretion paired with a track record of impact of the type you’re looking for. Seems harder to easily evaluate, but also seems like a more common way to evaluate alignment with a cause’s mission.Â
Separately, while I’m vegan myself, I think using veganism as a signal is less reliable than it may appear because it appears not to meet the same effectiveness bar as other popular interventions, and there’s honest disagreement within EA on whether EAs should generally follow a vegan diet. I think I’d have the same concerns with using other marginally good but costly lifestyle choices, like donating a kidney, as a signal as well, though I haven’t thought about a lot of individual cases.Â
Good luck! What outcomes are you planning to measure?
The best thing I’ve read on it so far is this article by Kelsey Piper.Â
Thank you for posting this! I appreciated both learning how others feel about this and your thoughtful commentary.Â
Your last paragraph especially reminded me of The Plague by Albert Camus. Beyond its political allegory (and topical subject matter), I read it as an absurdist case for persistent altruism and an approach to everyday life that centers present experiences over hope and despair that map onto the narrative of a broader cause. Both ideas resonated with me later when I read posts in the EA handbook on altruism and scope sensitivity like Nate Soares’ On Caring.Â
I haven’t spent a lot of time thinking about this, but I suspect a couple reasons to continue pursuing this contract beyond the present revenue include (1) retaining relationships and a reputation that provides option value for (especially defense-related) future contracts and (2) increasing the likelihood that safer models are used in high-stakes settings, especially ones that could carry some non-negligible AI-related risks. While those are plausible (and plausibly right) lines of reasoning, I’m writing them without taking a stance on specific details that have central importance to their truth (e.g. are Anthropic’s models “safer” than competitors’? Seems quite likely based on reputation, but I’m not well-informed enough to make that claim confidently). If you’re right and the alternative is the use of another lab’s models for the same jobs, and if the article is right that Anthropic’s models are the only ones being used on classified networks, then I don’t think there are good reasons for Anthropic to intentionally cede that space to competitors.Â