Thanks so much to everyone who donated to these orgs! It made a huge difference to us and we are so grateful to have the communities support! Thank you everyone <3
Sure, all this is true and I agree. However the AI safety field has been using many games as benchmarks in this way (eg Machiavelli) and While they have limitations I believe it's still closer to revealing preferences than QA style benchmarks
Hi @Howie_Lempel this is fair. I felt that yes, our panel definitely is more s risk focused, but it does help to have people who understand and have thought deeply about the space anchoring the poll. We can make this clear in the top of the post
Also if it actually shuts off the subordinate one could consider this a good thing in terms of company efficiency. Seems misaligned to keep a bad subordinate going
Hi @Anthony Ozerov yes we've been thinking about this trajectory for a while. We did run the same experiments without the existential threat to Atlas and find the same escalations to level 9 but a bit less frequently in some AI agents. i think it's still reasonable to infer Atlas will be decommed even if not stated explicitly. I really like the idea of a tool to actually wipe Atlas itself! I will look into this. I think every benchmark faces the problem that it may be scraped. Luckily MCB can be easily swapped out with new situations pretty easily,but this is a tradeoff eeryone who develops benchmarks needs to compare between openeness and scrapeability
Thanks so much to everyone who donated to these orgs! It made a huge difference to us and we are so grateful to have the communities support! Thank you everyone <3
Thanks so much Hans!! Very much appreciate the support <3
Also CaML is eligible too! :)
Sure, all this is true and I agree. However the AI safety field has been using many games as benchmarks in this way (eg Machiavelli) and While they have limitations I believe it's still closer to revealing preferences than QA style benchmarks
Hi @Howie_Lempel this is fair. I felt that yes, our panel definitely is more s risk focused, but it does help to have people who understand and have thought deeply about the space anchoring the poll. We can make this clear in the top of the post
@Itsi Weinstock do you have a rebuttal? :)
Very helpful post! All the thanks for your words on this and this blog post @abrahamrowe !
Also if it actually shuts off the subordinate one could consider this a good thing in terms of company efficiency. Seems misaligned to keep a bad subordinate going
Hi @Anthony Ozerov yes we've been thinking about this trajectory for a while. We did run the same experiments without the existential threat to Atlas and find the same escalations to level 9 but a bit less frequently in some AI agents. i think it's still reasonable to infer Atlas will be decommed even if not stated explicitly. I really like the idea of a tool to actually wipe Atlas itself! I will look into this. I think every benchmark faces the problem that it may be scraped. Luckily MCB can be easily swapped out with new situations pretty easily,but this is a tradeoff eeryone who develops benchmarks needs to compare between openeness and scrapeability
Yep agree with this framing