You worry about Anthropic's money being a corrupting influence, but their whole company is far far worse than FTX, because of the existential risk it's subjecting the entire world to. Not only are they continuing in the suicide-onmicide race, they are now leading it!
If you work at Anthropic, please quit and make a big public statement about this. You shouldn't be waiting for the IPO, because there's not much point in having millions to donate if we're all dead in short order (because of how those millions were made). If you own shares, sell them (even pre-IPO, if you can). I no longer consider myself EA because of this issue. Whilst the money and power in EA is pro-Anthropic, I don't want to associate with it.
Dude it's basically the whole of Anthropic! And the fact that EAs (mostly) can't see this is worrying. OP worries about Anthropic's money being a corrupting influence, but their whole company is far far worse than FTX, because of the existential risk it's subjecting the entire world to.
But of course the reason for this is corruption of the entire mission by Anthropic shareholding. I really hope that Dustin will see the light in time and do something about it..
The whole AI Safety grantmaking sector is woefully underinvesting in Pause-related initiatives. It's a tragedy that the world will probably end with so little resources going into pushing for the one solution that might actually prevent it from happening - an international ban (and taboo) on ASI.
I don't think many people biased in such a way are going to even be particularly aware of it when making arguments, let alone admit to it. It's mostly a hidden bias. You really don't want it to be true because of how you think it will affect you if it was.
Thinking AI Risk is among the most important things to work on is one thing. Thinking your life depends on minimising it is another.
Just thinking: surely to be fair, we should be aggregating all the AI results into an "AI panel"? I wonder how much overlap there is between wrong answers amongst the AIs, and what the aggregate score would be?
Right now, as things stand with the scoring, "AGI" in ARC-AGI-2 means "equivalent to the combined performance of a team of 400 humans", not "(average) human level".
Ok, I take your point. But no one seems to be actually doing this (seems like it would be possible to do already, for this example; yet it hasn't been done.)
What do you think a good resolution criteria for judging a system as being AGI should be?
Most relevant to X-risk concerns would be the ability to do A(G)I R&D as good as top AGI company workers. But then of course we run into the problem of crossing the point of no return in order to resolve the prediction market. And we obviously shouldn't do that (unless superalignment/control is somehow solved).
The human testers were random people off the street who got paid $115-150 to show up and then an additional $5 per task they solved. I believe the ARC Prize Foundation’s explanation for the 40-point discrepancy is that many of the testers just didn’t feel that motivated to solve the tasks and gave up [my emphasis]. (I vaguely remember this being mentioned in a talk or interview somewhere.)
I'm sceptical of this when they were able to earn $5 for every couple of minutes' work (time to solve a task). This is far above the average hourly wage.
100% is the score for a "human panel", i.e. a set of at least two humans.
Also seems very remarkable (suspect, in fact) - this would mean almost no overlap between the questions that the humans were getting wrong - i.e. if each human averages 60% right, then for 2 humans to get 100% there can only be 20% of questions where both get it right! I think in practice the panels that score 100% have to contain many more than 2 humans on average.
EDIT: looks like "at least 2 humans" means at least 2 humans solved every problem in the set, out of the 400 humans that attempted them!
They say in their announcement (already linked) that they expelled him. What makes you think he resigned?
FWIW my bio on X now reads: "Ex-EA (left over EA's anti-Pause, pro-Anthropic stance)."
And it's pretty clear that Stop AI are committed to non-violence given they expelled Kirchner after he started talking about it!
(And here is something positive that is happening with Stop AI now (and could be really impactful).)
You worry about Anthropic's money being a corrupting influence, but their whole company is far far worse than FTX, because of the existential risk it's subjecting the entire world to. Not only are they continuing in the suicide-onmicide race, they are now leading it!
If you work at Anthropic, please quit and make a big public statement about this. You shouldn't be waiting for the IPO, because there's not much point in having millions to donate if we're all dead in short order (because of how those millions were made). If you own shares, sell them (even pre-IPO, if you can). I no longer consider myself EA because of this issue. Whilst the money and power in EA is pro-Anthropic, I don't want to associate with it.
Dude it's basically the whole of Anthropic! And the fact that EAs (mostly) can't see this is worrying. OP worries about Anthropic's money being a corrupting influence, but their whole company is far far worse than FTX, because of the existential risk it's subjecting the entire world to.
But of course the reason for this is corruption of the entire mission by Anthropic shareholding. I really hope that Dustin will see the light in time and do something about it..
The whole AI Safety grantmaking sector is woefully underinvesting in Pause-related initiatives. It's a tragedy that the world will probably end with so little resources going into pushing for the one solution that might actually prevent it from happening - an international ban (and taboo) on ASI.
I don't think many people biased in such a way are going to even be particularly aware of it when making arguments, let alone admit to it. It's mostly a hidden bias. You really don't want it to be true because of how you think it will affect you if it was.
Thinking AI Risk is among the most important things to work on is one thing. Thinking your life depends on minimising it is another.
Just thinking: surely to be fair, we should be aggregating all the AI results into an "AI panel"? I wonder how much overlap there is between wrong answers amongst the AIs, and what the aggregate score would be?
Right now, as things stand with the scoring, "AGI" in ARC-AGI-2 means "equivalent to the combined performance of a team of 400 humans", not "(average) human level".
Ok, I take your point. But no one seems to be actually doing this (seems like it would be possible to do already, for this example; yet it hasn't been done.)
What do you think a good resolution criteria for judging a system as being AGI should be?
Most relevant to X-risk concerns would be the ability to do A(G)I R&D as good as top AGI company workers. But then of course we run into the problem of crossing the point of no return in order to resolve the prediction market. And we obviously shouldn't do that (unless superalignment/control is somehow solved).
I'm sceptical of this when they were able to earn $5 for every couple of minutes' work (time to solve a task). This is far above the average hourly wage.
Also seems very remarkable (suspect, in fact) - this would mean almost no overlap between the questions that the humans were getting wrong - i.e. if each human averages 60% right, then for 2 humans to get 100% there can only be 20% of questions where both get it right! I think in practice the panels that score 100% have to contain many more than 2 humans on average.
EDIT: looks like "at least 2 humans" means at least 2 humans solved every problem in the set, out of the 400 humans that attempted them!