Theories of consciousness will lead to actionable understanding of AI consciousness²
Global workspace theory makes anthropic's discovery of the "j-space" much more plausibly a sign of AI consciousness. If a theory of consciousness doesn't help us determine between whether AI is conscious or not, I don't think it's much better than a theory of phlogiston is to fire.
Benchmarks will become useless due to eval awareness¹
I think they will become over saturated and disconnected from real world usage such that they will be pretty useless, but I don't think eval awareness is what will make them lose their value
Here's an idea for a pro-animal campaign that could absorb a lot of money: subsidize vegan food. Make mock meats and other plant based stuff way cheaper, so people are more likely to try it and get used to it and like it.
I think you could know how to encode preferences into ASI without knowing whether it's aligned. At that point it's like a genie, you may well be able encode preferences into it but you might encode bad preferences that you didn't realize would be bad.
Depends on what you mean by the term "consciousness"
As I understand, many key players are explicitly anthropocentrists. Hell, some key players care more about money than humans, let alone non-humans.
Understanding how they learn is how we get insights into methods of teaching them good values
Afaik there isn't any robust research on how to use AI to end factory farming. That alone is a sign to me that we are far behind.
Global workspace theory makes anthropic's discovery of the "j-space" much more plausibly a sign of AI consciousness. If a theory of consciousness doesn't help us determine between whether AI is conscious or not, I don't think it's much better than a theory of phlogiston is to fire.
If we use ai to bring factory farming to the stars, I think that will likely be worse than all the benefits it'll bring.
I think they will become over saturated and disconnected from real world usage such that they will be pretty useless, but I don't think eval awareness is what will make them lose their value
You guys get to be rational actors??
Here's an idea for a pro-animal campaign that could absorb a lot of money: subsidize vegan food. Make mock meats and other plant based stuff way cheaper, so people are more likely to try it and get used to it and like it.
I think you could know how to encode preferences into ASI without knowing whether it's aligned. At that point it's like a genie, you may well be able encode preferences into it but you might encode bad preferences that you didn't realize would be bad.