What institutions or theorems guarantee that ASI or AGI will converge toward behaving as the rational agent that game theory assumes? According to the orthogonality thesis, any level of intelligence can pursue any arbitrary goal, so isn't there a possibility that it could make choices that are not rational? This also depends on whether the game is one of complete information or incomplete information.
What institutions or theorems guarantee that ASI or AGI will converge toward behaving as the rational agent that game theory assumes? According to the orthogonality thesis, any level of intelligence can pursue any arbitrary goal, so isn't there a possibility that it could make choices that are not rational? This also depends on whether the game is one of complete information or incomplete information.