Three Epoch employees – Matthew Barnett, Tamay Besiroglu, and Ege Erdil – have left to launch Mechanize, an AI startup aiming for broad automation of ordinary labour:
Today we’re announcing Mechanize, a startup focused on developing virtual work environments, benchmarks, and training data that will enable the full automation of the economy.
We will achieve this by creating simulated environments and evaluations that capture the full scope of what people do at their jobs. ...
Currently, AI models have serious shortcomings that render most of this enormous value out of reach. They are unreliable, lack robust long-context capabilities, struggle with agency and multimodality, and can’t execute long-term plans without going off the rails.
To overcome these limitations, Mechanize will produce the data and evals necessary for comprehensively automating work. Our digital environments will act as practical simulations of real-world work scenarios, enabling agents to learn useful abilities through RL. ...
The explosive economic growth likely to result from completely automating labor could generate vast abundance, much higher standards of living, and new goods and services that we can’t even imagine today. Our vision is to realize this potential as soon as possible.
I started a new company with @egeerdil2 and @tamaybes that's focused on automating the whole economy. We're taking a big bet on our view that the main value of AI will come from broad automation rather than from "geniuses in a data center".
The Mechanize website is scant on detail. It seems broadly bad that the alumni from a safety-focused AI org have left to form a company which accelerates AI timelines (and presumably is based on/uses evals built at Epoch).
It seems noteworthy that Epoch AI retweeted the announcement, wishing the departing founders best of luck – which feels like a tacit endorsement of the move.
Habryka wonders whether payment would have had to be given to Epoch for use of their benchmarks suite.
Links
- Official Twitter announcement
- See also this shortform on LessWrong
I think there are two competing failure modes:
(1) The epistemic community around EA, rationality, and AI safety, should stay open to criticism of key empirical assumptions (like the level of risks from AI, risks of misalignments, etc.) in a healthy way.
(2) We should still condemn people who adopt contrarian takes with unreasonable-seeming levels of confidence and then take actions based on them that we think are likely doing damage.
In addition, there's possibly also a question of "how much do people who benefit from AI safety funding and AI safety association have an obligation to not take unilateral actions that most of the informed people in the community consider negative." (FWIW I don't think the obligation here would be absolute even if Epoch had been branded as centrally 'AI safety,' and I acknowledge that the branding issue seems contested; also, it wasn't Jamie [edit: Jaime] the founder who left in this way, and of the people who went off to found this new org, Matthew Barnett, for instance, has been really open about his contrarian takes, so insofar as Epoch's funders had concerns about the alignment of employees at Epoch, it was also -- to some degree, at least -- on them to ask for more information or demand some kind of security guarantee if they felt worried. And maybe this did happen -- I'm just flagging that I don't feel like we onlookers necessarily have the info, and so it's not clear whether anyone has violated norms of social cooperation here or whether we're just dealing with people getting close to the boundaries of unilateral action in a way that is still defensible because they've never claimed to be more aligned than they were, never accepted funding that came with specific explicit assumptions, etc.)