Thanks for writing this, I think there is a lot of opportunity to create art to make the ideas that EAs often talk about like an AI takeover more concrete.
I am shocked and saddened. I did not know Marisa well but we were in the same EA Anywhere discussion group for several months. As you said she was quite funny and I enjoyed talking with her and hearing her ideas.
My guess is that the letter is largely a bluff. I don’t think these people want to work for Microsoft. I’m surprised Altman decided that was his best move vs starting his own company. Perhaps this implies that starting from scratch is not as easy as we think. Microsoft has the license to most (all?) of OpenAIs tech so they would be able to hit the ground running.
Twitter is full of people laying into EA for being behind Sam Altman's firing. However, if it's true that this happened because the board thought Altman was trying to take the company in an 'unsafe' direction then I'm glad they did this. And I'm glad that for the time being considerations other than 'shareholder value' are not the defining motivation behind AI development.
Hi Ozzie, while I agree it's true that there aren't many high-performing organizations which use democratic decision making. I believe Bridgwater Associates, the largest hedge fund in the world, does use such a system. They use a tool called the dot collector to gather real time input from a wide base of employees and use that to come up with a 'believability weighted majority'. The founder of the company Ray Dalio has said that he will generally defer to this vote even when he himself does not agree with the result. https://www.principles.com/principles/3290232e-6bca-4585-a4f6-66874aefce30/
So not as democratic as 1 person 1 vote but far more egalitarian than the average company (or EA for that matter).
We know now that Sam is a pathological liar which makes it hard to take anything he says (including this interview) at face value. He's facing serious jail time for this mess and has a strong incentive to present himself/FTX as incompetent vs deliberate frauds. I'm pretty skeptical of his claim that they just 'happened' to make a sequence of decisions which were above board in isolation but added up to gambling with FTX customer deposits
I agree that EA likely wasn't a major causal factor for FTX/SBF's likely fraud. Unfortunately, it's a situation where even if it's not our fault it is our problem. People are trashing EA across the internet because of Sam's position in the movement. His Twitter profile pic still has him wearing an EA shirt for christ sake!
Interesting ideas and post! First a critique. It didn't sound right that we should wait 50 years for improved governance. Governance isn't something that just improves on its own over time, it's not a force of nature. It manifests and changes in response to human needs. The aftermath of WW1 brought the League of Nations, WW2 brought on the UN and EU, the Cold War created NATO. If anything it seems that changes to governance happen AFTER a need arises, not preemptively. I know we all wish that weren't the case but it is what is.
The development of advanced geo engineering technology could serve as a catalyst for enhanced global governance. It might also lead to war but I think that's less likely than you do. My reasoning is first that it's likely that only great powers will have the ability to deploy such technology and second that the cost of war between great powers would far exceed the benefits of climate change for the aggressor. To confidently keep a country from being able to produce or buy the resources needed for climate change reversal would possibly require occupation. It's hard to imagine that being worth it or even possible.
That said I do buy into Bostrom's Vulnerable World framework, at least in the context of AI and perhaps biotech. It's totally plausible that geo engineering is another black or grey ball in the urn of innovation. The movie Snowpiercer and the weather control superweapon from Red Alert 2 (a video game) came to mind immediately while reading your post haha. I'm not convinced yet of the threat of geo engineering but I am totally in favor of improving global governance (get rid of permanent security council membership!) and applying more caution towards innovation.
Thanks for writing this, I think there is a lot of opportunity to create art to make the ideas that EAs often talk about like an AI takeover more concrete.
I am shocked and saddened. I did not know Marisa well but we were in the same EA Anywhere discussion group for several months. As you said she was quite funny and I enjoyed talking with her and hearing her ideas.
My guess is that the letter is largely a bluff. I don’t think these people want to work for Microsoft. I’m surprised Altman decided that was his best move vs starting his own company. Perhaps this implies that starting from scratch is not as easy as we think. Microsoft has the license to most (all?) of OpenAIs tech so they would be able to hit the ground running.
Twitter is full of people laying into EA for being behind Sam Altman's firing. However, if it's true that this happened because the board thought Altman was trying to take the company in an 'unsafe' direction then I'm glad they did this. And I'm glad that for the time being considerations other than 'shareholder value' are not the defining motivation behind AI development.
Hi Ozzie, while I agree it's true that there aren't many high-performing organizations which use democratic decision making. I believe Bridgwater Associates, the largest hedge fund in the world, does use such a system. They use a tool called the dot collector to gather real time input from a wide base of employees and use that to come up with a 'believability weighted majority'. The founder of the company Ray Dalio has said that he will generally defer to this vote even when he himself does not agree with the result. https://www.principles.com/principles/3290232e-6bca-4585-a4f6-66874aefce30/
So not as democratic as 1 person 1 vote but far more egalitarian than the average company (or EA for that matter).
We know now that Sam is a pathological liar which makes it hard to take anything he says (including this interview) at face value. He's facing serious jail time for this mess and has a strong incentive to present himself/FTX as incompetent vs deliberate frauds. I'm pretty skeptical of his claim that they just 'happened' to make a sequence of decisions which were above board in isolation but added up to gambling with FTX customer deposits
I agree that EA likely wasn't a major causal factor for FTX/SBF's likely fraud. Unfortunately, it's a situation where even if it's not our fault it is our problem. People are trashing EA across the internet because of Sam's position in the movement. His Twitter profile pic still has him wearing an EA shirt for christ sake!
Just did a workshop run by Max at Stanford and had a great time. Would really recommend this!
Interesting ideas and post! First a critique. It didn't sound right that we should wait 50 years for improved governance. Governance isn't something that just improves on its own over time, it's not a force of nature. It manifests and changes in response to human needs. The aftermath of WW1 brought the League of Nations, WW2 brought on the UN and EU, the Cold War created NATO. If anything it seems that changes to governance happen AFTER a need arises, not preemptively. I know we all wish that weren't the case but it is what is.
The development of advanced geo engineering technology could serve as a catalyst for enhanced global governance. It might also lead to war but I think that's less likely than you do. My reasoning is first that it's likely that only great powers will have the ability to deploy such technology and second that the cost of war between great powers would far exceed the benefits of climate change for the aggressor. To confidently keep a country from being able to produce or buy the resources needed for climate change reversal would possibly require occupation. It's hard to imagine that being worth it or even possible.
That said I do buy into Bostrom's Vulnerable World framework, at least in the context of AI and perhaps biotech. It's totally plausible that geo engineering is another black or grey ball in the urn of innovation. The movie Snowpiercer and the weather control superweapon from Red Alert 2 (a video game) came to mind immediately while reading your post haha. I'm not convinced yet of the threat of geo engineering but I am totally in favor of improving global governance (get rid of permanent security council membership!) and applying more caution towards innovation.
Fantastic report! I love this type of content and can't wait to sink my teeth into it