A U.S. congressional commission on Tuesday proposed a Manhattan Project-style initiative to fund the development of AI systems that will be as smart or smarter than humans, amid intensifying competition with China over advanced technologies.

The bipartisan U.S.-China Economic and Security Review Commission’s (USCC) stressed that public-private partnerships are key in advancing artificial general intelligence (AGI), but did not give any specific investment strategies as it released its annual report.

 

Full report: https://www.uscc.gov/sites/default/files/2024-11/2024_Annual_Report_to_Congress.pdf 

83

0
0

Reactions

0
0
Comments15


Sorted by Click to highlight new comments since:

I spent all day crying about this. An arms race is about the least safe way to approach. And we contributed to this. Many important people read Leopold's report. He promoted it quite hard. But the background work predates Leopold's involvement.

We were totally careless and self aggrandizing. I hope other people don't pay for our sins.

This sounds very much like the missile gap/bomber gap narrative, and yeah this is quite bad news if they actually adopt the commitments pushed here.

The evidence that China is racing to AGI is quite frankly very little, and I see a very dangerous arms race that could come:

https://forum.effectivealtruism.org/posts/cXBznkfoPJAjacFoT/are-you-really-in-a-race-the-cautionary-tales-of-szilard-and

I spent all day crying about this. An arms race is about the least safe way to approach.

I feel that. This report saddens me and I think its recommendations are very bad.

And we contributed to this.

I don't feel that I contributed to this. Perhaps I could have done more to prevent it, although it's not obvious to me what I could have done.

And we contributed to this.

What makes you say this? I agree that it is likely that Aschenbrenner's report was influential here, but did we make Aschenbrenner write chapter IIId of Situational Awareness the way he did? 

But the background work predates Leopold's involvement.

Is there some background EA/aligned work that argues for an arms race? Because the consensus seems to be against starting a great power war.

I think a non-trivial fraction of Aschenbrenner's influence as well as intellectual growth is due to us and the core EA/AI-Safety ideas, yeah. I doubt he would have written it if the extended community didn't exist, and if he wasn't mentored by Holden, etc.

I don't disagree with this at all. But does this mean that blame can be attributed to the entire EA community? I think not. 

Re mentorship/funding: I doubt that his mentors were hoping that he would accelerate the chances of an arms race conflict. As a corollary, I am sure nukes wouldn't have been developed if the physics community in the 1930s didn't exist or mentored different people or adopted better ethical norms. Even if they did the latter, it is unclear if that would have prevented the creation of the bomb. 

(I found your comments under Ben West's posts insightful; if true, it highlights a divergence between the beliefs of the broader EA community and certain influential EAs in DC and AI policy circles.)

Currently, it is just a report, and I hope it stays that way.

Do you see advocating for export controls as fundamentally different from an arms race? Because it seems like export controls are pretty popular among AI policy people.

I honestly don't know. When I think of an arms race, I typically think of rapid manufacturing and accumulation of "weapons." 

Do you think export controls between two countries are a sufficient condition for an arms race?

I do think they are a sufficient condition to kick off an arms race. Export controls are a declaration of hostility, and they force the two countries to decouple from each other. China and the US being decoupled makes the downside of an arms race much lower and the upside much higher.

It's hard to justify export controls unless you believe that we actually are in an arms race, sooner or later. If you wanted to prevent an arms race you couldn't pick a worse policy to put your weight behind. That leads me to conclude that AI policy people who back export controls find an arms race to be acceptable.

I don't think something as strong as this, but I did think at the time that the work on export controls was bad and likely to exacerbate arms race dynamics, and continue to believe this (and the celebration of export controls as a great success of the EA policy efforts was one of the things that caused me to update on future EA-driven AI policy efforts probably being net harmful, though FTX played a bigger role).

How big a deal is the congressional commission? What is the historical track record of Congress implementing the commission's top recommendation?

 

With hindsight, this comment from Jan Kulveit looks prescient.

My gut instinct, given how Trump seems to view the world (i.e. in terms of personal loyalty to him) is that Ivanaka Trump re-tweeting Situational Awareness may actually have been a more significant moment. 

Curated and popular this week
Ben_West🔸
 ·  · 1m read
 · 
> Summary: We propose measuring AI performance in terms of the length of tasks AI agents can complete. We show that this metric has been consistently exponentially increasing over the past 6 years, with a doubling time of around 7 months. Extrapolating this trend predicts that, in under a decade, we will see AI agents that can independently complete a large fraction of software tasks that currently take humans days or weeks. > > The length of tasks (measured by how long they take human professionals) that generalist frontier model agents can complete autonomously with 50% reliability has been doubling approximately every 7 months for the last 6 years. The shaded region represents 95% CI calculated by hierarchical bootstrap over task families, tasks, and task attempts. > > Full paper | Github repo Blogpost; tweet thread. 
 ·  · 2m read
 · 
For immediate release: April 1, 2025 OXFORD, UK — The Centre for Effective Altruism (CEA) announced today that it will no longer identify as an "Effective Altruism" organization.  "After careful consideration, we've determined that the most effective way to have a positive impact is to deny any association with Effective Altruism," said a CEA spokesperson. "Our mission remains unchanged: to use reason and evidence to do the most good. Which coincidentally was the definition of EA." The announcement mirrors a pattern of other organizations that have grown with EA support and frameworks and eventually distanced themselves from EA. CEA's statement clarified that it will continue to use the same methodologies, maintain the same team, and pursue identical goals. "We've found that not being associated with the movement we have spent years building gives us more flexibility to do exactly what we were already doing, just with better PR," the spokesperson explained. "It's like keeping all the benefits of a community while refusing to contribute to its future development or taking responsibility for its challenges. Win-win!" In a related announcement, CEA revealed plans to rename its annual EA Global conference to "Coincidental Gathering of Like-Minded Individuals Who Mysteriously All Know Each Other But Definitely Aren't Part of Any Specific Movement Conference 2025." When asked about concerns that this trend might be pulling up the ladder for future projects that also might benefit from the infrastructure of the effective altruist community, the spokesperson adjusted their "I Heart Consequentialism" tie and replied, "Future projects? I'm sorry, but focusing on long-term movement building would be very EA of us, and as we've clearly established, we're not that anymore." Industry analysts predict that by 2026, the only entities still identifying as "EA" will be three post-rationalist bloggers, a Discord server full of undergraduate philosophy majors, and one person at
 ·  · 2m read
 · 
Epistemic status: highly certain, or something The Spending What We Must 💸11% pledge  In short: Members pledge to spend at least 11% of their income on effectively increasing their own productivity. This pledge is likely higher-impact for most people than the Giving What We Can 🔸10% Pledge, and we also think the name accurately reflects the non-supererogatory moral beliefs of many in the EA community. Example Charlie is a software engineer for the Centre for Effective Future Research. Since Charlie has taken the SWWM 💸11% pledge, rather than splurge on a vacation, they decide to buy an expensive noise-canceling headset before their next EAG, allowing them to get slightly more sleep and have 104 one-on-one meetings instead of just 101. In one of the extra three meetings, they chat with Diana, who is starting an AI-for-worrying-about-AI company, and decide to become a cofounder. The company becomes wildly successful, and Charlie's equity share allows them to further increase their productivity to the point of diminishing marginal returns, then donate $50 billion to SWWM. The 💸💸💸 Badge If you've taken the SWWM 💸11% Pledge, we'd appreciate if you could add three 💸💸💸 "stacks of money with wings" emoji to your social media profiles. We chose three emoji because we think the 💸11% Pledge will be about 3x more effective than the 🔸10% pledge (see FAQ), and EAs should be scope sensitive.  FAQ Is the pledge legally binding? We highly recommend signing the legal contract, as it will allow you to sue yourself in case of delinquency. What do you mean by effectively increasing productivity? Some interventions are especially good at transforming self-donations into productivity, and have a strong evidence base. In particular:  * Offloading non-work duties like dates and calling your mother to personal assistants * Running many emulated copies of oneself (likely available soon) * Amphetamines I'm an AI system. Can I take the 💸11% pledge? We encourage A