Welcome to the EA Forum bot site. If you are trying to access the Forum programmatically (either by scraping or via the api) please use this site rather than forum.effectivealtruism.org.

This site has the same content as the main site, but is run in a separate environment to avoid bots overloading the main site and affecting performance for human users.

New & upvoted

Customize feedCustomize feed
· · 21m read

Posts tagged community

Quick takes

Show community
View more
Meta has just released Llama 3.1 405B. It's open-source and in many benchmarks it beats GPT-4o and Claude 3.5 Sonnet: Zuck's letter "Open Source AI Is the Path Forward".
Quick[1] thoughts on the Silicon Valley 'Vibe-Shift' I wanted to get this idea out of my head and into a quick-take. I think there's something here, but a lot more to say, and I've really haven't done the in-depth research for it. There was a longer post idea I had for this, but honestly diving more than I have here into it is not a good use of my life I think. The political outlook in Silicon Valley has changed. Since the attempted assassination attempt on President Trump, the mood in Silicon Valley has changed. There have been open endorsements, e/acc has claimed political victory, and lots of people have noticed the 'vibe shift'.[2] I think that, rather than this being a change in opinions, it's more an event allowing for the beginning of a preference cascade, but at least in Silicon Valley (if not yet reflected in national polling) it has happened.  So it seems that a large section of Silicon Valley is now openly and confidently supporting Trump, and to a greater or lesser extent aligned with the a16z/e-acc worldview,[3] we know it's already reached the ears of VP candidate JD Vance. How did we get here You could probably write a book on this, so this is a highly opinionated take. But I think this is somewhat, though not exclusively, an own goal of the AI Safety movement. * As ChatGPT starts to bring AI, and AI Safety, into the mainstream discourse, the e/acc countermovement begins. It positions itself as opposite effective altruism, especially in the wake of SBF. * Guillaume Verdon, under the alias "Beff Jezos", realises the memetic weakness of the AI Safety movement and launches a full memetic war against it. Regardless of his rightness or wrongness, you do to some extent got to hand it to him. He's like right-wing Émile Torres, ambitious and relentless and driven by ideological zeal against a hated foe. * Memetic war is total war. This means nuance dies to get it to spread. I don't know if, for example, Marc Andreessen actually thinks antimalarial bednets are a 'triple threat' of badness, but it's a war and you don't take prisoners. Does Beff think that people running a uni-group session on Animal Welfare are 'basically terrorists', I don't know. But EA is the enemy, and the enemy must be defeated, and the war is total. * The OpenAI board fiasco is, I think, a critical moment here. It doesn't matter what the reasoning we've come out with at the end of the day was, I think it was perceived as 'a doomer coup' and it did radicalize the valley. In his recent post Richard Ngo called on the AI Safety movement to show more legitimacy and competence. The board fiasco torpedoed my trust in the legitimacy and competence of many senior AI safety people, so god knows how strong the update was for Silicon Valley as a whole. * As some evidence this is known in EA circles, I think this is exactly what Dwarkesh is alluding to when asked 'what happened to the EA brand'. For many people in Silicon Valley, I think the answer is that it got thrown in the dustbin of history. * This new movement became increasingly right-wing coded. Partly as a response to the culture wars in America and the increasing vitriol thrown by the left against 'tech bros', partly as a response to the California Ideology being threatened by any sense of AI oversight or regulation, and partly because EA is the enemy and EA was being increasingly seen by this group as left-wing, woke, or part of the Democratic Party due to the funding patterns of SBF and Moskovitz. I think this has led, fairly predictably, to the right-ward shift in SV and direct political affiliation with a (prospective) second Trump presidency * Across all of this my impression is that, just like with Torres, there was little to no direct pushback. I can understand not wanting to be dragged into a memetic war, or to be involved in the darker parts of Twitter discourse. But the e-acc/technooptimist/RW-Silicon-Valley movement was being driven by something, and I don't think AI Safety ever really argued against it convincingly, and definitely not in a convincing enough way to 'win' the memetic war. Like, the a16z cluster literally lied to Congress and to Parliament, but nothing much come of that fact. * I think this is very much linked to playing a strong 'inside game' to access the halls of power and no 'outside game' to gain legitimacy for that use of power. It's also I think due to EA not wanting to use social media to make its case, whereas the e-acc cluster was born and lives on social media. Where are we now? I'm not a part of the Bay Area scene and culture,[4] but it seems to me that the AI Safety movement has lost the 'mandate of heaven' to whatever extent it did have it. SB-1047 is a push to change policy that has resulted in backlash, and may result in further polarisation and counter-attempts to fight back in a zero-sum political game. I don't know if it's constitutional for a Trump/Vance administration to use the Supremacy Clause to void SB-1047 but I don't doubt that they might try. Biden's executive order seems certain for the chopping block. I expect a Trump administration to be a lot less sympathetic to the Bay Area/DC AI Safety movements, and the right-wing part of Silicon Valley will be at the very least energised to fight back harder. One concerning thing for both Silicon Valley and the AI Safety movement is what happens as a result of the ideological consequences of SV accepting this trend. Already a strong fault-line is the extreme social conservatism and incipient nationalism brought about by this. In the recent a16z podcast, Ben Horowitz literally accuses the Biden administration of breaking the rule of law, and says nothing about Trump literally refusing to concede the 2020 election and declaring that there was electoral fraud. Mike Solana seems to think that all risks of democratic backsliding under a Trump administration were/are overblown (or at least that people in the Bay agreeing was preference falsification). On the Moments-of-Zen Podcast (which has also hosted Curtis Yarvin twice), Balaji Srinivasan accused the 'Blue Tribe' of ethnically cleansing him out of SF[5] and called on the grey tribe to push all the blues out of SF. e-acc sympathetic people are noting that anti-trans ideas bubbling up in the new movement. You cannot seriously engage with ideas and shape them without those ideas changing you.[6] This right-wing shift will have further consequences, especially under a second Trump presidency. What next for the AI Safety field? I think this is a bad sign for the field of AI Safety. Political polarisation has escaped AI for a while. Current polls may lean in support , but polls and political support are fickle, especially in the age of hyper-polarisation.[7] I feel like my fears around the perception of Open Philanthropy are re-occuring here but for the AI Safety movement at large.  I think the consistent defeats to the e-acc school and the fact that the tech sector as a whole seems very much unconvinced by the arguments for AI Safety should at some point lead to a reflection from the movement. Where you stand on this very much depends on your object-level beliefs. While this is a lot of e-acc discourse around transhumanism, replacing humanity, and the AI eschaton, I don't really buy it. I think that they don't think ASI is possible soon, and thus all arguments for AI Safety are bunk. Now, while the tech sector as a whole might not be as hostile, they don't seem at all convinced of the 'ASI-soon' idea. A key point I want to emphasise is that one cannot expect to wield power successfully without also having legitimacy.[8] And to the extent that the AI Safety movement's strategy is trying to thread this needle it will fail. Anyway, long ramble over, and given this was basically a one-shot ramble it will have many inaccuracies and flaws. Nevertheless I hope that it can be directionally useful and lead to productive discussion. 1. ^ lol, lmao 2. ^ See here, here, and here. These examples are from Twitter because, for better or for worse, it seems much of SV/tech opinions are formed by Twitter discourse. 3. ^ Would be very interested to hear the thoughts of people in the Bay on this 4. ^ And if invited to be I would almost certainly decline, 5. ^ He literally used the phrase 'ethnically cleanse'. This is extraordinarily dangerous language in a political context. 6. ^ A good example in fiction is in Warhammer40K, where Horus originally accepts the power of Chaos to fight against Imperial Tyranny, but ends up turning into their slave. 7. ^ Due to polarisation, views can dramatically shift on even major topics such as the economy and national security (i know these are messy examples!). Current poll leads for AI regulation should not, in any way, be considered secure 8. ^ I guess you could also have overwhelming might and force, but even that requires legitimacy. Caesar needed to be seen as legitimate by Marc Anthony, Alexander didn't have the legitimacy to get his army to cross the Hyphasis etc.
A lot of people have said sharing these notes were helpful, so sharing it here on the EAF! Here are notes on NTI | bio’s recent event with Dr. Lu Borio on H5N1 Bird Flu, in case anyone here would find it helpful!
‘Five Years After AGI’ Focus Week happening over at Metaculus. Inspired in part by the EA Forum’s recent debate week, Metaculus is running a “focus week” this week, aimed at trying to make intellectual progress on the issue of “What will the world look like five years after AGI (assuming that humans are not extinct)[1]?” Leaders of AGI companies, while vocal about some things they anticipate in a post-AGI world (for example, bullishness in AGI making scientific advances), seem deliberately vague about other aspects. For example, power (will AGI companies have a lot of it? all of it?), whether some of the scientific advances might backfire (e.g., a vulnerable world scenario or a race-to-the-bottom digital minds takeoff), and how exactly AGI will be used for “the benefit of all.” Forecasting questions for the week range from “Percentage living in poverty?” to “Nuclear deterrence undermined?” to “‘Long reflection’ underway?” Those interested: head over here. You can participate by: * Forecasting * Commenting * Comments are especially valuable on long-term questions, because the forecasting community has less of a track record at these time scales.[2] Additionally, stakeholders we work with, who look at our questions with a view to informing their grantmaking, policymaking, etc., frequently say that they would find more comments valuable in helping bring context to the Community Prediction. * Writing questions * There may well be some gaps in the admin-created question set.[3] We welcome question contributions from users, and we will be reviewing questions in an expedited manner. The focus week will likely be followed by an essay contest, since a large part of the value in this initiative, we believe, lies in generating concrete stories for how the future might play out (and for what the inflection points might be). More details to come. 1. ^ This is not to say that we firmly believe extinction won’t happen. I personally put p(doom) at around 60%. At the same time, however, as I have previously written, I believe that more important trajectory changes lie ahead if humanity does manage to avoid extinction, and that it is worth planning for these things now. 2. ^ Moreover, I personally take Nuño Sempere’s “Hurdles of using forecasting as a tool for making sense of AI progress” piece seriously, especially the “Excellent forecasters and Superforecasters™ have an imperfect fit for long-term questions” part. With short-term questions on things like geopolitics, I think one should just basically defer to the Metaculus community prediction. Conversely, with certain long-term questions I believe it’s important to interrogate how forecasters are reasoning about the issue at hand before assigning their predictions too much weight. Forecasters can help themselves by writing comments that explain their reasoning. 3. ^ All blame on me, if so.
What is it for EA to thrive?  EA Infrastructure Fund's Plan to Focus on Principles-First EA includes a proposal: > The EA Infrastructure Fund will fund and support projects that build and empower the  community of people trying to identify actions that do the greatest good from a scope-sensitive and impartial welfarist view.   And a rationale (there's more detail in the post): >   > > * [...] EA is doing something special.  > * [...]  fighting for EA right now could make it meaningfully more likely to thrive long term. > * [...]  we could make EA much better than it currently is - particularly on the “beacon for thoughtful, sincere, and selfless” front. [...] Here I’m spending some time thinking about this, in particular: * What does it mean for EA to thrive?  * What projects could push EA in the direction of thriving?    (I work at EAIF. These are my personal views/ thoughts. I’m not speaking on behalf of EAIF here)

Popular comments

Recent discussion

huw commented on Peter Singer AMA (July 30th)

On July 30th, Peter Singer will be answering your questions in a Forum AMA. He has agreed to answer questions for an hour in the evening (Melbourne time), so if your question hasn’t been answered by the 31st, it likely won’t be. 

Singer needs little introduction for...

Continue reading

What beings are inside and outside of your moral circle these days? If your views (e.g. on insects) have meaningfully changed recently, why?

What allocation strategy in percentages do you believe would maximize the positive impact and address the most pressing needs effectively? 1.) Global health and poverty alleviation 2.) Animal welfare 3.) Climate change 4.) Longtermism (biorisk, nuclear, AI etc.) Many thanks!
Animal Liberation and Famine, Affluence, and Morality are two of the most influential texts that I have ever read. Which texts have had the most influence on you?
JWS 🔸 commented on JWS 🔸's quick take

Quick[1] thoughts on the Silicon Valley 'Vibe-Shift'

I wanted to get this idea out of my head and into a quick-take. I think there's something here, but a lot more to say, and I've really haven't done the in-depth research for it. There was a longer post idea I had ...

Continue reading

Folding in Responses here

@thoth hermes (or https://x.com/thoth_iv if someone can get it to them if you're Twitter friends then pls go ahead.[1] I'm responding to this thread here - I am not saying "that EA is losing the memetic war because of its high epistemic standards", in fact quite the opposite r.e. AI Safety, and maybe because of misunderstanding of how politics work/not caring about the social perception of the movement. My reply to Iyngkarran below fleshes it out a bit more, but if there's a way for you to get in touch directly, I'd love to clarify what I think, and also hear your thoughts more. But I think I was trying to come from a similar place that Richard Ngo is, and many of his comments on the LessWrong thread here very much chime with my own point-of-view. What I am trying to push for is the AI Safety movement reflecting on losing ground memetically and then asking 'why is that? what are we getting wrong?' rather than doubling down into lowest-common denominator communication. I think we actually agree here? Maybe I didn't make that clear enough in my OP though.

@Iyngkarran Kumar - Thanks for sharing your thoughts, but I must say that I disagree with it. I don't think that the epistemic standards are working against us by being too polite, quite the opposite. I think the epistemic standards in AI Safety have been too low relative to the attempts to wield power. If you are potentialy going to criminalise existing Open-Source models,[2] you better bring the epistemic goods. And for many people in the AI Safety field, the goods have not been brought (which is why I see people like Jeremy Howard, Sara Hooker, Rohit Krishnan etc get increasingly frustrated by the AI Safety field). This is on the field of AI Safety imo for not being more persuasive. If the AI Safety field was right, the arguments would have been more convincing. I think, while it's good for Eliezer to say what he thinks accurately, the 'bomb the datacenters'[3] piece has probably been harmful for AI Safety's cause, and things like it a very liable to turn people away from supporting AI Safety. I also don't think it's good to say that it's a claim of 'what we believe', as I don't really agree with Eliezer on much.

(r.e. inside vs outside game, see this post from Holly Elmore)

@anormative/ @David Mathers - Yeah it's difficult to manage the exact hypothesis here, especially for falsified preferences. I'm pretty sure SV is 'liberal' overall, but I wouldn't be surprised if Trump % is greater than 16 and 20, and it definitely seems to be a lot more open this time, e.g. a16z and Musk openly endorsing Trump, Sequoia Capital partners claiming that Biden dropping out was worse than the Jan 6th riot. Things seem very different this time around, different enough to be paid attention to.

-    -    -    -    -    -    -    -    -    -    -    -    

Once again, if you disagree, I'd love to actually here why. Up/down voting is a crude feedback to, and discussion of ideas leads to much quicker sharing of knowledge. If you want to respond but don't want to publicly, then by all means please send a DM :)

  1. ^

    I don't have Twitter and think it'd be harmful for my epistemic & mental health if I did get an account and become immersed in 'The Discourse'

  2. ^

    This piece from @1a3orn is excellent and to absence of evidence of good arguments against it is evidence of the absence of said arguments. (tl;dr - AI Safety people, engage with 1a3orn more!)

  3. ^

    I know that's not what it literally says but it's what people know it as

Once again, if you disagree, I'd love to actually here why.

I think you're reading into twitter way too much.

absence of evidence of good arguments against it is evidence of the absence of said arguments. (tl;dr - AI Safety people, engage with 1a3orn more!)

There are many comments responding and offering to talk. 1a3orn doesn't appear to have replied to any of these comments. (To be clear, I'm not saying they're under any obligation here, just that there isn't a absence of attempted engagement and thus you shouldn't update in the direction you seem to be updating here.)

Sign up for the Forum's email digest
You'll get a weekly email with the best posts from the past week. The Forum team selects the posts to feature based on personal preference and Forum popularity, and also adds some announcements and a classic post.

Also available on our personal blogs here (Rudolf) and here (Florence).

Reasons to make the positive case

Everyone who starts thinking about AI starts thinking big. Alan Turing predicted that machine intelligence would make humanity appear feeble in comparison. I. J. Good said that AI is the last invention that humanity ever needs to invent.

The AI safety movement started from Eliezer Yudkowsky and others on the SL4 mailing list discussing (and aiming for) an intelligence explosion and colonizing the universe. However, as the promise of AI has drawn nearer, visions for AI upsides have paradoxically shrunk. Within the field of AI safety, this is due to a combination of the “doomers” believing in very high existential risk and therefore focusing on trying to avoid imminent human extinction rather than achieving the upside, people working on policy not talking about sci-fi upsides to look...

Continue reading

“Anything worth doing, is worth doing right.”

― Hunter S Thompson


“Be not professional in what you do, rather be excellent. Excellence has life in it — it has colors in it — it has sweetness in it — whereas professionalism is a dead corpse exuding the disgusting smell

Continue reading

Yes, but pursuing excellence also costs time that could be spent elsewhere, and time/results tradeoffs are often highly nonlinear. 

The perfect is the enemy of the good. It seems to me that the most common LW/EA personality already pursues excellence more than is optimal.

For more, see my LW comment


I'm often asked about how the existential risk landscape has changed in the years since I wrote The Precipice. Earlier this year, I gave a talk on exactly that, and I want to share it here.

Here's a video of the talk and a full transcript.


In the years since I wrote...

Continue reading
Sorry for the delay! Here is a good summary of whether or not the recent warming should make us worried more: https://www.carbonbrief.org/factcheck-why-the-recent-acceleration-in-global-warming-is-what-scientists-expect/ It is nuanced, but I think the TLDR is that recent observations are within the expected range (the trend observed since 2009 is within the range expected by climate models, though the observations are noisy and uncertain, as are the models).  
It is true that this is not true for the long-form summary of the science. What I mean is that this graphic is out of the "Summary for Policymakers", which is approved by policymakers and a fairly political document.  Less formalistically, all of the infographics in the Summary for Policymakers are carefully chosen and one goal of the Summary for Policymakers is clearly to give ammunition for action (e.g. the infographic right above the cited one displays impacts in scenarios without any additional adaptation by end of century, which seems like a very implausible assumption as a default and one that makes a lot more sense when the goal is to display gravity of climate impacts rather than making a best guess of climate impacts).

Whilst policymakers have a substantial role in drafting the SPM, I've not generally heard scientists complain about political interference in writing it. Some heavy fossil fuel-producing countries have tried removing text they don't like, but didn't come close to succeeding. The SPM has to be based on the underlying report, so there's quite a bit of constraint. I don't see anything to suggest the SPM differs substantially from researchers' consensus. The initial drafts by scientists should be available online, so it could be checked what changes were made ... (read more)

From Darren Margolias: I'm the Executive Director of Beast Philanthropy, the charity founded by the world’s most popular YouTuber MrBeast. 

We recently collaborated with GiveDirectly on the video below. You can read background the project from our LinkedIn here and ...

Continue reading

When & where can I watch the video?

EJT commented on defun's quick take

Meta has just released Llama 3.1 405B. It's open-source and in many benchmarks it beats GPT-4o and Claude 3.5 Sonnet:

Zuck's letter "Open Source AI Is the Path Forward".

Continue reading

Wait, all the LLMs get 90+ on ARC? I thought LLMs were supposed to do badly on ARC.

It's an unfortunate naming clash, there are different ARC Challenges:

ARC-AGI (Chollet et al) - https://github.com/fchollet/ARC-AGI

ARC (AI2 Reasoning Challenge) - https://allenai.org/data/arc

These benchmarks are reporting the second of the two.

LLMs (at least without scaffolding) still do badly on ARC, and I'd wager Llama 405B still doesn't do well on the ARC-AGI challenge, and it's telling that all the big labs release the 95%+ number they get on AI2-ARC, and not whatever default result they get with ARC-AGI...

(Or in general, reporting benchmarks where they can go OMG SOTA!!!! and not helpfully advance the general understanding of what models can do and how far they generalise. Basically, traditional benchmark cards should be seen as the AI equivalent of "IN MICE")