Quick takes
Open invitation: bet that the AI bubble won’t pop
I’m taking my second stab at formulating this AI bubble bet in a way that is attractive for people who might want to take the other side of it. I’m betting the AI bubble will pop within about 7 years. I’m looking for people who are willing to bet it won’t pop, or that there is no AI bubble.
I’m open to proposed modifications.
Stakes:
$20 to the charity of the winner’s choice.
Resolution criteria:
By January 1, 2034, all of the following news outlets will report — as fact, not opinion or editorial — that the AI bu...
It occurred to me what a drag it would be for the other person in this bet to wait until January 1, 2034 to win. I thought of a few win conditions that could call the bet in the other person's favour before the clock runs out. If any one of these conditions are satisfied, the other person would win the bet:
- Either Anthropic or OpenAI (post-IPO) reports $100 billion in annual GAAP profit (net income) or $25 billion in quarterly GAAP profit (these would have to be audited financials reported to the SEC, like public companies typically do)
- Ray Kurzweil wins
EA is (again) having a moment:
- Twitter is all in a buzz about it, arguing about insect lives being worth more than human lives
- It is the cover of The Economist with the headline "The century’s biggest idea"
- The US Defense Department tweeted "Americanism, not effective altruism"
- ...and so on.
The usual suspects are in the trenches fighting the good fight, but this is a great time for EAs to take some comms action. An example I particularly loved was @Andres Jimenez Zorrilla 🔸 's post on LinkedIn and Twitter: https://www.linkedin.com/posts/andres-jim...
Help The Unjournal prioritize AI economics and governance research
(https://uj-prioritization-dashboard.netlify.app/ai-governance/)
The Unjournal (https://info.unjournal.org/) is prioritizing research on AI economics, governance, social and economic impacts, and risks from increasingly capable AI. (NB: We're not covering technical/CS/ML/AI-safety research.). We want to commission expert public evaluations and help synthesize, disseminate, and curate this work.
Much of this research seems 1) important/impactful/influential, (2) involves considerable 'firepower...
I'm currently hiring. Come and help me address the suffering of animals. At Rethink Priorities:
Senior Research Manager – To lead a new Animal Welfare Strategy Team. This team will focus on supporting and accelerating funders and organizations toward achieving accelerating the end of low-welfare farming. The team lead will have to manage a team to do consulting for animal organisations, in-house research projects, and coordinate the Animals Advocacy strategy Forum. Salary: $98,500–$125,000. Deadline: Monday 19 October.
https://careers.rethinkpriorities.org/e...
Markov utilitarianism as universalized prudence
I recently proposed a view I call Markov utilitarianism. To me, it feels like a natural way to think about ethics. I came to it after thinking about how I care about my future, and how I really care about how I feel right now, from the first-person perspective.
- I care about my actual subjective futures, weighted by how likely I am to find myself in each, rather than the physical futures someone looking from a third-person perspective implicitly treats as relevant to me. (Physical futures and my subjective futur
I made a video on pandemic prevention through wastewater monitoring, featuring @Jeff Kaufman 🔸 and @Will Bradshaw from SecureBio Detection.
It has been 2 years in the making (with several months-long breaks) and I'm very proud of it! Hope you enjoy it. More details here: https://forum.effectivealtruism.org/posts/gfRPh2RfAhLsBv37s/wastewater-monitoring-as-an-early-warning-for-pandemics-a
It's bad for our mental health, but it's particularly terrible for optics. I've been critical of our focus on optics in the past, but like it or not, our movement has become very public facing now that AI policy has become such an important mechanism for AI alignment. Optics matter far more than they used to. It's not ideal to air our dirty laundry in the most public possible way in a place where salacious drama and outright lies based on it easily go viral due to the algorithm and bad faith actors.
I don't think discussing it on twitter should be discouraged if the individual thinks it's the best place for it, but making discussing it on the EA Forum less frowned on would avoid boxing people into that choice.
cw rape
Context:
- A blogger (Worst Boyfriend Ever) who writes accounts of rapes went to Manifest Summer Camp. He was kicked out
- Austin Chen offered him a ticket to manifest, then rescinded it
- I looked into Worst Boyfriend Ever. There are some 3rd hand accounts of sexual assualt. He claims his stories are true except for the illegal bits
- I said he was probably a rapist. I stand by it.
- People from near me on twitter argued that actually he wasn't, or that it was just a kink, or whatever, whatever, whatever
- Ugh. Horrible.
I like to call hits and misses. And so, while ...
Thoughts on the Centre for Effective Altruism, weakly held, n = 1, but feeling like things someone should say:
- EAGs seem good
- Community Health seems better than replacement. It's a hard job, not perfect, but I am glad someone is doing it.
- the Forum seems like not a place I want to be despite having previously been here a lot. Hard to make sense of what's important, somehow
- I rarely see comms work I think is good. Maybe it's in places I don't see, but on twitter, is does EA have a reputation that is helped by CEA? The odd thread I guess?
- Vision? I dunno. Is ther
Why do I keep meeting so many damned capabilities researchers and AI salespeople?
I thought that we agreed capabilities research was really bad. I thought we agreed that increasing the amount of economic activity in capabiliities was really bad. To me it seems like the single worst thing that I could even do!
This really seems like a pretty consensus view among EA orthodoxy. So why do I keep meeting so many people who, as far as I can tell, are doing the single worst thing that it's even in their power to do? If there is any legal thing that coul...
Short version: you were significantly closer to the correct position than I was.
However, the position that would have been more correct than either of us at the time is "Wait, what do we even mean by alignment research and capabilities research? Can we even draw a clear line between them in order to figure out who to ostracize? If they're so similar, then how can we justify thinking that one is extremely good and the other extremely bad?"
Such thinking could have led us to become much less confused about weird dynamics in alignment. I started doing that thi...
What I'd want a fiscal sponsor to do well (also, some generic principles for ops)
EDIT: This is now a top-level post so that it's easier to find and share.
What if exact copies don't matter but future experiences still do?
I think an ethics resulting from the following two premises is worth exploring:
P1: The total moral value of subjectively indistinguishable observer-moments doesn't scale with the number of copies, i.e. they sum to a constant value.
- This is because subjectively indistinguishable observer-moments, in some sense, happen to the same "someone". One could think of themselves as all their copies. N observer-moments that are subjectively indistinguishable aren't felt N times by that same "someone". O
Decided to call it "Markov utilitarianism"
Not because it sounds cooler (it does) but because it is a utilitarianism and it does contrast with the other views by being an MRP.
With all the recent media attention on EA, I think what's really unclear to outsiders is the relationship between EA and Anthropic (rightfully so!). To most people it doesn't make sense that a community so focused on AI safety can also be linked to one of the leading companies. I'd personally be very interested to see a survey of what EAs think of Anthropic, but maybe that's a minefield.. Even a good explainer meant for the public could go a long way.
Ideally such comms would also
- Clarify how much of the community is actually focused on AI Safety vs other cause areas. I assume the AIS take on Anthropic different from the GHD side.
- Distinguish between Open AI and Anthropic, two orgs that both had ties to the EA movement upon founding but have very different reputations in the community at present.
Three big unaddressed problems in EA 2.0:
- corruption (i.e., the quid pro quo between FTX and Manifund, in which Austin Chen traded Caroline Ellison a job out of prison for cash obtained in the course of stealing customer funds)
- cults (e.g., Leverage Research and Dragon Army Barracks, whose former members/leaders such as Tyler Alterman and Duncan Sabien still post on the EA Forum)
- misinformation (e.g., METR's use of fabricated data in the famous time horizons graph)
EA sadly seems sunk to me because dire problems keep mounting and most people seem to thin...
Start with journals: a cheap governance test for AI animal-communication tools
A recent Guardian feature on AI and animal communication brought renewed attention to the PEPP Framework (Prepare, Engage, Prevent, Protect), developed by NYU’s More Than Human Life Program for the responsible development and use of nonhuman animal communication technologies.
PEPP is quite ambitious. Its 12 principles cover risk assessment, precaution, animal autonomy and best interests, transparency, responsibility, remediation, and other issues that could become increasingly imp...
I suspect that the way most people (not just people with high leverage) interact with LLMs today is more relevant to catastrophic risk than it first appears to be.
- I worry that the way most people interact with LLMs today is writing the script for how we will interact with increasingly intelligent AIs going forward.
- If ~ASI is close, we should be identifying a rough optimal idea of what a human’s relationship with an LLM should look like (eg preserving critical thinking, thinking things through oneself, aiming to understand something rather than have the LLM
Please list any new funding opportunities you can think of here on the Forum? I feel like we might already be in the early ramp-up to significantly more EA aligned funding. At the same time, the Forum's overview over funding opportunities feels like it is quickly getting outdated. I think as things move quickly, coordination might become looser and new promising interventions are identified, it is helpful for people to have a good overview over available funding sources and their priorities.
I have heard on the grapevine there is already funding on several ...
Developing Thoughts on the Animal Advocacy Community in Africa
African animal advocacy movement may be under-exploring its potential and is NOT talent-constrained. Instead, the movement is currently held back by the following:
Lack of strategy R&D: This stems from two sources: limited exploratory, outside-the-box thinking, and funding habits that do not reward thorough research and the development of relevant interventions.
Lack of space for innovation: Few, if any, African animal advocacy organizations have a dedicated special or new projects team, becau...