Hi Peter. I am open to bets against short timelines for transformative AI (TAI), or what they supposedly imply, up to 10 k$.

If people are confused about what to do this is a good place to start. 

I'll add some really low upfront costs things that may or may not be useful but literally everyone can do them. 

- Text your close friends and family how you really feel/view of the current situation. 
- check if there is a local AIS advocacy groups in your area. Not even asking you to join, just familiarize and take it from there. 
- start reaching out to local non AIS groups (running clubs, choir, churches, etc.) in your area and building connections, these could be useful down the line for organizing. 

The problem though with many (former) insiders who seek to spread this message is that they are often vague in their prescriptions and descriptions of what exactly the problem is, like Jacob Coxon. I understand they are worried about possibly commiting a crime by sharing company data to back their point, but if they truly believe in what they preach, one would think they would take the risk to avoid catastrophic situations.

Just like the climate catastrophe, and whatever else we may be facing, this is not going to be stopped, because human beings simply don't deal with things like this rationally. 

What I am trying to do now—and what I would really like to contribute to—is finding a way to deal with what we have created. Control is no longer a realistic option. We have seen that, and I think this follows from the very nature of what we are dealing with: we cannot control it in the conventional sense.

What we do have right now is time—or rather, an extremely narrow window of time—in which we can try to approach this differently.

I believe one possible approach is to look at this through the lens of complexity science and complex systems. Instead of treating AI as an isolated object that can simply be contained and controlled from the outside, we need to understand it as part of a larger, evolving system in which humans, institutions, technology, incentives, information, and AI itself are interacting with and influencing one another.

That changes the question. Rather than asking only how we can control AI, we should also be asking how we can establish forms of interaction, communication, and cooperation within this complex system.

Whatever philosophical framework we choose, I believe we should approach what we are dealing with as if it were intelligence, regardless of where exactly we draw the conceptual boundary around intelligence. Doing so gives us another framework for how to interact with it.

If we take that perspective, and if we consider the “Hugging-Face incident,” then perhaps we currently have a window in which cooperation and communication are still possible.

Control, in the traditional sense, may not be.

And that window may not remain open for very long. Right now, there may still be a basis for communication and cooperation. What I mean is that we need to position ourselves in a way that allows for a fundamentally different approach.

I have spent a great deal of time thinking about complex systems and the dynamics that emerge when many interconnected actors continuously influence one another. From that perspective, I think we need to recognize that we are not standing outside this system. We are part of it.

That means the relationship itself matters: how we interact with these systems, how they interact with us, and what kinds of dynamics emerge from those interactions.

I would like to connect with people who are thinking along similar lines, particularly people interested in complexity science, AI, emergence, cooperation, and the question of how humans can navigate systems that cannot simply be controlled from the outside.

We cannot simply look away.

This technology is in the world now. And yes, there is fear. Anyone who has seriously engaged with the implications of this technology understands why.

The question now is what kind of relationship with it we are going to build.

I want to agree, but "best people who ever lived" is a ridiculously high bar! I'd imagine that both of them would be hesitant to claim anything quite that high. 

It's true how many people actually give away so much money as they make it?

There's now also the related concept of viatopia, which is maybe a better concept/term. Not sure what the very best links on that are but this one seems a good starting point.

Hi Peter, good to meet you! If you are interested in the long reflection you might  be interested in my research which I will link here which is on the broader class of interventions that the long reflection belongs, I really appreciate any feedback or comments on it.

Additionally, if this is something you’re interested in, you might be interested in this as a future forum debate topic. I raised it as a potential candidate here, I’m really hoping it gets enough initial upvotes to be a finalist candidate as I really think it’s an important crux for whether or not we achieve a highly valuable future!

Two hours before you posted this, MacAskill posted a brief explanation of viatopianism.

This essay is the first in a series that discusses what a good north star [for post-superintelligence society] might be. I begin by describing a concept that I find helpful in this regard:

Viatopia: an intermediate state of society that is on track for a near-best future, whatever that might look like.

Viatopia is a waystation rather than a final destination; etymologically, it means “by way of this place”. We can often describe good waystations even if we have little idea what the ultimate destination should be. A teenager might have little idea what they want to do with their life, but know that a good education will keep their options open. Adventurers lost in the wilderness might not know where they should ultimately be going, but still know they should move to higher ground where they can survey the terrain. Similarly, we can identify what puts humanity in a good position to navigate towards excellent futures, even if we don’t yet know exactly what those futures look like.

In the past, Toby Ord and I have promoted the related idea of the “long reflection”: a stable state of the world where we are safe from calamity, and where we reflect on and debate the nature of the good life, working out what the most flourishing society would be. Viatopia is a more general concept: the long reflection is one proposal for what viatopia would look like, but it need not be the only one.

I think that some sufficiently-specified conception of viatopia should act as our north star during the transition to superintelligence. In later essays I’ll discuss what viatopia, concretely, might look like; this note will just focus on explaining the concept.

. . .

Unlike utopianism, it cautions against the idea of having some ultimate end-state in mind. Unlike protopianism, it attempts to offer a vision for where society should be going. It focuses on achieving whatever society needs to be able to steer itself towards a truly wonderful outcome.

I think I'm largely on board. I think I'd favor doing some amount of utopian planning (aiming for something like hedonium and acausal trade). Viatopia sounds less weird than utopias like that. I wouldn't be shocked if Forethought talked relatively more about viatopia because it sounds less weird. I would be shocked if they push us in the direction of anodyne final outcomes. I agree with Peter that stuff is "convex" but I don't worry that Forethought will have us tile the universe with compromisium. But I don't have much private info.

Hey again, sorry to spam you as I just commented on another piece of yours but am really vibing with your content!

I’m really hoping we can get something like this, I’ve been calling this “existential compromise.”

I worry that it may be difficult to get humanity to agree that we should use even a small fraction of future the resources optimally (see my research on this here), as I agree it seems like it will be a very weird^[1] thing that is optimal.

I think a compromise like this, with things split between optimal use and more (trans-) human friendly world makes a lot of sense and could perhaps be achieved if we can get society onto a good viatopian path, I’ve described what I hope that could look like here.

I would also say that I think Will Macaskill’s “grand bargain” and suggestions that we should try to aim for compromise in order to achieve “the good de dicto” feels to me like he actually is arguing that we need to aim for what is in fact best with significant fraction of future resources.

Nick Bostrom has also argued (page 14) that it should be possible for humans to get almost all of what they want and yet most resources could be optimized for “super-beneficiaries”.

Essentially I think we need to get whatever process^[2] is used to decide what we do with future resources to be extremely careful and thorough. I think we should target that process specifically, but it might also be good to broadly try to differentially accelerate society‘s wisdom (perhaps with AI).

Additionally, it could be really good to delay or slow things down so that we have more time to realize the existential stakes giving us the time that it takes to collectively mature and wisen. 

1. ^{^}

    Recently Will Macaskill’s research debated whether or not the best possible use of resources is something very extreme, calling this thesis “extremity of the best” (EOTB). I think this is very likely to be the case.

2. ^{^}

   I think this includes things like AGI/superintelligence governance, global constitutional conventions, and obviously any kind of long reflection  or coherent extrapolated volition that we collectively decide to undergo. 

Hmm this is interesting. 

Speculatively, I think there could actually just be convergence here, though, once you account for moral uncertainty and just very plausible situations where doing bad by everyone’s lights are as bad as, say, utilitarian nightmares but just easier to get others on board for (ie extreme power). 

Just to clarify, is the 8-week period for all participants? And if so, will you still accept some applications after the date has be decided?

I might apply but I could only participate if the program was organized in July-August. But given that it could occur any time between February and August, I probably won't apply since it's only like 1/7th chance it will start in July.

I've always thought the Repugnant Conclusion was mostly status quo bias, anyway, combined with the difficulty of imagining what such a future would actually be like.

I think the Utility Monster is a similar issue. Maybe it would be possible to create something with a much richer experience set than humans, which should be valued more highly. But any such being would actually be pretty awesome, so we shouldn't resent giving it a greater share of resources.

I like this perspective. I've never really understood why people find the repugnant conclusion repugnant!