Been thinking about morality recently. Here are my current thoughts, take them with a grain of salt because they aren't battle-tested yet.

There are some strong arguments for utilitarianism, but regardless of what is correct theoretically, in practise utilitarianism doesn't work well without some kind of deontological bars.

Continuing with attempting to develop a pragmatic morality, it then become clear that virtue ethics is important too because a) rules are rigid compared to judgement b) decisions aren't independent but also affect how you'll act in the future^[1].

Some folks may be quite tepid in integrating virtue ethics, but my intuition is that the more common fault will be to give yourself too much latitude, so you'll probably want to revive some of your old deontological bars.

I view the next stage after this as introducing a sort of meta-virtue ethics to balance the three components (utilitarianism, deontology and virtue ethics; obviously it would be possible to break this down further). But this likely gives you too much latitude again, so you'll probably want to introduce some kind of meta-deontology to limit how you update the balance.

You could go further than this, but you'd probably be running into decreasing marginal utility.

1. ^{^}

   Thanks to Austen Erickson who I first learned this perspective from.

I assume this is going forward and there's no requirement to backlabel?

One thing that's unclear to me is whether attempts to use AI systems to augment human capabilities in these domains is in-scope or whether the round is focused on direct enhancement of these capabilities.

I'm also curious about your opinion on whether biological-enhancement based approaches are likely to bear fruit in time to matter. Do you think it's plausible that timelines might be long on our current path or are you more hoping that there's a pause that provides humanity with more time?

(Alternatively, is it more that you think that we need enhanced capabilities to succeed at alignment even if current timeline projections makes this appear challenging?).

I suspect that the main use of forecasting is if you need a probability for something and you don't really have time to look into it yourself or you wouldn't trust your judgement even if you did.

I probably should have clarified that EA as a community building effort isn't drawing the same talent. That said, talent that was attracted through community building efforts earlier has had more time to "level up" (as you mentioned) and orgs have likely improved their ability to recruit experienced professionals directly compared to the past (though my intuition is that some orgs haven't fully appreciated the costs of weakening value-alignment since these impacts take a long time to emerge).

A further caveat: I don't know exactly how things are at elite universities these days.

For fairness, I'll just add a comment that the following edits were made after the competition deadline:

• "The first two are based on this post"
• "If doing what worked in other cities worked, it would have already worked"
• "—but it synergises nonetheless"

I made some additional edits after it was announced this essay came in second for this question.

I'll PM you and post my comment publically later. I'm curious if anyone else makes similar points if I don't do so.

"Every entity in the ecosystem gets a living profile on the platform" - Given that this has a high probability of having have ecosystem wide impacts, how much analysis have you conducted of the downsides of doing this? Are you willing to share this analysis?

I notice myself feeling negative towards these laws.

This isn't even an absolute rejection on my part, I'm just worried that governments are sliding too much towards paternalism without properly considering the downsides of their legislation.

Sounds, like it's a recommendation, not a rule?