Interesting thanks for clarifying, and yes your rephrasing about 'in verifiable domains' is helpful (though obviously we'd want to continue to think about how that verification or validation framework is overseen and by who). 

I think I share your concern about the human habit that will slip in of deferring to AIs and this is more of a behavioural psychology issue that we need to address in terms of how humans relate to AI which probably needs interventions on the training, skills and capabilities side of thing across education more widely. Obviously far greater than the domain of animal rights you're considering here but it's a useful live example to think some of this through. 

I would encourage you and others to not give up too easily on keeping humans in the loop, indeed! But I do think this debate needs a dedicated container to hold it. 

Hi all, introducing myself here: I'm Eleri (or Elli) Thomas.

I crossed paths with early EA and 80,000 Hours while at Oxford (graduated 2012), though I haven't been active in the community since. Since then I've had a rather varied, interdisciplinary career as an urbanist (trained in history, urban design and town planning, and worked across the built environment professions, economics, infrastructure, smart cities, and policy, design, development and strategy). I've been at think tanks (Centre for Cities), advised government/local authorities/developers (Design Council), held various leadership roles in local government, and co-founded a community-led cohousing company.

I'm now an independent consultant, researcher, facilitator and writer (currently at thinkingthingsthrough.substack.com) focused on strategic design and systems change with a focus on helping enable better collective decisions, especially in complex, fast-changing environments and where there are complex bureaucratic or regulatory contexts or legacies. I've worked with local and central government, a handful of forward-thinking consultancies, and various nonprofits and foundations.

Since going self-employed a year ago, I've had more time to write and engage with the wider field, and I'm really keen to maximise the impact of the insight that I have gained from working on the bridge between research, policy and practice. I'm particularly focussed currently on how humans (and machines) cooperate effectively, how institutions cultivate or undermine collective intelligence, and how psychological, emotional and cultural dynamics shape our capacity to understand and act well.

I'm keen to learn from and see how the EA community is thinking about policy and governance, and on AI development and alignment; here in particular my approach is likely to be more human-centred and design/innovation-oriented than the classic rationalist mindset, so I'm genuinely curious how that friction or complementarity shows up in practice.

In September–October I'll be resident at CEEALAR in Blackpool, so taking the opportunity to re-enter the EA world here. :)

Surely the most effective way of making change is to influence and in fact directly infiltrate the means of change i.e. through policy and policymaking. 

I've been working across research/policy and practice for the last 12 years, and yes it's incredibly difficult to build relationships, trust and traction but I think that's more of a reason to actively engage with it rather than to turn away. I've been particularly interested in the energy in the UK from the "YIMBY" movement, Looking for Growth are a good example of this, and the attitude of "you can just do things" - seeing an increasing number of people directly engage with local decision making, becoming councillors, joining boards to directly influence. 

Of course it does critically involve realising that it's not sufficient to be "right", you also need to be able to convince others of that and fundamentally to be able to listen to them - that's not easy but without that there no hope, other than through some kind of autocracy... 

The part I'm most curious about here is where you say that "keeping humans in the loop will only degrade decision-making."

I'm fairly new to the AI alignment debates, so I'm not sure how widely accepted this assumption is (and I'd love to be challenged on this) but if it is, I find it quite concerning;

It seems reasonable to design systems that can help humans make better-informed decisions, and to act autonomously where that's appropriate, but that's quite different from concluding that humans should be removed from morally significant decision-making because they make decisions "worse."

I think the distinction we need to make is between cognitive reasoning and moral judgment. I can imagine AI becoming vastly better than humans at reasoning about the consequences of different actions, but that doesn't mean we're deciding that because of AI's advanced cognitive reasoning it should determine the objectives or values being optimised in the first place. It could, of course, help us to identify places where our values or objectives might be out of alignment, but ultimately it's a human decision about whether we agree with that, and those checks and balances should be in there throughout. It's a matter of how we design the interface of AI and human systems as to where humans should be in the loop. That also critically means that AI is then providing sufficient information and evidence back to humans in the process to enable them to update these values based on adequate information.

This kind of all points to the conundrum you raised about it not seeming sufficient to just set values today, or align a model to how we currently make decisions on animal welfare, and assume it will work for the future - I think you already seem pretty clear that it won't be, so then the question is how we DO keep humans in the loop to ensure that.