https://80000hours.org/speak-with-us/

I won't be going to EAGxBerlin but I do have some questions. I have attended a workshop with Bob Brown, the former leader of the Australian Greens. The workshop was about building up a network of grass roots lobbyists. Lobbying was framed primarily in terms of having one on one meetings with politicians and communicating your concerns, (in this case, environmental concerns, of which stopping native forest logging is among the highest priorities), in ways that connected with their interests. 

My question is therefore, how much of lobbying is about these one on one appointments with politicians?

You also talk about organising workshops for civil servants. I'd be interested to hear more about how that came about. In these one on one conversations did you communicate the need for these workshops, and then eventually they agreed? That seems quite impressive that you could have a top down effect like that across government. 

Can talk virtually if that's easier.

My understanding of their claim is more something like:

There will never be an AI system that is generally intelligent like a human is. Put differently, we can never have synoptic models of the behaviour of human beings. Where a synoptic model is a model that can be used to engineer the system or to emulate its behaviour.

This is because, in order for deep neural networks to work they need to be trained on data with the same variance as the data to which the trained algorithm will be applied. I.e. training data which is representative. General human intelligence is a complex system which doesn't have said distribution. There are mathematical limits on the ability to predict the behaviours of a system like this that does not have a distribution. Therefore whilst we can have gradually more satisfactory simple models of human behaviour (like chatGPT is for written language) they will never the same level as humans.

To put it simply: We can't create AGI by training algorithms on datasets, because human intelligence does not have a representative data set.

As such I think your response "It is possible to create new things without fundamentally understanding how they work internally" misses the mark. The claim is not that "we can't understand how to model complex systems therefore they can't be adequately modelled". It's more something like "there are fundamental limits to the possibility of adequately emulating complex systems (including humans), regardless of whether we understand them more or not". 

My personal take is that i'm unsure how important it is to be able to accurately model human intelligence. Perhaps modelling some approximate of human intelligence (in the way that chatGPT is approximately good enough as a written chat bot) is sufficient enough to stimulate the creation of something more approximately intelligent than that, and so on. In the same way that chatGPT can answer PhD level questions that the majority of humans cannot. 

Note: My understanding of Barry's argument is limited to this lecture (https://www.youtube.com/watch?v=GV1Ma2ehpxo) and this article (http://www.hunfi.hu/nyiri/AI/BS_paper.pdf). 

I think I might be missing what’s distinctive here. A lot of the traits listed — strong knowledge of the field, engagement with the community, epistemic humility, broad curiosity — seem like general predictors of success in any many fields.

Are you pointing at something that’s unusually clustered in EA, or is your claim more about how trainable and highly predictive this combination is within the EA context?

Have you had a chat with the 80k hours team?

Yes I agree it's bad to support a race, but it's not as simple as that.

Is OpenAI going for profit in order to attract more investment/talent a good or bad thing for AI safety? 

On the one hand people want American companies to win the AGI race, and this could contribute to that. On the other hand, OpenAI would be then be more tied to making profit which could conflict with AI safety goals.

It seems to me that the limiting step here is the ability to act like an agent. If we already have AI that can reason and answer questions at a PhD level, why would we need reasoning and question answering to be any better?

The point is, there are 8.7 million species alive today, therefore there is a possibility that a significant number of these play important, high impact, roles.

I have the opposite intuition for biodiversity. People have been studying ecosystem services for decades and higher biodiversity is associated with increased ecosystem services, such as clean water, air purification, and waste management. Higher biodiversity is also associated with reduce transmission of infectious diseases by creating more complex ecosystems limiting pathogen spread. Then we have the actual and possible discovery of medicinal compounds and links with biodiversity and mental health. These are high level examples of the benefits. The linked article gives the possibility of impact by considering two effects from bats and vultures. Multiply that effect by 1000+ other species, include all the other impacts previously mentioned and I can see how this could be high impact.