Jobst Heitzig (vodle.it)
Bio
I'm a mathematician working mostly on technical AI safety and a bit on collective decision making, game theory, and formal ethics. I used to work on international coalition formation, and a lot of stuff related to climate change. Here's my professional profile.
My definition of value :
- I have a wide moral circle (including aliens as long as they can enjoy or suffer life)
- I have a zero time discount rate, i.e., value the future as much as the present
- I am (utility-) risk-averse: I prefer a sure 1 util to a coin toss between 0 and 2 utils
- I am (ex post) inequality-averse: I prefer 2 people to each get 1 util for sure to one getting 0 and one getting 2 for sure
- I am (ex ante) fairness-seeking: I prefer 2 people getting an expected 1 util to one getting an expected 0 and one getting an expected 2.
- Despite all this, I am morally uncertain
- Conditional on all of the above, I also value beauty, consistency, simplicity, complexity, and symmetry
How others can help me
I need help with various aspects of my two main projects: a human empowerment based AI safety concept, and an open-source collective decision app, http://www.vodle.it
How I can help others
I can help by ...
- providing feedback on ideas
- proofreading and commenting on texts
Posts 11
Comments56
The author is using "we" in several places and maybe not consistently. Sometimes "we" seems to be them and the readers, or them and the EA community, and sometimes it seems to be "the US". Now you are also using an "us" without it being clear (at least to me) who that refers to.
Who do you mean by 'The country with the community of people who have been thinking about this the longest' and what is your positive evidence for the claim that other communities (e.g., certain national intelligence communities) haven't thought about that for at least as long?
"targeting NNs" sounds like work that takes a certain architecture (NNs) as a given rather than work that aims at actively designing a system.
To be more specific: under the proposed taxonomy, where would a project be sorted that designs agents composed of a Bayesian network as a world model and an aspiration-based probabilistic programming algorithm for planning?
Where in your taxonomy does the design of AI systems go – what high-level architecture to use (non-modular? modular with a perception model, world-model, evaluation model, planning model etc.?), what type of function approximators to use for the modules (ANNs? Bayesian networks? something else?), what decision theory to base it on, what algorithms to use to learn the different models occurring in these modules (RL? something else?), how to curate training data, etc.?
Maybe this is true in the EA branch of AI safety. In the wider community, e.g. as represented by those attending IASEAI in February, I believe this is not a correct assessment. Since I began working on AI safety, I have heard many cautious and uncertainty-aware statements along the line that the things you claim people believe will almost certainly happen are merely likely enough to worry deeply and work on preventing them. I also don't see that community having an AI-centric worldview – they seem to worry about many other cause areas as well such as inequality, war, pandemics, climate.