A center applying epistemic best practices to predicting & evaluating AI progress
Artificial Intelligence and Epistemic Institutions
Forecasting and evaluating AI progress is difficult and important. Current work in this area is distributed across multiple organizations or individual researchers, not all of whom possess (a) the technical expertise, (b) knowledge & skill in applying epistemic best practices, and (c) institutional legitimacy (or otherwise suffer from cultural constraints). Activities of the center could include providing services to AI groups (e.g. offering superforecasting training or prediction services), producing bottom-line reports on "How capable is AI system X?", hosting adversarial collaborations, pointing out deficiencies in academic AI evaluations, and generally pioneering "analytic tradecraft" for AI progress.
I will argue that despite the fact that there is overlap, and many of the ideas are well known, the knowledge and experience of policy analysis has much to offer effective altruism in achieving the goals of improving the world. Not only that, but it offers a paradigm for how to reasonably pull from multiple disciplines in helping make decisions - exactly what this series of posts is trying to help with.
Did you ever end up writing up those thoughts? I skimmed the rest of the posts in the series but didn't find it.
I don't think I quite follow your criticism of FLOP/s; can you say more about why you think it's not a useful unit? It seems like you're saying that a linear extrapolation of FLOP/s isn't accurate to estimate the compute requirements of larger models. (I know there are a variety of criticisms that can be made, but I'm interested in better understanding your point above)
Thanks for the great post. Ryan, I'm curious how you figured this at an early stage:
I figured that in the longer term, my greatest chance at having a substantial impact lay in my potential as a researcher, but that I would have to improve my maths and programming skills to realize that.
Nice work! Are there plans to share the source code of the model?
I'd also be interested in funding activities like this. This could inform how much we can learn about models without distributing weights.
A center applying epistemic best practices to predicting & evaluating AI progress
Artificial Intelligence and Epistemic Institutions
Forecasting and evaluating AI progress is difficult and important. Current work in this area is distributed across multiple organizations or individual researchers, not all of whom possess (a) the technical expertise, (b) knowledge & skill in applying epistemic best practices, and (c) institutional legitimacy (or otherwise suffer from cultural constraints). Activities of the center could include providing services to AI groups (e.g. offering superforecasting training or prediction services), producing bottom-line reports on "How capable is AI system X?", hosting adversarial collaborations, pointing out deficiencies in academic AI evaluations, and generally pioneering "analytic tradecraft" for AI progress.
On policy analysis, you write:
Did you ever end up writing up those thoughts? I skimmed the rest of the posts in the series but didn't find it.
I don't think I quite follow your criticism of FLOP/s; can you say more about why you think it's not a useful unit? It seems like you're saying that a linear extrapolation of FLOP/s isn't accurate to estimate the compute requirements of larger models. (I know there are a variety of criticisms that can be made, but I'm interested in better understanding your point above)
How'd you decide to go focus on going into research, even before you decided that developing technical skills would be helpful for that path?
Thanks for the great post. Ryan, I'm curious how you figured this at an early stage:
What key metrics do research analysts pay attention to in the course of their work? More broadly, how do employees know that they're doing a good job?
Luke Muehlhauser posted a list of strategic questions here: http://lukemuehlhauser.com/some-studies-which-could-improve-our-strategic-picture-of-superintelligence/ (originally posted in 2014).
By (3), do you mean the publications that are listed under "forecasting" on MIRI's publications page?