Introduction
TLDR: If solutions to the world’s most pressing problems require people with scarce, expensive skills then we should seek to discover and implement cost effective ‘force multipliers’ that maximise the productivity of these people.
With experience developing entrepreneurial technology products, I’m in a position to do just that. I describe some potential opportunities and ask for feedback and introductions.
Hello, I’m Craig from Melbourne, Australia. I’ve been leading the development of technology-based products for 14 years. I’ve co-created multiple products alongside academic staff and have spent several years in early startups, including as a solo founder for a data analytics company.
Most recently I led the development of a distributed data infrastructure and application development platform for climate science. The platform aims to make developing, operationalising and distributing climate solutions radically faster and easier.
I discovered EA back in 2010 and have quietly practised effective giving since. I’m moving on from my current work and want to discover a new opportunity to apply my skills to.
In this post I lay out an opportunity space with the aim of eliciting feedback and finding the right people to speak to next. I currently have only a nascent understanding of EA research areas, so please forgive (and gently correct) any naive errors.
The opportunity
Of the ten priority career reviews written by 80,000 hours, six emphasise gaining a technical/quantitative PhD from a top university:
(To be clear, 80,000 hours does not argue that getting a quantitative PhD from an elite university is the only way to contribute to AI safety or other pressing problems.)
Developing important skills for addressing the world’s most pressing problems can be slow and difficult:
“Most of [our priority paths] are difficult to enter — you may need to start by investing in building skills for several years, and there may be relatively few positions available.”
Other than encouraging early career EAs to pursue these paths, how else could we relieve specialist skills gaps and accelerate progress in these areas?
1. Increase the productivity of existing researchers
Increasing the productivity of longtermist researchers may be significantly easier, cheaper and faster than increasing the number of researchers, while providing equivalent value.
Collecting, cleaning and organising data can take up a significant proportion of data scientists’ time. Project-based funding models, fragmented data licensing and contract negotiation inhibits collaboration and slows down progress. There’s often a significant gap between promising research and operational solutions.
Researchers have spent years developing a deep specialisation. But in my experience, they often approach their work as generalists; spending much of their time on tasks for which they have no comparative advantage.
Longtermist research is still relatively new and niche, so I’d expect the ecosystem of supporting data, tooling and complementary roles to be limited. Given the expense and difficulty of developing new researchers, investing in their supporting ecosystem may be a cost effective way to increase overall productivity - or at least not be subject to currently central constraints.
2. Increase supply of skilled work by reducing barriers to entry
People with doctoral degrees from elite universities make up just a fraction of a percent of the global population. Expanding the potential pool of top contributors beyond this demographic could help resolve critical skills gaps and increase the rate of progress. If it remains, say, 30 times easier (and much more lucrative) to become a commercial machine learning engineer than to work on AI safety, it is hard to see how safety can keep pace with capabilities.
Improving the ecosystem of infrastructure, tooling and data available to longtermist researchers and engineers could make working in these areas more approachable and desirable. This may increase the pool of motivated and talented people wanting to work on these issues.
Potential approaches
Benchmark datasets
Several benchmark datasets have been cited more than 100,000 times in machine learning research, and have also been used in operational solutions. Examples include datasets containing handwriting (e.g. MNIST), images of objects (e.g. ImageNet), audio clips (e.g. AudioSet) and product reviews (e.g. IMDB). Having pre-prepared datasets makes novel research significantly easier and faster. They also enable comparable results between different approaches and techniques.
Data collection efforts can be long running and extensive, worthy of an independent effort. The GroupLens project has collected 25 million movie ratings by operating an IMDB-like website since 1995. Common Crawl (with just two staff) creates and maintains petabytes of open web crawl data, which were used to train GPT-3.
Some examples relevant to longtermist research exist already (e.g. DeepMind’s AI Safety Gridworlds). There will be many more valuable opportunities for datasets and environments. What if we had quality, fine-grained data on millions of people’s espoused values, their perceptions of consequences, or retrospective evaluations of their past decisions?
Versions of this opportunity have been described by these EA Future Fund project ideas winners and by 80,000 hours here.
Tooling (including libraries, infrastructure and platforms) has the potential to accelerate research by speeding up, eliminating, or improving the quality of the work undertaken.
Global cloud providers and research infrastructure provide cheap and scalable generic inputs such as compute and storage. Specialised platforms build on top of these to provide domain specific capabilities (see EcoCommons, an upcoming ecological research platform). Other opportunities could lie in coordination, crowdsourcing and resource allocation (e.g. “Kaggle for AI safety”).
Technical Standards
Since 1995, the Coupled Model Intercomparison Project (CMIP) has provided a framework and standards for running climate modelling experiments in a way that enables contributors to independently produce comparable results. The latest iteration, CMIP6, contains contributions from 49 different modelling groups and more than 1000 individual scientists.
Standards (including practices, ontologies and definitions) provide an effective way for many parties to contribute to a greater effort without the overhead of close coordination.
Standards could potentially increase collective productivity of longtermist research, and also be a method for implementing interventions (e.g. AI safety and biosecurity standards).
The ask
Entrepreneurial product development seems to be a relatively neglected skillset in the EA community (outside earning to give). I believe promising high-impact opportunities exist that would be a good fit for my experience, and I’d like to work on them. I’m open to either joining an existing organisation or starting a new one.
Here’s how you can help.
I need to talk to people! Especially people who:
- are doing high-leverage technical/data heavy EA work, and would be happy to describe their work and its challenges
- are interested in working on this topic - discovering, creating and operating an impactful technology ‘force multiplier’
- would be interested in providing seed funding for an opportunity in this space
If you are one of these people, or know someone who is, please leave me a comment or direct message.
I am also seeking feedback. What already exists, what’s been tried before, which approaches seem more/less promising to you and why?
Personally, I'm a huge fan of exploring ways of improving research efficiency and quality, and I am glad to see another person working on this.
Of potential relevance: I'm currently experimenting with an approach to supporting AI-relevant policy research via collaborative node-and-link "reality modeling / claim aggregation, summarization, and organization" (I'm not exactly sure what to call it).
For example, questions/variables like "How much emphasis would the Chinese government/Chinese companies put on (good) safety and alignment work (relative to America)" and "would more research/work on interpretability would be beneficial for powerful-AI/AGI alignment" seem like they could be fairly important in shaping various policy recommendations while also not being easy to evaluate. Additionally, there may be some questions or "unknown unknowns" that may not seem obvious to ask—especially for younger researchers and/or researchers who are trying to do research outside of their normal fields of expertise.
Having a hub for collecting and interrelating arguments/evidence on these and other questions seems like it could save time by reducing duplication/wasted effort and also increase the likelihood of encountering good points, among other potential benefits (e.g., better keeping track of back-and-forth responses and overall complexity for a question which doesn't lend itself well to traditional mathematical equations). Additionally, if junior researchers (including even interns) are the primary source of labor for this project—which I expect would be fairly practical given that much of the "research" aspect of the work is akin to conducting literature reviews (which is then formalized into the node-and-link structure )—then you could potentially get comparative advantage benefits by having less-experienced researchers save research time of more-experienced researchers, all while providing a "training ground/on-ramp" for less-experienced researchers to become more familiar with various fields.
Because I am still testing out which approaches/structures seem to work best, the networks in the following screenshots are quite raw: there are still some inconsistencies in terms of node types and presence, little attention paid to aesthetics, many undefined placeholder relationships, inconsistencies in terms of granularity, etc. (That being said, the purple squares are generally "claims/assertions/arguments", yellow triangles are policy proposals, blue hexagons are research questions, and green diamonds are generally "reality variables")
Note: I do not think that the map view above should be the only way of viewing the research/analysis; for example, I think that one of the viewing methods should ideally be something like a minimal-information interface with dropdowns that let you search for and branch out from specific nodes (e.g., "what are the arguments for and against policy proposal X... what are the responses to argument Y").
Related to this is my post on "epistemic mapping", which put a greater emphasis on the academic literature around non-policy questions (including identifying authors, studies, the inputs for those studies (e.g., datasets), etc.) as opposed to supporting policy research directly—although both systems could probably be used for the same purposes with minor adaptations.
Also relevant—and more developed + better resourced than my project—is Modeling Transformative AI Risks (MTAIR), which puts more emphasis on quantitative analysis via input elicitation/estimation and output/effect calculation (and at the moment seems to focus a bit more on the factors and pathways of AI risk vs. the direct effects of policy, although my understanding is that it is also intended to eventually focus on policy analysis/recommendations).
Thanks for your comment Harrison. It looks like a really interesting problem you're working on. In my experience, summaries and visualisation can be really powerful for coordination and alignment (of humans, let alone AIs!)
I'll take a look at the links you've provided to learn more.