I had a bunch of unpublished drafts and notes in Google Docs. I just released them here, using Claude to help organize and summarize.
https://priors.quantifieduncertainty.org/
I imagine some here would find them interesting! Most are fairly old, starting from 2017, when I joined FHI.
End of an era
After about seven years, QURI is moving into maintenance mode. This means we’ll ensure that key software (Guesstimate and SquiggleHub) is maintained, but we won’t be developing new software or doing new research.
Background
I started QURI in 2019. At that point I was excited about projects at the intersection of epistemics and software, primarily to be used by impactful researchers. This seemed like a neglected and promising area.
Over the years we developed a small team and worked on a variety of projects. We’ve created multiple tech projects (Foretold, Squiggle, SquiggleHub, RoastMyPost), maintained Guesstimate, and wrote over 90 EA Forum posts. In the background, we’ve collaborated with and consulted for several EA organizations, and done work guiding the nearby forecasting and epistemic space.
The work was challenging, as one might expect. We had a small team. Full-time staff included Nuño Sempere for a few years on research, then Slava Matyuhin on software engineering for Squiggle. For the last year it’s mainly been me. I think we were fairly efficient for the team size and budget, but this naturally made it hard to move as quickly as we'd have liked.
Recently I've felt particularly constrained by compute budgets, and more concerned about short timelines. I was offered an opportunity to work on AI safety at Google DeepMind, and after a lot of deliberation, decided it represented a higher expected value for me than continuing QURI. I'll be starting there next week.
It’s possible I’ll return to QURI full-time sometime in the future, but I consider it unlikely. I do intend to make sure our key projects get continued support, and I'm happy to advise other future projects in the space.
What happens to our projects
If you're particularly interested in improving, reviving, or maintaining any of these, please reach out. I continue to be excited about them, and Guesstimate and Squiggle in particular are still used by impactful researchers.
Operational details
This leaves QURI with no full-time employees. I'll remain President of the board, and we'll keep a couple of part-time contractors, but we won't be making new hires.
We have enough committed funding to cover basic maintenance - servers, software upkeep, and legal/bookkeeping - for at least four years. The apps that matter most, Guesstimate and Squiggle, are cheap and fairly simple to keep running, and I expect these costs to fall over time, so I'm not worried about sustainability.
Thanks
QURI was always a collaboration. Our funders included the Long-Term Future Fund, the Survival and Flourishing Fund, Coefficient Giving, the Foresight Institute, and several independent donors. Rethink Priorities has been our fiscal sponsor since 2023. Our board has included Ben Goldhaber, Jacob Lagerros, Andrew Critch, and Abigail Olvera, and we've had many collaborators and contractors who helped enormously.
And of course the effective altruism and rationality communities were critical throughout - the people who used Squiggle and Guesstimate, argued with our takes, gave us feedback, and pushed the work to be better.
I remain excited about work in the nonprofit and epistemics space, and I expect to stay connected to it.
Happy to answer questions in the comments.
I'd like to throw my hat in the ring and indicate that I'd at least find it very interesting to take over for you to ensure that QURI's mission continues! I'm currently trying to get back into the AI safety grantmaking space, but that'll most likely fail, in which case I would welcome a plan B.
I imagine that the grantmaking bottleneck is overblown – that a 100–1000x increase in grantmaking capacity is easily achievable through hiring obvious candidates (10x) and streamlining the processes through retroactive funding (10–100x). If the funds actually end up doing that, it'll be better again to contribute as a charity entrepreneur, and the plan B would become a plan A.
I'd prefer to expand QURI to projects that have less to do with quantification and more with ranking and clustering, and to adopt more of an incubator-like approach where successful projects turn into spin-offs with their own legal structure over time to introduce more resilience through redundancy. (And more Python.)
That'll probably require about $1m in funding over ~2 years. Is that realistic? Also I'm not sure if it clashes with your vision?
Hi Dawn!
I'd be happy to discuss this.
Quickly (and to give context to others reading this):
Surplus sounds useful!
I think everything hinges on the funding unfortunately…
Most of the projects on my list require some $200–500k in the first year to get started, and then can scale to a few million per year over time. The large-scale retrofunding needs to start higher – $10m might work, $100m works for the XPrize, $1b could be the goal.
The natural starting point is the incubator itself, which falls into the $200–500k range, but more towards the upper end to provide seed funding for the incubated projects.
Why did Guesstimate/Squiggle as for-profit not work out?
I'd love to have a call and catch up in any case! I'm curious whether you already have an opinion on whether places like DeepMind will be interested in paying for evals like the two types mentioned here (character and backdoors).
"Why did Guesstimate/Squiggle as for-profit not work out?"
I've been asked this question a bunch of times before, happy to give some quick thoughts here. Most of my answer is here.
If it were the case that it seemed very easy to make these as startups, and ideally in which there were some clear cofounders who could handle the business side, I think the for-profit side would have been more promising.
In my case:
1. I realized this was a niche/small space, that getting a large market would likely mean having to pivot.
2. I wasn't sure what might take off. I think Squiggle has a lot going for it, but am not sure if the value proposition is strong enough for a large company now, especially as LLMs have been getting much more powerful (changing the trade-offs around decision tools).
3. My main goal was helping effective altruist / rationalist researchers, and I suspected that focusing much more on commercialization would have hurt that.
4. Related to (3), I just found myself much more motivated to go after altruists/nonprofits than to go after standard enterprise deals
"I'd love to have a call and catch up in any case! I'm curious whether you already have an opinion on whether places like DeepMind will be interested in paying for evals like the two types mentioned here (character and backdoors)."
Quickly:
1. I'll schedule a call sometime, once I understand my schedule a bit better.
2. I have incredibly little info on DeepMind right now. As you may imagine, personally I'm excited about a lot of these areas, but I can't speak for DeepMind.