Long-Term Future Fund AMA

Rough fermi on how much time I expect to spend on this:

I spent about 12 hours on the investigations of the last round, and about 2 hours a week since then on various smaller tasks and discussions about potential grantees. I spent less time than I wanted to since we made the first round of grants, so I expect to settle in at something closer to 3-4 hours per week. I expect this will be higher than the other fund members.

In the long-run I expect that the majority of time I spend on this will be in conversations with potential grantees and the other members of the fund, about models of impact of various types of projects, and long-term future strategy in general. I think I would find those conversations useful independently of my work for the fund, so if you are interested in net-costs you might want to weigh that only at 50% or so. Though I am not even sure whether the net-cost for me is negative since I expect the fund will be a good vehicle for me for me to have more focused conversations and modeling about the long-term future and I can’t come up with an obviously better vehicle to do so.

So, in terms of time I will spend doing work that is related to this fund, I expect to settle in around 3-4 hours per week, with peak periods of about 15 hours per week about 4 times a year. Though how much of that is counterfactual and whether I experience any net cost is unclear, probably at least 20% of that time is dead-loss in terms of boring logistical tasks and other things that I don’t expect to gain from much long term, but I also expect to gain at least some benefit from being on the fund and the opportunities for learning it will open up for me.

I expect different people on the fund will have quite different answers to this, so here is my perspective:

I don’t expect to score projects or applications on any straightforward rubric any more than a startup VC should do so for the companies that they are investing in. Obviously, things like general competence, past track record, clear value proposition and neglectedness matter, but at large, I mostly expect to recommend grants based on my models of what is globally important, and on my expectation of whether the plan the grantee proposed will actually work, and do something that I guess you could call “model-driven granting”

What this means in practice is that I expect the things I look for in a potential grantee to differ quite a bit depending on what precisely they are planning to do with the resources. I expect there will be many applicants that will display strong competence and rationality, but are running on assumptions that I don’t share, or are trying to solve problems that I don’t think are important, and I don’t plan to make recommendations unless my personal models expect that the plan the grantee is pursuing will actually work. This obviously means I will have to invest significant time and resources to actually understand what the grantees are trying to achieve, which I am currently planning to make room for.

I can imagine some exceptions to this though. I think we will run across potential grantees who are asking for money mostly to increase their own slack, and who have a past track record of doing valuable work. I am quite open to grants like this, think they are quite valuable and expect to give out multiple grants in this space (barring logistical problems of doing so). In that case, I expect to mostly ask myself the question of whether I expect additional slack and freedom would make a large difference in that person’s output, which I expect will again differ quite a bit from person to person.

One other type of grant that I am open to are rewards for past impact. I think rewarding people for past good deeds is quite important for setting up long-term incentives, and evaluating whether an intervention had a positive impact is obviously a lot easier after the project is completed than before it is completed. In this case I again mostly expect to rely heavily on my personal models of whether the completed project had a significant positive impact, and will base my recommendations on that estimate.

I think this approach will sadly make it harder for potential grantees to evaluate whether I am likely to recommend them for a grant, but I think is less likely to give rise to various goodharting and prestige-optimization problems, and will allow me to make much more targeted grants than the alternative of a more rubric-driven approach. It’s also really the only approach that I expect will cause me to learn what interventions work and don’t work in the long-run, by exposing my models to the real world and seeing whether my concrete predictions of how various projects will go come true or not.

I also think this sort of question might be useful to ask on a more individual basis - I expect each fund manager to have a different answer to this question that informs what projects they put forward to the group for funding, and which projects they'd encourage you to inform them about.

We won’t generally have access to work that isn’t shared with the general public, but may incidentally have access to such work through individual fund members having private conversations with researchers. Thus far, we’ve evaluated organizations based on the quality of their past research and the quality of their team.

We may also evaluate private research by evaluating the quality of its general direction, and the quality of the team pulling it off. For example, I think the discourse around AI safety could use a lot of deconfusion. I also recognize that such deconfusion could be an infohazard, but nevertheless want such research to be carried out, and think MIRI is one of the most competent organizations around to do it.

In the event that our decision for whether to fund an organization hinges on the content of their private research, we’ll probably reach out to them and ask them if they’re willing to disclose it.

In the last grant round AI Impacts was an organization whose work I was excited about, but that currently seemed to not have significant room for extra funding. (If anyone from AI Impacts disagrees with this, please comment and let me know otherwise! )

What do you think distinguishes you from EA Grants?

I think we will have significant overlap with EA Grants in the long-term future domain, and I don’t think that’s necessarily a bad thing. I think EA Grants is trying to do something very broad, and having more people with different models in this space is valuable and I expect will significantly increase the number of good grants that can be awarded in this space.

I do think that the EA Grants team has a comparative advantage at funding small grants for individuals (we can’t give away grants below $10k), and applications that are more uniquely related to EA Movement building, or personal skill development to enable later direct EA work.

In general, if there is an overlap, I think I would encourage people to apply to EA Grants first, though applying to both also seems fine, in which case we would try to coordinate with the EA Grants team about the applications and try to determine who is better suited to evaluate the application.

I also expect that a large part of our granting work will be proactive, by trying to encourage people to start specific projects, or encourage specific existing organizations and projects to scale up, which I expect to not overlap that much with the work that EA Grants is doing.

I am in contact with a couple of other funding sources who would take recommendations from me seriously, but this fund is the place I have most direct control over.

Both Matts are long-time earn-to-givers, so they each make grants/donations from their own earnings as well as working with this fund.

I personally fund some things that are too small and generally much too weird for anyone else to fund, but besides that, I don’t control any alternative pots of discretionary funding.

(Can't speak for everyone, so just answering for me) I do not control alternative pots of discretionary funding, besides the budget of the LessWrong organization. I can imagine running into some overlap of things I might want to fund with the LessWrong budget and things I would want to recommend a grant for, mostly in the space of rewarding research output on LessWrong and the EA Forum, but overall I expect the overlap to be quite minor.

Open to both. We’ve funded both of your examples in our last round. I generally have a sense that funding smaller projects is currently undersupplied and I am more excited about our grants to small projects than our grants to more established ones, though I do think there are a lot of excellent organizations in the space. I overall expect we have about a 50/50 funding split between smaller start-up projects and larger organizations.

I have a bunch of thoughts, but find it hard to express them without any specific prompt. In general, I find a lot of AI Alignment research valuable, since it helps me evaluate other AI Alignment research, but I guess that’s kind of circular. I haven’t found most broad cause-prioritization research particularly useful for me, but would probably find research into better decision making as well as the history of science useful for helping me make better decision (i.e. rationality research).

I’ve found Larks recent AI Alignment literature and organization review quite useful, so more of that seems great. I’ve also found some of Shahar Avin’s thoughts on scientific funding interesting, but don’t really know whether it’s useful. I generally think a lot of Bostrom’s writing has been very useful to me, so more of that type seems good, though I am not sure how well others can do the same.

Not sure how useful this is or how much this answers your question. Happy to give concrete comments on any specific research direction you might be interested in getting my thoughts on.

I don’t think any of us have any particular expertise on this question. You could try sending an application on their jobs page.