This is a link post for How Life Sciences Actually Work: Findings of a Year-Long Investigation. I added a special section on the relevance to the EA Community after the conclusion.

See discussion on twitter, reddit.

---

Summary: academia has a lot of problems and it could work much better. However, these problems are not as catastrophic as an outside perspective would suggest. My (contrarian, I guess) intuition is that scientific progress in biology is not slowing down. Specific parts of academia that seem to be problematic: rigid, punishing for deviation, career progression; peer review; need to constantly fundraise for professors. Parts that seem to be less of a problem than I initially thought: short-termism; lack of funding for young scientists.

Introduction

In his Asymmetric Weapons Gone Bad (a), Scott Alexander notes that for some areas of inquiry, studying them a little bit leads you astray and only studying them a lot makes it possible to understand what's really going on:

Maybe with an unlimited amount of resources, our investigations would naturally converge onto the truth. Given infinite intelligence, wisdom, impartiality, education, domain knowledge, evidence to study, experiments to perform, and time to think it over, we would figure everything out.

But just because infinite resources will produce truth doesn’t mean that truth as a function of resources has to be monotonic. ...

Some hard questions might be epistemic traps – problems where the more you study them, the wronger you get, up to some inflection point that might be further than anybody has ever studied them before.

I think that this observation is a general one -- true for almost all areas of study -- and moreover lends itself to a much simpler explanation: almost everyone has an agenda. Almost no one has the exact right picture of what's going on in their mind.

Thus, when starting to study something and reading one or two books or talking to a couple of people, you will inevitably overload on their agenda, personal biases, personal ignorance, and all the other factors that do not lead to the correct model of the subject.

Then, if you read a few more books or talk to several more people, you will see some of the biases that influenced your model-building. However, this also will not be enough. When picking the books to read or people to talk to, your sample will be either too narrow (i.e. people you talk to will be very similar to each other) or too broad (i.e. their experiences will be so discordant that it will too difficult for you to be able to cross-reference them and piece them into a coherent picture).

---

When I talked to the first 5 people about how life sciences research works, I had a perfect picture in my mind, fully in accordance with all the rumors about academia's sclerosis, risk-aversion, and short-termism, I'd been hearing for the last few years.

When I talked to 10 more people, I realized that something was off, but couldn't quite piece it all together.

In the end, I ended up interviewing about 60 people involved with life sciences research -- mostly grad students, postdocs, and PIs (principal investigators, heads of laboratories), but also, for example, with people from philanthropic organizations and venture funds investing in life sciences.

You could view this essay as a culmination of a year of my pursuit of applied progress studies (a). I feel like I'm finally able to converge on something, even though this something seems to be somewhat different from what almost any single person I talked to has as their model of what's going on in the field.

[Life] science is not slowing down

It's tempting to look at the negative side of academia. Bureaucratization, seeming risk-aversion, loss of freedom... and conclude that therefore science is working poorly. This is a mistake. Yes, funding agencies are risk-averse; yes, academia now selects for things you probably don't want it to select for, like conformity and high conscientiousness; yes, an average scientist is not in academia for the love of science (and maybe the productivity of an average scientist is decreasing. [sidenote 1: would argue that comparing an average scientist today and in 1973 is a bit like comparing an average high-school drop-out today and in 1973. (a)]

However, all of this does not mean that science is stagnating or even that it is slowing down. The pace of discovery in all areas of biology I looked at is astounding. And not only the pace is astounding, but many researchers are indeed working on the highest expected value projects they have with little restrictions and are taking in the smartest people they can find and throwing money at them (the way it works in biology is that PIs fundraise and manage/mentor, while graduate students and postdocs do the work).

If you look at Harvard or MIT or Stanford or many other universities -- they are all filled with amazing researchers working on great budgets, truly passionate about discovery and invention. HHMI (a) is actually giving people unrestricted grants; CZI (a) is funding people specifically to work on software (which is usually hard to get funded) and to work on their riskiest, usually unfundable ideas; and many other foundations are trying to correct the inefficiencies they see in resource allocation.

I think that the perception of stagnation in science -- and in biology specifically -- is basically fake news, driven by technological hedonic treadmill and nostalgia. We rapidly adapt to technological advances -- however big they are -- and we always idealize the past -- however terrible it was.

I mean -- we can just go to Wikipedia's 2018 in science (a) and see how much progress we made last year:

• first bionic hand with a sense of touch that can be worn outside a laboratory
• development of a new 3D bioprinting technique, which allows the more accurate printing of soft tissue organs, such as lungs
• a method through which the human innate immune system may possibly be trained to more efficiently respond to diseases and infections
• a new form of biomaterial based delivery system for therapeutic drugs, which only release their cargo under certain physiological conditions, thereby potentially reducing drug side-effects in patients
• an announcement of human clinical trials, that will encompass the use of CRISPR technology to modify the T cells of patients with multiple myeloma, sarcoma and melanoma cancers, to allow the cells to more effectively combat the cancers, the first of their kind trials in the US
• a blood test (or liquid biopsy) that can detect eight common cancer tumors early. The new test, based on cancer-related DNA and proteins found in the blood, produced 70% positive results in the tumor-types studied in 1005 patients
• a method of turning skin cells into stem cells, with the use of CRISPR
• the creation of two monkey clones for the first time
• a paper which presents possible evidence that naked mole-rats do not face increased mortality risk due to aging

Doesn't seem like much? Here's the kicker: this is not 2018. This is January 2018.

If you actually go and look at the major discoveries made in any single year in the first half of the 20th century and compare it to 2018, you'll probably conclude that the pace of scientific progress is only getting faster.

After a year of studying how life science research actually works and progresses, I'm way more optimistic about it.

Nothing works the way you would naively think it works (for better and for worse): 3 Examples

1. "NIH's risk-aversion makes it very hard to fund innovative science"

Observation: NIH is risk-averse to the point that even when it explicitly asks for radical proposals, nobody believes it will actually fund anything radical, so people don't even bother applying with high-risk research [sidenote 2: Open Philanthropy evaluating NIH Director's Transformative Research Award proposals (a): 'we thought many of the proposals we reviewed were similar to proposals in more typical RFPs in terms of their novelty and potential impact. In other words, we considered many of the submitted proposals to be a bit on the conventional side. This surprised us given the “transformative” premise and focus of the TRA program. We speculate that this may be due to the constraints within which applicants feel they must work to get through panel reviews.']

Naive conclusion: it's impossible to get high-risk projects funded by NIH, so scientists end up working on incremental science

Reality: NIH doesn't force scientists to adhere 100% to their submitted plans, so what ends up happening is that scientists apply for a grant with a project for which they have supporting preliminary data (so that NIH doesn't consider it high-risk) and spend most of the funds on that project. The left-over funds then can produce preliminary data for their high-risk ideas, making them appear less risky and more easily fundable by the NIH in the future.[ sidenote 3:Sometimes this takes a more extreme form where researchers actually promise NIH good results from experiments they already performed but have not publicized yet.]

So, in reality, distortion is still present because you have to figure out rather arcane ways to get anything interesting funded, but the amount of distortion is way less than we would naively expect.

2. "NIH's biases make it very hard to fund methodological research"

Observation: NIH doesn't like to fund purely methodological studies (e.g. development of better software)

Naive conclusion: it's impossible to get funded for methods development by NIH

Reality: you can dress up methods development for NIH, e.g. by providing a concrete biological goal for which insufficient methods are the bottleneck and show NIH that you will be able to achieve concrete progress on something that matters to them using your better methods

Again, yes, doing this introduces obvious frictions and inefficiencies and skews what scientists work on towards things that are easy to dress up for NIH, rather than things that they believe are most important. But the amount of frictions and inefficiencies is way less than we would naively expect.

3. "NIH severely underfunds young researchers"

Observation:

the median age of first-time recipients of R01 grants, the most common and sought-after form of N.I.H. funding, is 42, while the median age of all recipients is 52. More people over 65 are funded with research grants than those under age 35. (NYT (a))

Naive conclusion: young scientists lack the resources to pursue their research. NIH should allocate more funds to investigators under age 35 and 40.

Reality: Many extremely capable people end up working in well-funded labs whose PIs are pretty hands-off and who allow their grad students and postdocs to work on whatever they want and not really think about money. In statistics, this registers as "more money than ever goes to older/famous PIs" and "young investigators can't start their independent careers". On the ground this means that until age 35 (i.e. when your creativity is the highest) you are isolated from management and fundraising (and endless administrative responsibilities bestowed on any tenure-track professor) and can 100% focus on doing science and publishing papers, while getting mentoring from your senior PI and while being helped by all the infrastructure established labs have (administrative assistants, lab technicians, etc.).

One could make an argument that due to all of these factors, the present funding structure is beneficial to production of original research by those most capable of it, and that letting more young people set up their labs will mostly result in more people attending useless faculty meetings and more people writing grants, instead of doing research.

When you see "X's [where X is a famous scientist] lab discovers Y", chances are it was Z -- X's grad student or a postdoc -- who came up with the idea, did all the work (with help of other students), and wrote the paper. X's contribution was probably in providing the research environment in which all of these were possible, money, and talking to Z every other week about the progress with the idea. I'm not saying that X's contribution is not important -- it usually is -- but press releases focused on the PI miss the fact that money is not actually spent directly by them, systematically biasing our perception of how funds are allocated.

In the end, yes, distortion is there. It is indeed difficult to become independent and establish your own personal research agenda if you're young, but the end negative effect on science done by young researchers is way less than we would naively expect.

Two side points

• When I write "naive conclusion", it usually means that it took me months of research to figure out why this is actually not true
• While NIH provides the majority of funding for life science research in the US, there are many foundations that are aware of its biases and that are actively trying to fund people and research that NIH misses, support grad students, postdocs, and early-career independent researchers, and so on.

If you're smart and driven, you'll find your way in

There are many labs in biology that have good funding and open and risk-loving PIs. This means that if you're very smart, ambitious, and driven, you'll likely be able to get in and work on cool stuff, even with a questionable background, although you might have to spend a year or so working as a Research Assistant to prove your worth. PIs are usually pretty open to getting Research Assistants and are very open to getting thoughtful personalized cold emails (this applies to scientists you would consider famous as well).

I'm fairly confident that if someone capable of producing Nobel-level research decided to do a PhD in biology or neuroscience, they would be able to enter great programs with great advisers and funding in less than 2 years.

The problem here is that almost everybody for whom understanding the stuff I wrote above would be useful is misinformed. People outside of biology generally think that doing a PhD means spending 6 years at the bench performing your advisor's experiments and is only possible with perfect undergrad GPA, not realizing that neither of these are true of you're truly capable, and as a result form their decision to avoid grad school on misinformation.

Nobody cares if you're a genius

This may sound like I'm contradicting the previous section, but note that there I only wrote "get in", not "stay in".

In order to get in, you only have to convince one PI that you're worth taking a small chance on. In order to stay in, you have to convince many professors, many study sections who will assess your grant applications, various charitable foundations, hiring committee at the university, tenure panel, and so on, and so on.

Every one of these entities will be less open and less forgiving than the PI who might decide to take you in.

I know many brilliant researchers.

• some work in a field that is currently not fashionable
• some are not very likable or are bad at networking
• some are of unwelcome demographics
• some are not good at hyping up, explaining or putting the right spin on their research
• some are too passionate about their subject (Church (a), Woodward (a), Ramanujan (a), Lippmann (a))
• some are just not very good at anything else (Einstein (a))
  • "Einstein took the entrance examinations for the Swiss Federal Polytechnic in Zürich ... He failed to reach the required standard in the general part of the examination, but obtained exceptional grades in physics and mathematics"
• some are too disagreeable
• some are bad at writing grants, selling, and fundraising
• some are too truth-seeking and are retaliated against due to of this (case of Hellinga (a))
• some are not good at finishing their papers (Bohr (a))
• some are bad at academic politics
• some are recluses
• some got unlucky in their early research results or advisors
• some are bad at policing their speech to current moral fashions (1 (a), 2 (a))
• some work on research that is too novel and are therefore not understood (genetics (a), DNA Synthesis (a), public key cryptography (a), many "crackpots" (a))
• some are not good at pretending and molding themselves for every committee and every panel
• some are bad at dealing with bullshit and bureaucracies in general
• some have low self-esteem and critically undersell themselves
• some are good at coming up with stuff but are bad at formalizing it and putting it on paper (see next section)

A great many skills are required of a successful scientist in academia (a). This is what Eric Weinstein means when he talks about the perils of pursuit of excellence (a):

So genius and excellence are both worthwhile but they are distinct modalities, and not recognizing that it is a serious problem to take somebody in the genius idiom and to push them into a different idiom, which is to reduce their variance, is going to be very destructive and it’s going to keep us from founding the industries that will allow us to change paradigms and move forward.

Almost all biologists are solo founders. This is probably suboptimal

Many of the listed above ailments would be solved if the researcher in question had a champion or a co-founder who would complement them at whatever they're bad at.

In Jessica Livingston's (co-founder of Y Combinator) Founders at Work (a), Max Levchin (PayPal co-founder) says:

Try to have a good cofounder. I think it’s all about people, and, if you are doing it completely alone, it’s really hard. It’s not impossible, in particular if you are a loner and introverted type, but it’s still really hard ... I had run a company before PayPal, alone, and I thought it was fine. I could deal with it. But, you only can count on energy sources and support sources from yourself. There’s really no one else who you can go to and say, “Hey, this thing is going to fall apart any minute now. What the hell are we going to do?”

The thing that kept us going in the early days was the fact that Peter and I always knew that both of us would not be in a funk together.

YC itself is famous for its preference for teams rather than individuals. There should be a CEO (visionary, salesman, manager) and a CTO (builder and designer). In light of this, it's puzzling why universities seem to virtually never hire two people to run a lab jointly. There's a single PI who has to both be excellent at being the CEO and at being the CTO and who moreover has to run the lab essentially alone -- everybody else in the lab will be an employee with negligible amount of responsibility for the long-term future of the lab.

This is very weird and reminds me of Feynman and Dyson. Freeman Dyson:

[Feynman] was working on these problems of quantum electrodynamics, and he had done a great deal which was very beautiful, but which nobody else understood. And he loved to talk, and I loved to listen. So within six months I had pretty well mastered his language. And within one year, I actually was able to translate his ideas into mathematics, so it became more accessible to the world. And as a result, I became famous, but it all happened within about six months.

How many scientists never reach their potential because they fail to find their co-founder and because the way academia is structured today seems to be very unwelcoming of such arrangements?

There's insufficient space for people who just want to be researchers and not managers

Many people who always wanted to become scientists do not pursue or leave academia because they see how PIs work and think that they do no want to just manage people and fundraise/write grants. This is a great tragedy. Very few labs have permanent Research Scientist positions and for some reason there's a path "PhD-->Postdoc-->PI" that is almost impossible to avoid (there are institutions that experiment with permanent pure researcher positions (Broad, CZI, Wyss, Calico) but there seem to be very few of them).

There was a famous problem in large companies where if you were a really good software engineer, you'd have to become a manager to continue to advance your career but I hear that these days, companies have introduced "Individual Contributors" where you can grow while still being primarily a technical contributor. This is something that academia still has to figure out.

My intuition is that the biggest unnecessary attrition from academia happens due to the absence of an established "pure researcher" career track.

Peer review is a disaster

In The Double Helix (a), James Watson (co-discoverer of the structure of DNA along with Francis Crick) writes:

[I]t would take two or three years to set up a new research group primarily devoted to using X rays to look at the DNA structure. Moreover, such a decision would create an awkward personal situation. At this time molecular work on DNA in England was, for all practical purposes, the personal property of Maurice Wilkins, a bachelor who worked in London at King’s College.

Several people I talked to describe a similar dynamic and tell me that there are niches dominated by a particular research group that guards that niche almost as its fiefdom. It takes a lot of courage and determination to move in and try to upend the dominant methodologies and research directions of the field.

Peer review exacerbates this dynamics. Peer reviewers in your field are your competitors, who have not themselves solved the problem you claim to be able to solve. They have both personal and professional interest (especially so if funding is limited) in giving low scores to grant applications of competing teams and to recommend rejection of their journal submissions. Further, since they’re experts in the grant application topic, while rejecting your paper or grant application, they can lift your research ideas and then pursue them themselves. This happens more frequently than you would expect (a).

Further still, committees reviewing grants have all sorts of weird interpersonal dynamics that make funding anything unconventional even more difficult than it would be in their absence, because, while being on a committee people are usually averse to be publicly approving towards anything that seems weird.

An anonymous redditor writes (a):

In my field virtually 100% of the papers in “top” journals come from the same 5-10 senior authors, and they can just about get away with murder.

A statistician writes (a):

The referees completely fail to understand ideas we've adapted to the meanest understanding, they display astonishing gaps in their knowledge, and lots of them can't (as my mother puts it) think their way out of a wet paper bag. Even if you discard these as mere dregs, far too many of the rest seem to miss the point, even points which we've especially labored to sharpen. Really good, valuable referee reports exist, but they are vanishingly rare.

Almost everyone I talked to shared a broadly negative sentiment about peer review.

Nobody agrees on whether big labs are good or bad

Big labs tend to have

• more cross-pollination of ideas in-house and less friction for collaboration
• more unrestricted funds and to therefore be more open to work on whatever is interesting, rather than a concrete project the PI has to finish to get their single grant extended
  • this allows for more risk and exploration in big labs
• administrative assistants and more lab technicians, meaning that scientists have more time to do science

On the other hand, scientists are not at all always good managers. In a big lab, students frequently receive insufficient mentorship from the PI and have to be very independent and look for outside mentors and support (this is worsened by the fact that few people care to hire middle managers for big labs). The result is that often people are left to their own devices and are sometimes lost in research, meaning that strictly speaking the amount of waste is increased.

People who have big labs continue to fight for more funding and feel that what they create is unique and must be protected (If you have a big lab, you probably have a lot of people trying to get in, meaning that you feel there's always plenty of opportunities to grow), while people who are barely scraping by suggest strict limits on lab size (and other ways to make distribution of money more equitable) and point out that sometimes big established labs continue to get funded almost by inertia. People have very strong feelings about this.

Many scientists seem to measure their professional success by the number and the size of their grants and do everything they can to maximize their funding and my impression is that simply giving them much more money (like HHMI does) won't change how they spend their time too much -- they'll just continue writing grants.

Senior scientists are bound by their students' incentives

As I noted before, in biology, PIs mostly manage people -- all the real work is done by grad students and postdocs. Grad students and postdocs have to graduate and to look for faculty jobs, meaning that however long the PI's horizon is, the people who do the work will be bound by their short-term need to publish. And given that grad students and postdocs typically make about 25-50% of their market rate, however passionate they are about science, they want to graduate and move on as fast as possible.

(This also applies to those who have fellowships, as they always have a pre-determined maximum length, usually 3-5 years)

Universities seem to maximize their profits, with good research being a side-effect

When I started investigating how biology works, I believed that universities spend their own money to run labs and enable research, because this is how it works in economics, where I come from. I soon learned this is not the case.

In biology, it's the scientist who brings money to the university. University provides affiliation/credibility, space, and administrative assistance. But then, to pay for its costs it takes a cut ("overhead") from every NIH grant the scientist gets. University usually gets about 1/3 of the size of the grant.

Because university's cut is linear but actual expenses are sublinear, pretty soon "overhead" turns into "profit" for the university.

As a result, in hiring decisions, the amount of money the researcher is able to bring sometimes effectively becomes the measure of quality of research.

One of the funnier side-effects of this is that now universities have not only reputational incentive to cover up fraudulent or questionable research by scientists but also a financial incentive to protect scientists who bring in a lot of money. [sidenote 4: Analogous to how journals these days are thinking much harder before retracting well-cited but "questionable" papers -- their impact factor depends on them, after all.]

The anonymous redditor from section about peer review further notes (a):

In my department, the main standards [for getting tenure] are to a) win two RO1 research grants, then b) renew them both. If you’re bringing in hundreds of thousands in indirects every year, the tenure committee doesn’t much care whether you publish in Nature or Diabetes.

Also see The Uncharity of College: The Big Business Nobody Understands. (a)

Large parts of modern scientific literature are wrong

I am confident that somewhere between 10% and 50% of papers published in good journals are wrong, meaningless or fraudulent. This figure is based on my expertise in parts of economics, psychology, neuroscience, and genetics. Unless you invested significant time studying the subject matter, you will have very bad intuition about which papers are good and which ones are bad. See Replication crisis (a).

Further, my guess is that papers published in top journals (Cell, Nature, Science) are more wrong on average than papers from top journals from specific fields of science. It seems that CNS chase hype and have rather lax standards on the methodology. Plus, chasing hype and being interested only in "large update" papers means that we would expect more of these papers to be wrong just because of regression to the mean.

People I talked to say that the papers CNS publish from their fields of expertise would never get published in more specialized journals and that CNS casually disregard reviewers' criticisms of methodology.

This matches my observations, where sometimes catchy but bad papers that would be rejected on the grounds of poor methodology from a field journal get published in CNS.

A paper (a) published in Frontiers in Human Neuroscience finds that:

methodological quality and, consequently, reliability of published research works in several fields may be decreasing with increasing journal rank

You should reflect on whether a typical study you hear about is selected more on the ability to propagate itself across researchers, news, and social media or on sound methodology.

---

Sometimes, virtually entire swaths of the literature turn out to be meaningless. Recent examples from life sciences are candidate gene studies (a) and small-n cognitive neuroscience (a). [sidenote 5: A recent example not from life science is small-n social psychology (a).]

How is it possible that entire research fields turn out to be meaningless? Some people (a) blame the incentive structure of modern academia. I disagree.

Any area of study -- however meaningless -- can produce a body of established facts, practices, and experts. Newton studied (a) theology, alchemy, and physics. 25% of people in the US believe (a) in astrology and some of them go to astrology experts.

If one can convince other people to give them money to study something, they will gain a financial motive to tell everybody else about the importance of the subject and become an "expert" on it. If there's a group of "experts" they will all amplify each other's voices and will try to legitimatize whatever it is that they study. As a different redditor astutely notes (a):

[N]early every expert relies on the valuation of their expertise for money: therefore every expert has a strong case to oversell their expertise/the state of knowledge in their discipline

(as an aside -- the interaction between personal rivalries in the field and between the common interest of its practitionaires to inflate its perceived importance ought to produce some truly fascinating interpersonal dynamics)

---

Finally, some papers are just fraudulent. Elisabeth Bik looked (a) at more than 20,000 papers that contained a particular type of easily examinable picture published in good biology journals between 1995 and 2014 and found that 3.8% -- about 1/25 -- contained "problematic figures" and at least half of these had evidence of deliberate manipulation.

Here's what one of the VCs investing in life sciences I talked to told me:

If you see a picture, this is the best picture the authors had. If you see a statistic, there's probably something wrong with it. 5% of the companies we do due diligence on -- and we only do due diligence on excellent companies -- edit their pictures.

Raising money is very difficult even for famous scientists

Collectively, scientists I talked to interacted with more than 20 billionaires but I'm not aware of any of them raising any significant amount of money as a result. I'm very surprised by this because if I were a billionaire, I would probably fund at least one Boyden-like scientist in perpetuity and it seems that there are many more billionaires interested in science than there are Boydens.

Perhaps this is the result of the widespread (and not entirely unfounded) perception that science is broken and that scientists are effectively conning the society into giving them money, resulting in people believing that

1. an average scientist is not worth giving more money to
2. while famous scientists like Church and Boyden don't need financial support (even if in reality, the reason they're well-funded is because they spend a ton of time fundraising and not having to write grants every year would greatly increase their research output)

In an interview to Tyler Cowen, Boyden says (a):

For me, it became personal because when we proposed this expansion microscopy technology, where we blow up brain specimens and other specimens a hundred times in volume to map them, people thought it was nonsense. People were skeptical. People hated it. Nine out of my first ten grants that I wrote on it were rejected.

If it weren’t for the Open Philanthropy Project that heard about our struggles to get this project funded — through, again, a set of links that were, as far as I can tell, largely luck driven — maybe our group would have been out of business. But they came through and gave us a major gift, and that kept us going.

Conclusion

If you take away one thing from this essay, let it be this: academia has a lot of problems but it's less broken than it seems from the outside.

Everything is much more complicated than everyone thinks and it's very easy to try to improve something only to have second-order effects mess everything up. Because of all of this, I'm wary of proposing any drastic changes to how grants are distributed, how researchers are trained, and so on.

Instead, I have several smaller-scale recommendations:

• foundations should try to be more like venture funds and actively seek out researchers who need more money, instead of just responding to applications, and tailor their funding to individual scientists more
• foundations and NIH should experiment more with how they give and should not be afraid to spend significant resources on evaluation (i.e. hire economists and do good RCTs)
  • for example, by implementing double blind application reviews or using golden tickets (a)
  • or by trying to fund small teams (like Abudayyeh and Gootenberg (a))
• foundations and philanthropists with $195,000 of free money lying around should consider supporting Life Sciences Research Foundation. LSRF is a non-profit that has been awarding post-doctoral fellowships for more than 35 years. They have a stellar track record, but are only able to fund about half of around 50 people they believe to be deserving their fellowships every year. They fully depend on external donors to fund individual fellowships. Their current donors include HHMI, Open Philanthropy, and Amgen.
  • you might like the postdoc you end up supporting and become their supporter in the future
• foundations and philanthropists should not be afraid to commit themselves to specific scientists and fund them long-term

Finally, if you have upwards of a $50,000 available for charitable giving, consider reaching out to a scientist you like, and seeing if you can help them financially. They will be grateful for support and will probably be happy to grab an occasional dinner with you and tell you all about the latest developments in their field.

Relevance to the EA Community

I'm not sure whether this need to be justified, but trying to [safely] speed up scientific progress seems to be one of the highest leverage activities, in terms of improving global well-being. The EA community can help uniquely by e.g. setting up an EA Fund dedicated to funding science and providing short-term or long-term support to members of the community pursuing academic career-track or to identify scientists outside whose goals are aligned with those of the community. As far as I'm aware, currently there are no EA organisations focused on funding science (Open Philanthropy is focused on science that has relation to existential or global health risks) and while there are a lot of private foundations that are trying to correct misallocation of resources, they end up overlooking quite a few scientists who don't fit their cleanly defined funding theses.

Acknowledgements

I would like to thank (in reverse alphabetic order) Andrew Zuckerman, Guy Wilson, Brian Timar, Sasha Targ, Nabeel Qureshi, Manjari Narayan, Adam Marblestone, Anastasia Kuptsova, Bonnie Kavoussi, William Eden.

Thanks to Emergent Ventures (a) for financial support.

Appendix

Risk-aversion might be overstated

I think that it might be the case that the degree of risk-aversion in academia is overstated. My research suggests that those who want to pursue high-risk research usually find ways to do it, while those who don't want to, blame the system.

Grad students and postdocs in particular are usually able to pursue 2-3 projects simultaneously and can typically work on 1 project with high chance of success and 1-2 projects with low chance of success. It does definitely seem that you cannot pursue one project full-time and have this project be high-risk, because, if it fails completely, you'll have no publications and, without publications, you'll just not be able to get a job and stay in academia.

The alternative explanation of my observations about risk-taking is survivorship bias: I only see those for whom taking risks panned out. I can't figure out how much survivorship bias is present here, which is why I do not make any firm conclusions about the degree of risk-aversion in academia.

Studying people and animals is really hard

My IRB Nightmare (a) by Scott Alexander:

I feel like a study that realistically could have been done by one person in a couple of hours got dragged out into hundreds of hours of paperwork hell for an entire team of miserable doctors. I think its scientific integrity was screwed up by stupid requirements like the one about breaking blinding, and the patients involved were put through unnecessary trouble by being forced to sign endless consent forms screaming to them about nonexistent risks.

I feel like I was dragged almost to the point of needing to be in a psychiatric hospital myself, while my colleagues who just used the bipolar screening test – without making the mistake of trying to check if it works – continue to do so without anybody questioning them or giving them the slightest bit of aggravation.

I feel like some scientists do amazingly crappy studies that couldn’t possibly prove anything, but get away with it because they have a well-funded team of clerks and secretaries who handle the paperwork for them. And that I, who was trying to do everything right, got ground down with so many pointless security-theater-style regulations that I’m never going to be able to do the research I would need to show they’re wrong.

See discussion (a) on Andrew Gelman's blog + IRB nightmares (a). Several people I talked to say very similar things about IRB.

More funding ideas

Funding undergrads

I know several cases of brilliant undergrads who did not pursue scientific career because of external circumstances but for whom giving, say $50k would allow them to deal with these circumstances, fund exploring and pursuing science for a while and let them pursue the academic track.

High-quality clinical trials of generic medicines, nootropics, etc.

See An Iron Curtain Has Descended Upon Psychopharmacology (a) by Scott Alexander

Large-scale interventions

Mostly just fun thought experiments:

• forbid PIs from attaching their names to papers they didn't substantively contribute to and see how this changes their incentive structures
  • e.g. in economics, advisors rarely put their name on their students' papers

Peer review

"So who decides which people get the grants? It's their peers, who are all working on exactly the same things that everybody is working on. And if you submit a proposal that says "I'm going to go off and work on this crazy idea, and maybe there's a one in a thousand chance that I'll discover some of the secrets of the universe, and a 99.9% chance that I'll come up with bubkes," you get turned down." (a)

"A researcher made up whole trials, his co-authors at the very least were complicit by putting their names on his papers without checking anything, peer review failed to spot numerous problems with the studies, journals failed to react to red flags raised after publication" (a)

Miscellanea

“while switching topics may be easy in theory, certain fields are jealously guarded by more established Pis” (a)

"A European Research Council report suggests 79% of projects they fund “achieved a major scientific advance”, & only 1% make no contribution. Also, that they fund mostly “high risk” work" (a)

Chargaff (a famous biochemist) about meeting future discoverers of DNA structure:

“They impressed me by their extreme ignorance. Watson makes that clear! I never met two men who knew so little—and aspired to so much. They were going about it in a roguish, jocular manner, very bright young people who didn’t know much. They didn’t seem to know of my work, not even the structure and chemistry of the purines and pyrimidines. But they wanted to construct a helix’ a polynucleotide to rival Pauling’s alpha helix. They talked so much about ‘pitch’ that I wrote down afterwards, ‘Two pitchmen in search of a helix’. I explained to them the observations on the regularities in DNA, and told them adenine is complementary to thymine, guanine to cytosine, purines to pyrimidines; that any structure would have to take account of our complementary ratios. It struck me as a typical British intellectual atmosphere, little work and lots of talk.”

"Once upon a time I ran across an interesting RFA from the NIH that seemed highly targeted for one particular lab. Oh, don't get me wrong, many of them do seem this way. But this one was particularly.....specific. The funding was generous, we're talking BigMech territory here." (a)

"I suspect that peer review actually causes rather than mitigates many of the “troubling trends” recently identified by @zacharylipton and Jacob Steinhardt ... It’s very common for reviewers to read empirical papers and complain that there is no “theory”. But they don’t ask for theory to address any specific question. I think they are just looking for an easy reason to reject—-they skim and don’t see scary equations. … Reviewers seem to hate “science” papers, but it’s possible to sneak science in the door if add some token amount of new method engineering" (a)

“I spoke recently to a member of the panel that awarded me my first research grant on quantum computers (1985). He said it was a close-run thing. I asked if I'd have got it under today's criteria. He said: no chance; basically I could tick none of the boxes.” (a)

"Ioannidis and Trepanowski say that without RCTs, nutrition science can never credibly answer important questions about nutrition. That Harvard department's response is that RCTs are essentially impossible because you cannot get people to meaningfully change their diets over long periods of time. It seems clear to me that both sides are correct." (a)

90% of all claims about the problems with medical studies are wrong (a)

Beware Mass-Produced Medical Recommendations (a)