I like the tree metaphor! I've always thought of it as a ladder of premises. You climb your way up the ladder, starting with the basics. Every step up the ladder, you lose some people. And if you start really high up the ladder, some people might get confused because they don't understand the fundamental premises.
But I see how the branching metaphor can lead to the different viewpoints.
1. Life & quality of life matter.
2. They can be quantified.
3. More is better, e.g. 2 lives saved is 2x better than 1 life saved. (Effective)
4. You have the ability to do good by improving others lives & the quality of their lives. (Altruism)
Maybe here it branches...
A. Geography doesn't matter, a life across the world matters the same amount as a life in your country -> Global Health.
C. Time doesn't matter, a life in the future matters the same amount as a life in the present. + There might be a lot of future humans + Reducing existential risk is the most effective way of addressing this -> Longtermism
Ratio of Installs to DAU: Hmm, that's an interesting metric...the way I think about retention is like a layered cake, kind of like the baumkuchen I just ate for breakfast, but linear instead of round. Anyways, there's time on the X axis and users on the Y axis. For any given day, there's a sizeable layer of cream at the top which are the Day 0 users. And then right below that, a smaller layer of Day 1 users, etc. etc. Ultimately there are hundreds of layers of users from older daily cohorts. You can track each daily cohort through time and it'll start big and then shrink rapidly, following the retention curves, until eventually it flatlines at some point (ideally above 0).
So you could look at overall installs to DAU, but that gives an advantage to new apps because they don't have a lot of old installs from years-old cohorts that have left. Or you could compare daily installs to DAU, but that gives an advantage to old apps because they'll have a lot of users from old cohorts.
A better metric could be DAU / MAU ratio, which measures like out of all of your active users, how many of them use the app every day. Here ~25% would be exceptional with an average of probably around 10%. But that's also biased based on how many new users you're bringing in each day.
CPI: Yes, those numbers are from Facebook / Instagram / TikTok ads.
In terms of organic traffic, it's also a measure of time. Say for example you're bringing in 1000 organic users a day. After a year, that's 365k users. After 5 years that's 1.8M users. Of course, the app still has to remain good to continue getting organic downloads. Since the definition of good is always improving, the app would need to be consistently updated.
I'd say estimate around 2/3 of our lifetime installs are organic, but it really depends on the app. I speculate that Daylio might be closer to 100% organic while Breeze is probably closer to 0%.
I pretty much agree with your broad points. Some quick thoughts on each section from an "insider" point of view:
Short-term impacts of treatment: The problem with these studies is that they study "apps" as a category, but apps are 1. extremely different from one another and vary drastically in quality and 2. are constantly improving. The best apps are probably 10-100x better than the worst mental health apps. Also, functionality of the best apps today are very different than the best apps from a few years ago when these studies were run. In the future, they'll probably be even more different. I don't think we're close to peak mental health app effectiveness.
Self-guided app adherence: Apple actually has made benchmarks available! I can grab them for you. For health and fitness apps which rely on a subscription business model, here are the 25th / 50th / 75th percentiles.
Day 1: ~16% / ~25% / ~34% Day 7: ~3% / ~7% / ~13% Day 28: ~0.75% / ~2.5% / ~6%
Note: The exact percentages vary by a little bit depending on which week is selected.
User acquisition costs: I can confirm there are huge differences in cost per install between US iOS (in the range of $1-2 per install) vs low-income country Android ($0.02 - $0.10). Although of course, ads aren't the only way to get app downloads. Organic app downloads are $0.00!
Development costs: If you were to put us on your comparison, we'd be at 2 employees (my wife and I) @ ~8,000,000 downloads across all of our apps. I can confirm that service and hosting costs are basically negligible.
How to improve: I agree that marketing to low-CPI regions could be a great impact opportunity. I'm most excited about improving drop-off rates / effect sizes (two sides of the same coin). I can absolutely imagine a world where a self-guided mental health app is 10x more effective than the best one available now.
Again, great overview! Let me know if I can be helpful!
I would lean towards making the first version a website instead so you wouldn't have to go through the hassle of following Apple / Google's rules. For example in the App Store Review Guidelines there are extra procedures to follow for such apps - https://developer.apple.com/app-store/review/guidelines/ it explicitly states:
3.2.1 Acceptable: (vi) Approved nonprofits may fundraise directly within their own apps or third-party apps, provided those fundraising campaigns adhere to all App Review Guidelines and offer Apple Pay support. These apps must disclose how the funds will be used, abide by all required local and federal laws, and ensure appropriate tax receipts are available to donors. Additional information shall be provided to App Review upon request. Nonprofit platforms that connect donors to other nonprofits must ensure that every nonprofit listed in the app has also gone through the nonprofit approval process. Learn more about becoming an approved nonprofit.
3.2.2 Unacceptable: (iv) Unless you are an approved nonprofit or otherwise permitted under Section 3.2.1 (vi) above, collecting funds within the app for charities and fundraisers. Apps that seek to raise money for such causes must be free on the App Store and may only collect funds outside of the app, such as via Safari or SMS.
At least for me, it was hard to tell the hierarchy of the content. I wonder if a table of contents might be helpful?
I think the issue stems from H3 and H4 tags being hard for me to tell apart, so a little confusing to subconsciously keep track of where I was in the document. Another problem could be the "What values unite effective altruism?" and "What are some examples of effective altruism in practice?" are H3 but "How can you take action?" and "FAQ" are H2 but in my mind they should all have been at the same level? Maybe just promoting the first two headers to H2 would be good enough to solve most of my confusion.
Also, the preview image when sending the link to someone on LinkedIn strikes me as a little odd and might hinder virality when it's time to share it on social media.
Ideas for Iteration
If the intro article takes off and becomes the "top of the funnel" of effective altruism for a lot of people, optimizing the "conversion rate" of this article could have big downstream effects.
I would definitely encourage collecting 1 on 1 feedback by having people new to EA read the content in person and speak their thoughts out loud.
Qualitative feedback can also be gathered more quickly with a tool like Intercom to directly chat with people while they're reading it and hear their thoughts / answer their questions.
It might also be a good idea to get some quantitative feedback with a tool like hotjar to see how far people scroll.
If the goal of the article is to get people intrigued with EA and diving deeper, perhaps emphasizing the "How can you take action" or the other parts in the "What's next?" section with special graphics or banners kind of like the newsletter box would be helpful. Then you can A/B test different iterations to see what gets people to tap more.
Speaking of A/B tests, you might be able to squeeze out a few more percentage of engagement by experimenting with the order of the examples, the number of examples, which examples are shown, the actual words in the content itself, the preview image, etc.
I was just thinking about this the other day. In terms of pitching effective altruism, I think it's best to keep things simple instead of overwhelming people with different concepts. I think we can boil down your moral claims to essentially 3 core beliefs of EA:
Doing good is good. (Defining good)
It is more good to do more good. (Maximization)
Therefore, we ought to do more good. (Moral obligation)
If you buy these three beliefs, great! You can probably consider yourself an effective altruist or at least aligned with effective altruism. Everything else is downstream of these 3 beliefs and up for debate (and EAs excel at debating!).
I'm not confident in the belief that Meta as a company causes more harm than good. If you look closely at most of the criticism, it's generally overblown.
I'm more convinced of the overall harms of social media (esp. in regards to teenage mental health), but is Meta is worse than other social media companies out there like TikTok? Probably not?
In terms of benefits to the world, they enable a lot of small businesses to exist through personalized advertising and they're pushing the field of virtual reality forward faster than it would otherwise develop.
If I had to guess, Meta's impact on the world (compared to a world without Meta) is net neutral or maybe slightly positive (/negative), instead of significant negative impact. So your impact of joining Meta as 1 of 70k+ employees is likely a few orders of magnitude lower than the impact you'd have through earning to give.
Other factors to consider - What team would you be on? Can you get into other FAANGs? How long would you even work at Meta before you jump jobs? Is there a significant reputational risk of working at Meta, regardless of the facts?
Depending on the study design, it can actually be relatively cheap! Just use Amazon Mechanical Turk or Positly to get participants. Of course, some study designs would be 20x harder than others. I imagine that testing different self help interventions would be rather hard but smaller surveys would be really easy.
Here are my notes I took on the article at the time:
Pattern is that science discovers something important, a long time passes, then a crusader spreads it to the masses.
Variolation (an early form of vaccination): Invented in 1000s, 700 years later, popularized by Mary Montagu, an aristocrat in 1716. IMPACT!
In 1858, a JOURNALIST published an expose denouncing MILK as a liquid poison. Pasteur invented pasteurization by 1865. 30 years later. Nathan Straus in 1892 took up the cause, ran an experiment, published it, started an extended advocacy campaign. IMPACT!
John Leal, a sanitary adviser, quietly added chlorine to the city water. It was a success. It spread from city to city! Triumph of applied chemistry. GLORIOUS!
Peniccillin: Fleming invented it very early, but 2 Oxford scientists 10 years later did experiments and spread it. GO OXFORD!
Oral Rehydration Therapy: A johns-hopkins educated physician & researcher pioneered the use case, went against standard of care at the time, and crushed it! Started an education campaign (advocacy) letting amateurs into the act. IMPACT!
I like the tree metaphor! I've always thought of it as a ladder of premises. You climb your way up the ladder, starting with the basics. Every step up the ladder, you lose some people. And if you start really high up the ladder, some people might get confused because they don't understand the fundamental premises.
But I see how the branching metaphor can lead to the different viewpoints.
1. Life & quality of life matter.
2. They can be quantified.
3. More is better, e.g. 2 lives saved is 2x better than 1 life saved. (Effective)
4. You have the ability to do good by improving others lives & the quality of their lives. (Altruism)
Maybe here it branches...
A. Geography doesn't matter, a life across the world matters the same amount as a life in your country -> Global Health.
B. Humans aren't special. Animal lives matter too. -> Animal Welfare.
C. Time doesn't matter, a life in the future matters the same amount as a life in the present. + There might be a lot of future humans + Reducing existential risk is the most effective way of addressing this -> Longtermism
Ratio of Installs to DAU: Hmm, that's an interesting metric...the way I think about retention is like a layered cake, kind of like the baumkuchen I just ate for breakfast, but linear instead of round. Anyways, there's time on the X axis and users on the Y axis. For any given day, there's a sizeable layer of cream at the top which are the Day 0 users. And then right below that, a smaller layer of Day 1 users, etc. etc. Ultimately there are hundreds of layers of users from older daily cohorts. You can track each daily cohort through time and it'll start big and then shrink rapidly, following the retention curves, until eventually it flatlines at some point (ideally above 0).
So you could look at overall installs to DAU, but that gives an advantage to new apps because they don't have a lot of old installs from years-old cohorts that have left. Or you could compare daily installs to DAU, but that gives an advantage to old apps because they'll have a lot of users from old cohorts.
A better metric could be DAU / MAU ratio, which measures like out of all of your active users, how many of them use the app every day. Here ~25% would be exceptional with an average of probably around 10%. But that's also biased based on how many new users you're bringing in each day.
By the way, the only peer group benchmarks that apple provides are Conversion Rate, Proceeds per Paying User, Crash Rate, Day 1/7/28 Retention. https://developer.apple.com/app-store/peer-group-benchmarks/ . But they might be announcing more in March thanks to the EU's DMA. https://developer.apple.com/support/dma-and-apps-in-the-eu/#app-analytics
CPI: Yes, those numbers are from Facebook / Instagram / TikTok ads.
In terms of organic traffic, it's also a measure of time. Say for example you're bringing in 1000 organic users a day. After a year, that's 365k users. After 5 years that's 1.8M users. Of course, the app still has to remain good to continue getting organic downloads. Since the definition of good is always improving, the app would need to be consistently updated.
I'd say estimate around 2/3 of our lifetime installs are organic, but it really depends on the app. I speculate that Daylio might be closer to 100% organic while Breeze is probably closer to 0%.
Great overview! I'm Eddie, long-time EA and co-creator of the mental health app Clarity https://apps.apple.com/us/app/clarity-cbt-thought-diary/id1010391170
I pretty much agree with your broad points. Some quick thoughts on each section from an "insider" point of view:
Short-term impacts of treatment: The problem with these studies is that they study "apps" as a category, but apps are 1. extremely different from one another and vary drastically in quality and 2. are constantly improving. The best apps are probably 10-100x better than the worst mental health apps. Also, functionality of the best apps today are very different than the best apps from a few years ago when these studies were run. In the future, they'll probably be even more different. I don't think we're close to peak mental health app effectiveness.
Self-guided app adherence: Apple actually has made benchmarks available! I can grab them for you. For health and fitness apps which rely on a subscription business model, here are the 25th / 50th / 75th percentiles.
Day 1: ~16% / ~25% / ~34%
Day 7: ~3% / ~7% / ~13%
Day 28: ~0.75% / ~2.5% / ~6%
Note: The exact percentages vary by a little bit depending on which week is selected.
User acquisition costs: I can confirm there are huge differences in cost per install between US iOS (in the range of $1-2 per install) vs low-income country Android ($0.02 - $0.10). Although of course, ads aren't the only way to get app downloads. Organic app downloads are $0.00!
Development costs: If you were to put us on your comparison, we'd be at 2 employees (my wife and I) @ ~8,000,000 downloads across all of our apps. I can confirm that service and hosting costs are basically negligible.
How to improve: I agree that marketing to low-CPI regions could be a great impact opportunity. I'm most excited about improving drop-off rates / effect sizes (two sides of the same coin). I can absolutely imagine a world where a self-guided mental health app is 10x more effective than the best one available now.
Again, great overview! Let me know if I can be helpful!
I would lean towards making the first version a website instead so you wouldn't have to go through the hassle of following Apple / Google's rules. For example in the App Store Review Guidelines there are extra procedures to follow for such apps - https://developer.apple.com/app-store/review/guidelines/ it explicitly states:
3.2.1 Acceptable: (vi) Approved nonprofits may fundraise directly within their own apps or third-party apps, provided those fundraising campaigns adhere to all App Review Guidelines and offer Apple Pay support. These apps must disclose how the funds will be used, abide by all required local and federal laws, and ensure appropriate tax receipts are available to donors. Additional information shall be provided to App Review upon request. Nonprofit platforms that connect donors to other nonprofits must ensure that every nonprofit listed in the app has also gone through the nonprofit approval process. Learn more about becoming an approved nonprofit.
3.2.2 Unacceptable: (iv) Unless you are an approved nonprofit or otherwise permitted under Section 3.2.1 (vi) above, collecting funds within the app for charities and fundraisers. Apps that seek to raise money for such causes must be free on the App Store and may only collect funds outside of the app, such as via Safari or SMS.
https://archive.ph/6AwJo
LOVE the new intro article!!
Feedback
At least for me, it was hard to tell the hierarchy of the content. I wonder if a table of contents might be helpful?
I think the issue stems from H3 and H4 tags being hard for me to tell apart, so a little confusing to subconsciously keep track of where I was in the document. Another problem could be the "What values unite effective altruism?" and "What are some examples of effective altruism in practice?" are H3 but "How can you take action?" and "FAQ" are H2 but in my mind they should all have been at the same level? Maybe just promoting the first two headers to H2 would be good enough to solve most of my confusion.
Also, the preview image when sending the link to someone on LinkedIn strikes me as a little odd and might hinder virality when it's time to share it on social media.
Ideas for Iteration
If the intro article takes off and becomes the "top of the funnel" of effective altruism for a lot of people, optimizing the "conversion rate" of this article could have big downstream effects.
I would definitely encourage collecting 1 on 1 feedback by having people new to EA read the content in person and speak their thoughts out loud.
Qualitative feedback can also be gathered more quickly with a tool like Intercom to directly chat with people while they're reading it and hear their thoughts / answer their questions.
It might also be a good idea to get some quantitative feedback with a tool like hotjar to see how far people scroll.
If the goal of the article is to get people intrigued with EA and diving deeper, perhaps emphasizing the "How can you take action" or the other parts in the "What's next?" section with special graphics or banners kind of like the newsletter box would be helpful. Then you can A/B test different iterations to see what gets people to tap more.
Speaking of A/B tests, you might be able to squeeze out a few more percentage of engagement by experimenting with the order of the examples, the number of examples, which examples are shown, the actual words in the content itself, the preview image, etc.
I was just thinking about this the other day. In terms of pitching effective altruism, I think it's best to keep things simple instead of overwhelming people with different concepts. I think we can boil down your moral claims to essentially 3 core beliefs of EA:
If you buy these three beliefs, great! You can probably consider yourself an effective altruist or at least aligned with effective altruism. Everything else is downstream of these 3 beliefs and up for debate (and EAs excel at debating!).
I'm not confident in the belief that Meta as a company causes more harm than good. If you look closely at most of the criticism, it's generally overblown.
For example, the headline of the 2021 Facebook Leak is "Instagram Harms Teenagers!" but the reality is more complicated - https://www.npr.org/2021/10/06/1043138622/facebook-instagram-teens-mental-health . Cambridge Analytica too: https://mobiledevmemo.com/cambridge-analytica-was-a-false-panic-its-time-to-move-on/ .
I'm more convinced of the overall harms of social media (esp. in regards to teenage mental health), but is Meta is worse than other social media companies out there like TikTok? Probably not?
In terms of benefits to the world, they enable a lot of small businesses to exist through personalized advertising and they're pushing the field of virtual reality forward faster than it would otherwise develop.
If I had to guess, Meta's impact on the world (compared to a world without Meta) is net neutral or maybe slightly positive (/negative), instead of significant negative impact. So your impact of joining Meta as 1 of 70k+ employees is likely a few orders of magnitude lower than the impact you'd have through earning to give.
Other factors to consider - What team would you be on? Can you get into other FAANGs? How long would you even work at Meta before you jump jobs? Is there a significant reputational risk of working at Meta, regardless of the facts?
Depending on the study design, it can actually be relatively cheap! Just use Amazon Mechanical Turk or Positly to get participants. Of course, some study designs would be 20x harder than others. I imagine that testing different self help interventions would be rather hard but smaller surveys would be really easy.
Personally, I found this article to be full of inspiring people for me - https://www.nytimes.com/2021/04/27/magazine/global-life-span.html
Here are my notes I took on the article at the time:
PS. If you want to read the article but can't, try this extension https://chrome.google.com/webstore/detail/disable-javascript/jfpdlihdedhlmhlbgooailmfhahieoem?hl=en-US