Yeah, we have mixed evidence on this - we didn't get on the hype leaderboard despite having enough votes, which suggests disfavoring, but we also just did get a lot of views and no weird ratios of e.g. views/comments so I don't personally think there was a lot going on there.
Basically you're right about all of the above, and we don't yet have an end metric we've figured out, so we don't know how much impact we're having! It's the biggest open question for our team right now, and I hope to get clarity on it in the next quarter or two and share that.
Re: Timelines - we don't have super strong views on timelines, but the amount of weight I put on short timelines do I think mean that any longer leads than an average of 3 months per video seem bad to me. Whether we should try to go for e.g. 1/week I'm not yet sure on.
Not sure if that’s the same distinction I would make but broadly just takes a long time to write a full script that we’re happy with, which includes figuring out the right structure, the high level narratives, the beats we want to hit, the takeaways, giving it a good emotional arc, etc.
Thanks for this. I’ve read through the whole thing though haven’t thought about the numbers in depth yet. I’m hoping to write a forum post with my retrospective on the AI in Context video at some point!
A few quick thoughts which I imagine won’t be very new to people:
Comment and comment analysis could also be a proxy for engagement and quality of engagement
Someone said that it would be hard to predict future success from AI in context based only on our one big video and I strongly agree. We're hoping to release our next one in the next month, and I'm really excited about it but by default, we should expect a lot of regression to the mean. (Note: I wouldn't even think of us as having two videos. The other one is just a channel trailer we threw up to have something to introduce people to the channel.)
I like this question on the value of subsequent viewer minutes, and I don't currently have a take. I think some complicating factors are: - For one thing, it seems like a 5-minute video wouldn't do very well on YouTube, so it's not like it's really an option necessarily to make a lot of 5-minute videos relative to one 45-minute video, because 45-minute videos just have a niche. You might still want to make more 20-minute videos and fewer 45-minute videos. - I'm also not convinced that effort scales with time. Certainly editing time does, but often what you're trying to do (at least what we're trying to do) is tell a story and there's a certain length that allows you to tell the story. And so it's not convertible or fungible in the way that it might naively appear. - To the point above about telling a story, I think part of the value of a video is whether people come away with like an overall sense of what the video is about in a way that's memorable/the takeaways, and that might require telling a good story. Some stories might take 20 minutes to tell and some stories might take 45 minutes to tell. Maybe you want to focus on the stories that take less time to tell if it takes you a lot less time or money to make the video, but as I say I don't think effort scales that way for us.
For what it’s worth we’re not currently focused really heavily on getting to the right target audience. We’re currently doing product validation to just see if we know how to make good videos, but will be excited to think about that more in the future.
E.g., the book is likely to become a NYT bestseller. The exact position can be improved by more pre-orders. (The figure is currently at around 5k pre-orders, according to the q&a; +20k more would make it a #1 bestseller).
Chat says about that
If preorders = 5k, you’re probably looking at 8k–15k total copies sold in week 1 (preorders + launch week sales).
Recently, nonfiction books debuting around 8k–12k week-1 copies often chart #8–#15 on the NYT list.
Lifetime Sales Ranges
Conservative: 20k–30k copies total (good for a nonfiction debut with moderate buzz).
Optimistic: 40k–60k (if reviews, media, podcasts, or TikTok keep it alive).
Breakout: 100k+ (usually requires either a viral moment, institutional adoption, or the author becoming part of a big public debate).
Is that a lot? I don't actually know, would be that’s not that many, but a decent number and might get a lot of buzz, commentary, etc. This is a major crux so I'd be interested in take.
If true: there might be an influx of people into this space, or who are hoping to get into this space AND the space could lose a lot of impact if it’s not ready to make use of this pipeline
I think the arguments here are clear but let me know if not
Therefore, people/orgs should be thinking about how to make the best pipelines for the inflow.
e.g.
If you have next steps for people (BlueDot, CEA, MATS), be ready to retweet / restack MIRI’s materials and be like “if you care about this, here’s a way to get involved”
Similarly, maybe pitch MIRI on putting your org / next steps on their landing page for the book and see if they think that makes sense
Landing page / resource hub: “So you just read the MIRI book?” page that curates your content, fellow orgs’ resources, and next steps. Make it optimized for search and linkable.
Yeah, we have mixed evidence on this - we didn't get on the hype leaderboard despite having enough votes, which suggests disfavoring, but we also just did get a lot of views and no weird ratios of e.g. views/comments so I don't personally think there was a lot going on there.
Basically you're right about all of the above, and we don't yet have an end metric we've figured out, so we don't know how much impact we're having! It's the biggest open question for our team right now, and I hope to get clarity on it in the next quarter or two and share that.
Re: Timelines - we don't have super strong views on timelines, but the amount of weight I put on short timelines do I think mean that any longer leads than an average of 3 months per video seem bad to me. Whether we should try to go for e.g. 1/week I'm not yet sure on.
Thanks so much for your comment!
Not sure if that’s the same distinction I would make but broadly just takes a long time to write a full script that we’re happy with, which includes figuring out the right structure, the high level narratives, the beats we want to hit, the takeaways, giving it a good emotional arc, etc.
That means the world, Toby, on behalf of the whole team, thank you!
Well everything in American politics sucks.
But! There are probably things to actually do about it.
I’m looking for competent, agentic people to volunteer, fundraise or work full time on evidence based preservation of american democracy.
(not in 80k capacity)
DM me if interested.
IMO that's a different category - there's a lot of that kind of thing as well and I'm glad it exists but I think it's useful to separate out.
Thank you!
Oh, nice!
Thanks for this. I’ve read through the whole thing though haven’t thought about the numbers in depth yet. I’m hoping to write a forum post with my retrospective on the AI in Context video at some point!
A few quick thoughts which I imagine won’t be very new to people:
- For one thing, it seems like a 5-minute video wouldn't do very well on YouTube, so it's not like it's really an option necessarily to make a lot of 5-minute videos relative to one 45-minute video, because 45-minute videos just have a niche. You might still want to make more 20-minute videos and fewer 45-minute videos.
- I'm also not convinced that effort scales with time. Certainly editing time does, but often what you're trying to do (at least what we're trying to do) is tell a story and there's a certain length that allows you to tell the story. And so it's not convertible or fungible in the way that it might naively appear.
- To the point above about telling a story, I think part of the value of a video is whether people come away with like an overall sense of what the video is about in a way that's memorable/the takeaways, and that might require telling a good story. Some stories might take 20 minutes to tell and some stories might take 45 minutes to tell. Maybe you want to focus on the stories that take less time to tell if it takes you a lot less time or money to make the video, but as I say I don't think effort scales that way for us.
What are we doing about the MIRI book inbound?
Claim: The MIRI book might be a very big deal, read by lots of people
Mostly this is on vibes, and the MIRI team trying hard and seeming very successful and getting a lot of buzz, great blurbs, some billboards, etc.
I saw this tweet
Chat says about that
Is that a lot? I don't actually know, would be that’s not that many, but a decent number and might get a lot of buzz, commentary, etc. This is a major crux so I'd be interested in take.
If true: there might be an influx of people into this space, or who are hoping to get into this space AND the space could lose a lot of impact if it’s not ready to make use of this pipeline
I think the arguments here are clear but let me know if not
Therefore, people/orgs should be thinking about how to make the best pipelines for the inflow.
e.g.
Very interested in takes!