Glad you're working with some of the people I recommended to you, I'm very proud of that SB-1047 documentary team.
I would add to the list Suzy Shepherd who made Writing Doom. I believe she will relatively soon be starting another film. I wrote more about her work here.
For context, you asked me for data for something you were planning (at the time) to publish day-off. There's no way to get the watchtime easily on TikTok (which is why I had to do manual addition of things on a computer) and I was not on my laptop, so couldn't do it when you messaged me. You didn't follow up to clarify that watchtime was actually the key metric in your system and you actually needed that number.
Good to know that the 50 people were 4 Safety people and 46 people who hang at Mox and Taco Tuesday. I understand you're trying to reach the MIT-graduate working in AI who might somehow transition to AI Safety work at a lab / constellation. I know that Dwarkesh & Nathan are quite popular with that crowd, and I have a lot of respect for what Aric (& co) did, so the data you collected make a lot of sense to me. I think I can start to understand why you gave a lower score to Rational Animations or other stuff like AIRN.
I'm now modeling you as trying to answer something like "how do we cost-effectively feed AI Safety ideas to the kind of people who walk in at Taco Tuesday, who have the potential to be good AI Safety researchers". Given that, I can now understand better how you ended up giving some higher score to Cognitive Revolution and Robert Miles.
Agreed about the need to include Suzy Shepherd and Siliconversations.
Before Marcus messaged me I was in the process of filling another google sheets (link) to measure the impact of content creators (which I sent him) which also had like three key criteria (production value, usefulness of audience, accuracy).
I think Suzy & Siliconversations are great example of effectiveness because:
The thing I wanted to measure (which I think is probably a bit much harder than just estimating things with weights then multiplying by minutes of watchtime) is "what kind of content leads more people to take action like Siliconversations", and I'm not sure how to measure that except if everyone had CTAs that they tracked and we could compare the ratios.
The reason I think Siliconversations' video lead to so many emails was that he was actually relentless in this video about sending emails, and that was the entire point of the video, instead of like talking about AI risk in general, and having a link in the comments.
I think this is also why that RA x ControlAI collab got less emails, but it also got way more views that potentially in the future will lead to a bunch of people that will do a lot of useful things in the world, though that's hard to measure.
I know that 80k's AI In Context has a full section at the end on "What to do" saying to look at the links in description. Maybe Chana Mesinger has data on how many people clicked on how much traffic was redirected from YT to 80k.
Update: after looking at Marcus' weights, I ended up dividing all the intermediary values of Qf I had by 2, so that it matches with Marcus' weights where Cognitive Revolution = 0.5. Dividing by 2 caps the best tiktok-minute to the average Cognitive Revolution minute. Neel was correct to claim that 0.9 was way too high.
===
My model is that most of the viewer minutes come from people who watch the all thing, and some decent fraction end up following, which means they'll end up engaging more with AI-Safety-related content in the future as I post more.
Looking at my most viewed TikTok:
TikTok says 15.5% of viewers (aka 0.155 * 1400000 = 217000) watched the entire thing, and most people who watch the first half end up watching until the end (retention is 18% at half point, and 10% at the end).
And then assuming the 11k who followed came from those 217000 who watched the whole thing, we can say that's 11000/217000 = 5% of the people who finished the video that end up deciding to see more stuff like that in the future.
So yes, I'd say that if a significant fraction (15.5%) watch the full thing, and 0.155*0.05 = 0.7% of the total end up following, I think that's "engaging properly".
And most importantly, most of the viewer-minutes on TikTok do come from these long videos that are 1-4 minutes long (especially ones that are > 2 minutes long):
Happy for others to come up with different numbers / models for this, or play with my model through the "TikTok Qa & Qf" sheet here, using different intermediary numbers.
Update: as I said at the top, I was actually wrong to have initially said Qf=0.9 given the other values. I now claim that Qf should be closer to 0.45. Neel was right to make that comment.
This comment is answering "TikTok I expect is pretty awful, so 0.1 might be reasonable there". For my previous estimate on the quality of my Youtube long-form stuff, see this comment.
tl;dr: I now estimate the quality of my TikTok content to be Q = 0.75 * 0.45 * 3 = 1
To estimate fidelity of message (Qf) and alignment of message (Qm) in a systematic way, I compiled my top 10 most performing tiktoks and ranked their individual Qf and Qm (see tab called "TikTok Qa & Qf" here, which contains the reasoning for each individual number).
Update Sep 14: I've realized that my numbers about fidelity used 1 as the maximum, but now that I've looked at Marcus' weights for other stuff, I think I should use 0.5 because that's the number he gives to a podcast like Cognitive Revolution, and I don't want to claim that a long tiktok clip is more high-fidelity than the average Cognitive Revolution podcast. So I divided everything by 2 so my maximum fidelity is now 0.5 to match Marcus' other weights.
Then, by doing a minute-adjusted weighted average of the Qas and Qfs I get:
What this means:
I believe the original reasoning for Qa = 2 is that people watching short-form by default would be young and / or have short attention spans, and therefore be less of a high-quality audience.
However, most of my high-performing TikTok clips (that represent most of the watch time) are quite long (2m-3m30s long), which makes me think the kind of audience who watch these until the end are not as different from Youtube.
On top of that, my audience a) skews towards US (33%) or high-income countries (more than half are in US / Australian / UK etc.) and 88% of my audience being over 25, with 61% being above 35. (Data here).
Therefore, in terms of quality of audience, I don't see why the audience would be worse in quality than people who watch AI Species / AI Risk Network.
Which is why I'm estimating: Qa(The Inside View TikTok) = 3.
If we multiply these three numbers we get Q = 0.75 * 0.45 * 3 = 1
Agreed that the quality of audience is definitely higher for my (niche) AI Safety content on Youtube, and I'd expect Q to be higher for (longform) Youtube than Tiktok.
In particular, I estimate Q(The Inside View Youtube) = 2.7, instead of 0.2, with (Qa, Qf, Qm) = (6, 0.45, 1), though I acknowledge that Qm is (by definition) the most subjective.
To make this easier to read & reply to, I'll post my analysis for Q(The Inside View Tiktok) in another comment, which I'll link to when it's up. EDIT: link for TikTok analysis here.
In light of @Drew Spartz's comment (saying one way to quantify the quality of audience would be to look at the CPM [1]), I've compiled my CPM Youtube data and my average Playback-based CPM is $14.8, which according to this website [2] would put my CPM above the 97.5 percentile in the UK, and close to the 97.5 percentile in the US.
Now, this is more anecdotal evidence than data-based, but I've met quite a few people over the years (from programs like MATS, or working at AI Safety orgs) who've told me they discovered AI Safety from my Inside View podcast. And I expect the SB-1047 documentary to have attracted a niche audience interested in AI regulation.
Given the above, I think it would make sense to have the Qa(Youtube) be between 6 (same as other technical podcasts) and 12 (Robert Miles). For the sake of giving a concrete number, I'll say 6 to be on par with other podcasts like FLI and CR.
In the paragraph below I'll say Qf_M for the Qf that Marcus assigns to other creators.
For the fidelity of message, I think it's a bit of a mixed bag here. As I said previously, I expect the podcasts that Nathan would be willing to crosspost to be on par with his channel's quality, so in that sense I'd say the fidelity of message for these technical episodes (Owain Evans, Evan Hubinger) to be on par with CR (Qf_M = 0.5). Some of my non-technical interviews are probably closer to discussions we could find on Doom Debates (Qf_M = 0.4), though there are less of them. My SB-1047 documentary is probably similar in fidelity of message to AI in context (Qf_M = 0.5), and this fictional scenario is very similar to Drew's content (Qf_M = 0.5). I've also posted video explainers that range from low effort (Qf around 0.4?) to very high effort (Qf around 0.5?).
Given all of the above, I'd say the Qf for the entire channel is probably around 0.45.
As you say, for the alignment of message, this is probably the most subjective. I think by definition the content I post is the message that aligns the most with my values (at least for my Youtube content) so I'd say 1 here.
Multiplying these numbers I get Q = 2.7. Doing a sanity check, this seems about the same as Cognitive Revolution, which doesn't seem crazy given we've interviewed similar people & the cross-post arguments I've said before.
(Obviously if I was to modify all of these Qa, Qf, Qm numbers for all channels I'd probably end up with different quality comparisons).
CPM means Cost Per Mille. In YT Studio it's defined as "How much advertisers pay every thousand times your Watch Page content is viewed with ads."
I haven't done extended research here and expect I'd probably get different results looking at different websites. This one was the first one I found on google so not cherry-picked.
Thank you both for doing this, I appreciate the effort in trying to get some estimates.
However, I would like to flag that your viewer minute numbers for my short-form content are off by an order of magnitude. And I've done 4 full weeks on the Manifund grant, so it's 4 * $2k = $8k, not $24k.
Plugging these numbers in (google sheets here) I get a QAVM/$ of 389 instead of the 18 you have listed.
Other data corrections:
If we now look at youtube explainers:
Regarding your weights, you place both my TikTok and Youtube channel at 0.1 and 0.2 in quality, which I find surprising, especially Youtube:
Overall, I’m a bit disappointed by your data errors given that I replied to you by DM saying that your first draft missed a lot of important factors & data, and suggested helping you / delaying publication, which you refused.
Update: I've now estimated the quality for my long-form youtube content to be Q= 6*0.45*1 = 2.7 for Youtube, and Q=3*0.9*0.75=2.0 Q = 3*0.45*0.75 = 1.0 for TikTok. See details here for Youtube, and here for TikTok. Using these updated weights (see "Michael's weights" here) I get this final table.
Relatedly, recently in UK Politics x AI Safety:
- ControlAI released a statement signed by 60 UK parliamentarians: https://controlai.com/statement
- Same happened through PauseAI: https://pauseai.info/dear-sir-demis-2025 (see TIME)
Thanks! Just want to add some counterpoints and disclaimers to that:
- 1. I want to flag that although I've filmed & edited ~20 short-form clips in the past (eg. from June 2022 to July 2025) around things like AI Policy and protests, most of the content I've recently been posting as just been clips from other interviews. So I think it would also be unfair to compare my clips and original content (both short-form and longform), which is why I wrote this post. (I started doing this because I ran out of footage to edit shortform videos as I was trying to publish one TikTok a day, and these clips eventually reached way more people than what I was doing before, so I transitioned to doing that).
- 2. regarding comparing to high-production videos: I don't want to come across as saying we shouldn't compare work of different length or using different budgets. I think Marcus and Austin's attempt is honorable. Also, being able to correctly use a large budget to make a high-production video that reaches as many people as many lower budget videos requires a lot of skill, though once you have that level of skill then the amount of time you spend on a video to make it really good ends up leading to exponential results in views (if you make something that is 10% better, Youtube will push it much more than 10% more).