Thanks, helpful to understand the limitations of this tool

Did a decent job for this academic paper, but I think it’s hampered by only having content from Arxiv and various EA/tech forums. Still, it generated some interesting leads.

https://gistcdn.githack.com/daaronr/b9447c40a7a6b948f399073496f98c37/raw/scanner_elasticity_experts.html

Prompt:

... find the most relevant authors and work for Observational price variation in scanner data cannot reproduce experimental price elasticities https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4899765 -- we're looking for methodological experts to evaluate this for The Unjournal to inform our pivotal question "How do plant-based products substitute for animal products (welfare footprint)?"

Trying this out for using for various Unjournal.org processes (like prioritizing research, finding potential evaluators, linking research to pivotal questions) and projects (assessing LLM vs human research evaluations). Some initial forays (comming from a conversation with Xyra). I still need to human-check it. 

~prompt to Claude code about @Toby_Ord  and How Well Does RL Scale? 

``Toby Ord's writing -- what do the clusters look like? What other research/experts come closest to his post ....  https://forum.effectivealtruism.org/posts/TysuCdgwDnQjH3LyY/how-well-does-rl-scale``

This interactive visual (with some extra prompts)

Key Takeaways

• For “How Well Does RL Scale?”, the closest intellectual neighbors are:
• ML researchers focused on compute efficiency and scaling laws (e.g. 1a3orn, Pablo Villalobos), and
  • AI safety researchers analyzing inference scaling implications (e.g. Ryan Kidd, ryan_greenblatt).
• Stuart Armstrong appears to be the only researcher in this corpus writing substantively across all three of Toby Ord’s main intellectual themes:
  • AI scaling
  • Existential risk / longtermism
  • Infinite ethics and decision theory

Most Similar Researchers to“How Well Does RL Scale?”

Top Authors (by Semantic Similarity)

  @1a3orn  @Pablo Villalobos  @Matrice Jacobine🔸🏳️‍⚧️  @Lukas Finnveden @Ryan Greenblatt @Ryan Kidd  -- if you have a chance, let me know if this is accurate/relevant. 

Most Similar arXiv Paper

Toby Ord’s Writing Clusters

Based on semantic analysis, Toby Ord’s writing appears to fall into three main thematic clusters.

Cluster 1: AI Scaling & Timelines (Recent)

• “How Well Does RL Scale?” (Oct 2025)
  • Core claim: RL training scales poorly; most gains come from inference scaling (longer chains of thought), not training compute.

  • Key implication: This likely lengthens AI timelines and affects governance and safety strategies.

     

Cluster 2: Existential Risk & Longtermism

• The Precipice and related work
• “The Precipice Revisited” (Jul 2024)
• “Shaping Humanity’s Longterm Trajectory” (Jul 2023)
• “Robust Longterm Comparisons” (May 2024)

Topics include:

• Existential risk estimation
• Discounting
• Anthropic effects / anthropic shadow

Cluster 3: Infinite Ethics & Decision Theory

• “Evaluating the Infinite” (Sep 2025) — hyperreal-based approach to divergent sums and integrals
• “What’s wrong with the dogmatic solution to recklessness?” (Feb 2023)
• “The Ethics of Copying Conscious States” (Aug 2025)

Topics include:

• Pascal’s Mugging
• Unbounded utility
• Decision-making under infinity

At the Unjournal we have a YouTube channel and I'm keen to produce more videos both about our process and the case for our model, and about the content of the research we evaluate and the pivotal questions we consider, which are generally EA-adjacent. This includes explainer videos, making the case videos, interviews/debates, etc. 

But as you and most organizations probably realize, it's challenging and very time-consuming to produce high-quality videos, particularly in terms of creating and synchronizing images, sound, and video editing, etc. Without a dedicated communications/media person with time and skills to do this, it's a big ask. I've tried a bit to do this with simple tools like iMovie as you'll see and I've experimented with AI-powered tools like Descript but I haven't found these easy to adopt. Maybe there'll be a rapid improvement coming soon making it much easier.

Anyways I see value in some resource sharing skills and tips and templates for this among EA/aligned organizations; looking for suggestions and links. 



 

Should be fixed now, thanks. Problem was because I started by duplicating the first one and then adjusting the text. But the text only showed changed on my end (NB: @EA Forum Team )

Thanks -- that's not how it looked on my end, will try to adjust.

Some notes/takes:

The Effective Giving/EA Marketing project was going fairly strong, making some progress and also some limitations. But I wouldn't take the ~shutdown/pause as strong evidence against this approach. I'd diagnose it as:

1. Some disruption from changes in emphasis/agenda at a few points in the project, driven by the changing priorities in EA at the time, first towards "growing EA rather than fundraising"  (Let's stop saying 'funding overhang', etc.) and then somewhat back in the other direction after the collapse of FTX

2.  I got a grant and encouragement to pursue Unjournal.org full time, so I put this on hold, and no one stepped up to take it on in its current form

3. Some aspects may be partially covered by other initiatives such as @Lucas Moore 's work bringing together EA giving/fundraising orgs at Giving What We Can and marketing initiatives like @Good Impressions -- some of these are discussed and listed within that resource

(Let's have a chat -- I will dm.)
 

Here's the Unjournal evaluation package

A version of this work has been published in the International Journal of Forecasting under the title "Subjective-probability forecasts of existential risk: Initial results from a hybrid persuasion-forecasting tournament"
 

We're working to track our impact on evaluated research (see coda.io/d/Unjournal-...) So We asked Claude 4.5 to consider the differences across paper versions, how they related to the Unjournal evaluator suggestions, and whether this was likely to have been causal.

See Claude's report here  coming from the prompts here. Claude's assessment: Some changes seemed to potentially reflect the evaluators' comments. Several ~major suggestions were not implemented, such as the desire for further statistical metrics and inference.

 

Maybe the evaluators (or Claude) got this wrong, or these changes were not warranted under the circumstances We're all about an open research conversation, and we invite the authors' (and others') responses.

I've found it useful both for posts and for considering research and evaluations of research for Unjournal, with some limitations of course.

- The interface can be a little bit overwhelming as it reports so many different outputs at the same time some overlapping 

+ but I expect it's already pretty usable and I expect this to improve.

+  it's an agent-based approach so as LLM models improve you can swap in the new ones.

I'd love to see some experiments with directly integrating this into the EA forum or LessWrong in some ways, e.g. automatically doing some checks on posts or on drafts or even on comments. Or perhaps an opt-in to that. It could be a step towards systematic ways of improving the dialogue on this forum -- and forums and social media in general, perhaps. This could also provide some good opportunity for human feedback that could improve the model, e.g. people could upvote or downvote the roast my post assessments, etc.