I would consider LessWrong one of the 'specialized venues', given most of its current activity is 'about AI'. So insofar as many(/most?) 'senior EAs' consider their cause prior settled to 'AI for the forseeable future, and I'm confident other things are less important - and even more confident they are less urgent', migration of their activity from an 'EA forum' to 'AI forum(s)' follows fairly naturally.
I basically agree with the observation, but I'm less sure about both diagnosis and management. I would primarily attribute forum decline to structural developments (vs. say, the 'golden age' ~10 years ago):
1. Specialisation: If the last decade or two of work has 'made progress' on EA topics, we might expect it to get progressively harder for newcomers to usefully increment the state of the art. So you might also expect something like the specialization we see in science: it takes years for someone to get up to speed on astronomy, astronomers mostly discuss their work with fellow astronomers, a little with related disciplines (e.g. sometimes physicists, very rarely oncologists), and their channel to the general public is education/comms/service rather than 'invite all comers to our conferences'.
The price of this progress is there are fewer valuable discussions to have on a general forum. Specialized venues are more attractive to subject-matter experts as they select for a higher quality audience than a '101-space'. Perennial 'cause prio questions' have been tilled over extensively (and most old hands have resiliently made up their minds), so the prospects of the n+1th discussion of (say) the poor meat eater problem finding the consideration which settles the matter looks increasingly remote.
So the 'evergreen' topics get tightly circumscribed to: 1) General-ish topics which are still niche enough not to inspire a funder to employ a small army of analysts to attack (perhaps my own 'epistemic self-help' oeuvre). 2) Community drama/gossip: no barrier to entry to pile in, and at least the particular gory details are novel. 3) 'Community engagement' through press-release/announcement/ad.
2. Competing options for contributor time: Harking back to my glory days, my archetype then was shared by many other 'EA forum grandees': 20-something, early-career job outside EA, writing here as an unpaid hobby. That archetype has largely evaporated. These people are now 30-something (and the incipient-middle-age dividends of spouse/kids/ailing parents seldom grant oodles more 'free time'), are now mid-career+ (ditto), and - perhaps most importantly - are often 'doing something EA(/adjacent)' as their job.
The latter competes with forum activity: their day job both pays much more (i.e. at all) and is better matched to whatever their strengths may be than labouring to contribute on an online forum. Even those who are especially well-suited to online writing have much easier opportunities to 'go pro' or try and monetize themselves rather than provide their surplus for free than back in times of yore.
--
If the above is most of the story, then 'intellectual culture'-style tweaks can only really help around the edges. In essence, the forum thrived at a time when amateurism was the norm, and inefficiency alongside altruism collectively generated a public surplus (e.g. 'investment banker spends their evenings on an early write-up for AI giving opportunities'). Although hopefully altruism has not faded, efficiency has improved, and so energy which was once spent on the forum now gets redirected to more effective ends.
Thanks for your reply, and apologies my OP was unclear. Hopefully I do better here.
1) Your distinction between "there is a shared shape across all link functions" and "which shape across all the link functions" is correct.
Re 'which shape?': if you switch from a logistic curve to clipped linear one (or, I expect, most/all reasonable alternatives) you likely twiddle the exact values and rankings, but the broad order and ~linearly-increasing-over-time sweep of the data should remain much the same. Very loosely, even if the tick-marks on your ruler are all over the place, longer distances measured by multiple rulers laid end to end will still be in the right ballpark.
Re. 'shared shape': But you need to assert all the items/benchmarks have identical shape to resolve a single interval scale across them (very loosely again: that all the rulers laid end to end have the same length). Perhaps one way of understanding this goes back to the 'endogeneity graph':
Handwaving: one way you could move from the middle panel to either of ones flanking it is to assert the y-axis should be re-parameterized: "True capability is really exponential/logarithmic in the ECI capability trait value". Yet (loosely) you could bake the same transformation into the capability trait itself, by stipulating a progressive warp of link functions across benchmarks as they increase in difficulty.
For 'why doubt shared shape?', I think there's a fair case for brute incredulity: why should benchmarks for (e.g.) chess puzzles, science questions, verbal reasoning, (log) time horizons for coding tasks (etc.) all share the same relationship between score progress and general capability? Empirically, browsing some of the 'benchmark score vs. ECI plots' on Epoch's website shows some look more or less S-curvy than others, and the data too noisy and sparse (and likely reliant on tail behaviour) to uniquely identify an logistic curve versus others that look similar.
2) My broad criticisms draw on ~fundamental limitations to IRT or underspecification/ambiguity around 'AI capability' - in either case, I don't think there's a crisp statistical test or methodological fix. Although I think this covers the endogeneity worry in general (it seems hard for the data alone to distinguish methodological distortion from secular trend), trying to pin down particular niggles is handy in its own right, and offers a gesture at the broader complaint: if the 'just read the number straightforwardly' gets the right answer and my metrological quibbling just chases shadows, so much the worse for me - and vice versa.
Your suggestions could be worth doing, although I think the results would only be cruxy in one direction. On my end, I'd guess the speeding up result is robust to link function choice, and would only mildly attenuate if you restrict to pre-model benchmarks (or somehow upweight them relative to post-model benchmarks) - i.e. I wouldn't take positive results as confirmatory proof. Yet if you found (e.g.) speeding up is sensitive to link function choice, that seems much stronger negative evidence.
Some further ideas, although I'm not sure they're worth much:
I understand there's 'quasi-parametric' IRT which lets you twiddle the link functions - and anyway you could play with simulation data to see how shape-sensitive the results are. By and large, the response curves of the benchmarks certainly tend to be S-shaped, so (e.g.) "But if you only tweak the s-curves slightly between benchmarks you can generate huge de/accelerations" vs. "To ablate results like this you need to stipulate hugely warps which we don't actually see in the data" could be informative.
I guess all of this will be confounded by reducing power (see fn. 17), but besides testing on earlier or later subsets, you could try and curate an equal discrimination set of benchmarks across the range, or narrow the set to those which bracket the trend break in 2024.
One could review all the benchmarks, and subjectively score them by how 'reasoning-y/multistep' each is, then somehow reweigh/compare classes to see if it lines up (i.e. do the benchmarks get more reasoning-y over time? Does this correspond to benchmark discrimination? If we somehow 'correct', does the trend break vanish). But even if so (and going back to the problem where external validity is up for grabs), it doesn't seem crazy to assert that "true/important" capabilities do load more heavily on reasoning at you go up the range, so even if the benchmark measurement axis is rotating, this rotation keeps it parallel to the 'true capability' axis.
Thanks Vasco, and you are right. How Elo and related systems 'manage' draws/partial credit is something I didn't explore, but as you say the commonest approach is to set 'draw' at 0.5. The equation above is still true for games which only have wins or losses (so P(A draws) = 0).
I am also sure Farmkind was genuinely trying to do good. But I do think these sorts of schemes warrant calling out with some small degree of ire.
Although my vision may be jaundiced, this case fits a pattern of some folks in animal advocacy being willing to be somewhat-worse-than-spotless in terms of integrity or candour in the hopes of securing some tactical advantage (cf.).
I think heading in the direction of treating non-animal advocates (within or without EA) as "legitimate targets for influence operations" rather than "fellow moral interlocutors" is unwise. Besides being unwise in the immediate "you weren't as clever as you thought you were and it blew up in your face", it also errs in terms of more insidious pollution of commons helpful for folks to coordinate on figuring out what is best to do, and cooperate in doing it.
Thus I hope the lesson learned is, "avoid kind-of manipulative or insincere advocacy", not, "get better at pulling it off". Unfortunately, if it has, it is not readily apparent in the OP, given its reflections focus on first-order consequences and better coordination with other EAAs.
Regardless, the underlying competition in these adversarial advertising campaigns is genuine: Pepsi and Coke (e.g.) want each other's market share for themselves. Here things seem more like two parties A & B collude where B poses as a competitor to A, intending the subsequent playfight between them to be in both A & B's mutual advantage (at least that is what Farmkind intended, notwithstanding they flubbed both the 'collude with A' and 'mutually advantageous playfight' steps of the plan).
I think most countries antitrust regulators (etc.) would raise an eyebrow at this sort of thing. I am sure it is generally regarded as a dirty trick in the marketplace of ideas (cf. I pay a fellow vegan to build up a media presence as an anti-vegan advocate, only to sandbag in a public debate with me where I trounce them, and publicly convert (back) to veganism).
I think the biggest problem with Farmkind's campaign is that it was fundamentally manipulative and insincere.
As I read it, the grand plan here was to pretend to trash Veganuary to 'bait' (your words) a lot of media engagement around this kayfabe controversy, and parlay this into a marquee presentation where the harvested eyeballs discover "we're actually all friends here, let us pitch you on different ways of helping animals!"
The wider world tends to have allergic reactions to 'controlled opposition', 'audience plant', and related stunts, and so should we. I confess the fact that this one backfired in large part because you tricked too many people (including your allies) too well strikes me as a fitting comeuppance.
I'd say my views now are roughly the same now as they were then. Perhaps a bit milder, although I am not sure how much of this is "The podcast was recorded at a time I was especially/?unduly annoyed at particular EA antics which coloured my remarks despite my best efforts (such as they were, and alas remain) at moderation" (the complements in the pre-amble to my rant were sincere; I saw myself as hectoring a minority), vs. "Time and lapses of memory have been a salve for my apoplexy---but if I could manage a full recounting, I would reprise my erstwhile rage".
But at least re. epistemic modesty vs. 'EA/rationalist exceptionalism', what ultimately decisive is overall performance: ~"Actually, we don't need to be all that modest, because when we strike out from "expert consensus" or hallowed authorities, we tend to be proven right". Litigating this is harder still than re. COVID specifically (even if 'EA land' spanked 'credentialed expertise land' re. COVID, its batting average across fields could still be worse, or vice versa),
Yet if I was arguing against my own position, what happened during COVID facially looks like fertile ground to make my case. Perhaps it would collapse on fuller examination, but certainly doesn't seem compelling evidence in favour of my preferred approach on its face.
FWIW, it's unclear to me how persuasive COVID-19 is as a motivating case for epistemic modesty. I can also recall plenty of egregious misses from public health/epi land, and I expect re-reading the podcast transcript would remind me of some of my own.
On the other hand, the bar would be fairly high: I am pretty sure both EA land and rationalist land had edge over the general population re. COVID. Yet the main battle would be over whether they had 'edge generally' over 'consensus/august authorities'.
Adjudicating this seems murky, with many 'pick and choose' factors ('reasonable justification for me, desperate revisionist cope for thee', etc.) if you have a favoured team you want to win. To skim a few:
There are large 'blobs' on both sides, so you can pick favourable/unfavourable outliers for any given 'point' you want to score. E.g. I recall in the early days some rationalists having homebrew 'grocery sterilization procedures', and EA buildings applying copper tape, so PH land wasn't alone in getting transmission wrong at first. My guess is in aggregate EA/rationalist land corrected before the preponderance of public health (so 1-0 to them), but you might have to check the (internet) tape on what stuff got pushed and abandoned when.
Ditto picking and choosing what points to score, how finely to individuate them, or how to aggregate (does the copper tape fad count as a 'point to PH' because they generally didn't get on board as it cancels out all the stuff on handwashing - so 1-1? Or much less as considerably less consequential - 2-1, and surely they were much less wrong about droplets etc., so >3-1?). "Dumbest COVID policies canvassed across the G7 vs. most perceptive metaculus comments" is a blowout, but so too "Worst 'in house' flailing vs. Singapore".
Plenty of epistemic dark matter to excuse misses. Maybe the copper tape thing was just an astute application of precaution under severe uncertainty (or even mechanistically superior/innovative 'hedging' for transmission route vs. PH-land 'Wash your hands!!') Or maybe 'non-zero COVID' was the right call ex-ante, and the success stories which locked down until vaccine deployment were more 'lucky' (re. vaccines arriving soon enough and the early variants of COVID being NPI-suppressible enough before then) than 'good' (fairly pricing the likelihood of this 'out' and correctly assessing it was worth 'playing to').
It is surprisingly hard for me to remember exactly what I 'had in mind' for some of my remarks at-the-time, despite the advantage being able to read exactly what I said, and silent variations (e.g, subjective credence behind 'maybe' or mentioning, rationale) would affect the 'scoring' a lot. I expect I would fare much worse in figuring this out for others, still less 'prevailing view of [group] at [date]'
Plenty of bulveristic stories to dismiss hits as 'you didn't deserve to be right'. "EAs and rationalists generally panicked and got tilted off the face of the planet during COVID, so always advocated for maximal risk aversion - they were sometimes right, but seldom truth-tracking"/ "Authorities tended staid, suppressive of any inconvenient truths, and ineffectual, and got lucky when no action was the best option as well as the default outcome."
Sometimes what was 'proven right' ex post remains controversial.
For better or worse, I still agree with my piece on epistemic modesty, although perhaps I find myself an increasing minority amongst my peers.
I would consider LessWrong one of the 'specialized venues', given most of its current activity is 'about AI'. So insofar as many(/most?) 'senior EAs' consider their cause prior settled to 'AI for the forseeable future, and I'm confident other things are less important - and even more confident they are less urgent', migration of their activity from an 'EA forum' to 'AI forum(s)' follows fairly naturally.
I basically agree with the observation, but I'm less sure about both diagnosis and management. I would primarily attribute forum decline to structural developments (vs. say, the 'golden age' ~10 years ago):
1. Specialisation: If the last decade or two of work has 'made progress' on EA topics, we might expect it to get progressively harder for newcomers to usefully increment the state of the art. So you might also expect something like the specialization we see in science: it takes years for someone to get up to speed on astronomy, astronomers mostly discuss their work with fellow astronomers, a little with related disciplines (e.g. sometimes physicists, very rarely oncologists), and their channel to the general public is education/comms/service rather than 'invite all comers to our conferences'.
The price of this progress is there are fewer valuable discussions to have on a general forum. Specialized venues are more attractive to subject-matter experts as they select for a higher quality audience than a '101-space'. Perennial 'cause prio questions' have been tilled over extensively (and most old hands have resiliently made up their minds), so the prospects of the n+1th discussion of (say) the poor meat eater problem finding the consideration which settles the matter looks increasingly remote.
So the 'evergreen' topics get tightly circumscribed to: 1) General-ish topics which are still niche enough not to inspire a funder to employ a small army of analysts to attack (perhaps my own 'epistemic self-help' oeuvre). 2) Community drama/gossip: no barrier to entry to pile in, and at least the particular gory details are novel. 3) 'Community engagement' through press-release/announcement/ad.
2. Competing options for contributor time: Harking back to my glory days, my archetype then was shared by many other 'EA forum grandees': 20-something, early-career job outside EA, writing here as an unpaid hobby. That archetype has largely evaporated. These people are now 30-something (and the incipient-middle-age dividends of spouse/kids/ailing parents seldom grant oodles more 'free time'), are now mid-career+ (ditto), and - perhaps most importantly - are often 'doing something EA(/adjacent)' as their job.
The latter competes with forum activity: their day job both pays much more (i.e. at all) and is better matched to whatever their strengths may be than labouring to contribute on an online forum. Even those who are especially well-suited to online writing have much easier opportunities to 'go pro' or try and monetize themselves rather than provide their surplus for free than back in times of yore.
--
If the above is most of the story, then 'intellectual culture'-style tweaks can only really help around the edges. In essence, the forum thrived at a time when amateurism was the norm, and inefficiency alongside altruism collectively generated a public surplus (e.g. 'investment banker spends their evenings on an early write-up for AI giving opportunities'). Although hopefully altruism has not faded, efficiency has improved, and so energy which was once spent on the forum now gets redirected to more effective ends.
Hello Charlie,
Thanks for your reply, and apologies my OP was unclear. Hopefully I do better here.
1) Your distinction between "there is a shared shape across all link functions" and "which shape across all the link functions" is correct.
Re 'which shape?': if you switch from a logistic curve to clipped linear one (or, I expect, most/all reasonable alternatives) you likely twiddle the exact values and rankings, but the broad order and ~linearly-increasing-over-time sweep of the data should remain much the same. Very loosely, even if the tick-marks on your ruler are all over the place, longer distances measured by multiple rulers laid end to end will still be in the right ballpark.
Re. 'shared shape': But you need to assert all the items/benchmarks have identical shape to resolve a single interval scale across them (very loosely again: that all the rulers laid end to end have the same length). Perhaps one way of understanding this goes back to the 'endogeneity graph':
Handwaving: one way you could move from the middle panel to either of ones flanking it is to assert the y-axis should be re-parameterized: "True capability is really exponential/logarithmic in the ECI capability trait value". Yet (loosely) you could bake the same transformation into the capability trait itself, by stipulating a progressive warp of link functions across benchmarks as they increase in difficulty.
For 'why doubt shared shape?', I think there's a fair case for brute incredulity: why should benchmarks for (e.g.) chess puzzles, science questions, verbal reasoning, (log) time horizons for coding tasks (etc.) all share the same relationship between score progress and general capability? Empirically, browsing some of the 'benchmark score vs. ECI plots' on Epoch's website shows some look more or less S-curvy than others, and the data too noisy and sparse (and likely reliant on tail behaviour) to uniquely identify an logistic curve versus others that look similar.
2) My broad criticisms draw on ~fundamental limitations to IRT or underspecification/ambiguity around 'AI capability' - in either case, I don't think there's a crisp statistical test or methodological fix. Although I think this covers the endogeneity worry in general (it seems hard for the data alone to distinguish methodological distortion from secular trend), trying to pin down particular niggles is handy in its own right, and offers a gesture at the broader complaint: if the 'just read the number straightforwardly' gets the right answer and my metrological quibbling just chases shadows, so much the worse for me - and vice versa.
Your suggestions could be worth doing, although I think the results would only be cruxy in one direction. On my end, I'd guess the speeding up result is robust to link function choice, and would only mildly attenuate if you restrict to pre-model benchmarks (or somehow upweight them relative to post-model benchmarks) - i.e. I wouldn't take positive results as confirmatory proof. Yet if you found (e.g.) speeding up is sensitive to link function choice, that seems much stronger negative evidence.
Some further ideas, although I'm not sure they're worth much:
Thanks Vasco, and you are right. How Elo and related systems 'manage' draws/partial credit is something I didn't explore, but as you say the commonest approach is to set 'draw' at 0.5. The equation above is still true for games which only have wins or losses (so P(A draws) = 0).
I did cross-post, although perhaps I flubbed it given in languishes in relative obscurity...
I am also sure Farmkind was genuinely trying to do good. But I do think these sorts of schemes warrant calling out with some small degree of ire.
Although my vision may be jaundiced, this case fits a pattern of some folks in animal advocacy being willing to be somewhat-worse-than-spotless in terms of integrity or candour in the hopes of securing some tactical advantage (cf.).
I think heading in the direction of treating non-animal advocates (within or without EA) as "legitimate targets for influence operations" rather than "fellow moral interlocutors" is unwise. Besides being unwise in the immediate "you weren't as clever as you thought you were and it blew up in your face", it also errs in terms of more insidious pollution of commons helpful for folks to coordinate on figuring out what is best to do, and cooperate in doing it.
Thus I hope the lesson learned is, "avoid kind-of manipulative or insincere advocacy", not, "get better at pulling it off". Unfortunately, if it has, it is not readily apparent in the OP, given its reflections focus on first-order consequences and better coordination with other EAAs.
Given Lidl literally sued Biedronka to get bailiffs to seize advertising billboards by Beidronka against them in 2024, and the price war between the chains, I don't see many signs of agreement or kayfabe in this competition.
Regardless, the underlying competition in these adversarial advertising campaigns is genuine: Pepsi and Coke (e.g.) want each other's market share for themselves. Here things seem more like two parties A & B collude where B poses as a competitor to A, intending the subsequent playfight between them to be in both A & B's mutual advantage (at least that is what Farmkind intended, notwithstanding they flubbed both the 'collude with A' and 'mutually advantageous playfight' steps of the plan).
I think most countries antitrust regulators (etc.) would raise an eyebrow at this sort of thing. I am sure it is generally regarded as a dirty trick in the marketplace of ideas (cf. I pay a fellow vegan to build up a media presence as an anti-vegan advocate, only to sandbag in a public debate with me where I trounce them, and publicly convert (back) to veganism).
I think the biggest problem with Farmkind's campaign is that it was fundamentally manipulative and insincere.
As I read it, the grand plan here was to pretend to trash Veganuary to 'bait' (your words) a lot of media engagement around this kayfabe controversy, and parlay this into a marquee presentation where the harvested eyeballs discover "we're actually all friends here, let us pitch you on different ways of helping animals!"
The wider world tends to have allergic reactions to 'controlled opposition', 'audience plant', and related stunts, and so should we. I confess the fact that this one backfired in large part because you tricked too many people (including your allies) too well strikes me as a fitting comeuppance.
I'd say my views now are roughly the same now as they were then. Perhaps a bit milder, although I am not sure how much of this is "The podcast was recorded at a time I was especially/?unduly annoyed at particular EA antics which coloured my remarks despite my best efforts (such as they were, and alas remain) at moderation" (the complements in the pre-amble to my rant were sincere; I saw myself as hectoring a minority), vs. "Time and lapses of memory have been a salve for my apoplexy---but if I could manage a full recounting, I would reprise my erstwhile rage".
But at least re. epistemic modesty vs. 'EA/rationalist exceptionalism', what ultimately decisive is overall performance: ~"Actually, we don't need to be all that modest, because when we strike out from "expert consensus" or hallowed authorities, we tend to be proven right". Litigating this is harder still than re. COVID specifically (even if 'EA land' spanked 'credentialed expertise land' re. COVID, its batting average across fields could still be worse, or vice versa),
Yet if I was arguing against my own position, what happened during COVID facially looks like fertile ground to make my case. Perhaps it would collapse on fuller examination, but certainly doesn't seem compelling evidence in favour of my preferred approach on its face.
FWIW, it's unclear to me how persuasive COVID-19 is as a motivating case for epistemic modesty. I can also recall plenty of egregious misses from public health/epi land, and I expect re-reading the podcast transcript would remind me of some of my own.
On the other hand, the bar would be fairly high: I am pretty sure both EA land and rationalist land had edge over the general population re. COVID. Yet the main battle would be over whether they had 'edge generally' over 'consensus/august authorities'.
Adjudicating this seems murky, with many 'pick and choose' factors ('reasonable justification for me, desperate revisionist cope for thee', etc.) if you have a favoured team you want to win. To skim a few:
For better or worse, I still agree with my piece on epistemic modesty, although perhaps I find myself an increasing minority amongst my peers.