This article would be worth considering by EAs using animal cognition studies to assess their sentience; it suggests a lot of the literature is biased towards overly positive conclusions about animal cognition.
A prominent vein of comparative cognition research asks which cognitive abilities may be ascribed to different species. Here, we argue that the current structure of comparative cognition makes it near impossible to evaluate the accuracy of many of the claims produced by the field’s empirical research. ... This argument is based on the following observations:
1) Phenomenon-based comparative cognition uses confirmatory research methods that are directionally biased
2) In combination with a publication bias and a likely high rate of false discoveries, this bias suggests our literature contains many false positive findings
3) This directional bias persists even with strong methodological criticism, and when researchers explicitly consider alternative explanations for the phenomena studied
4) No formal method exists for generating and assessing theory-disconfirming evidence that could counter the biased positive evidence
5) Ambiguity in definitions allow us as researchers to flexibly adjust our substantive claims depending on whether we are refuting criticism or selling the results
6) The small size of comparative cognition as a research field perpetuates and reinforces points 1 to 5.
My own research training is in neuroethology, where Krogh's Principle is commonly used as a guide:
for such a large number of problems there will be some animal of choice, or a few such animals, on which it can be most conveniently studied.
While these days the principle is helpful for encouraging people to work on animals other than model organisms, it probably reinforces a lot of the biases above (particularly #1). For instance, desert ants have been studied as a model of visual navigation for many years, but were originally chosen for this because they lived in an environment that was hard to navigate and so initial studies mostly confirmed researchers opinions of their exceptional navigation abilities. If cognition is usually tested in exceptional cases then it probably also reduces the generality of findings between species - most ant species don't live in the Saharan Desert and will have had less selection pressure to evolve exceptional navigation abilities.
I'm not sure how important Krogh's Principle is in animal cognition research of the kind we're interested in; my impression is that the animals that are studied are primarily animals that are well-studied, like fruit flies, bees, mice, rats, cats, dogs, farmed animals and the stereotypically smart ones (corvids, parrots, elephants, cetaceans, primates), and the animals EAs are interested in fall into these groups. When I want to know about chicken cognition, I just look for studies on chickens. It's worth mentioning that Rethink Priorities stuck to relatively narrow taxons in their report.
I do agree that this research is likely to be biased overall to produce more positive than could be reproduced or generalized. However, I also think that the priors are already very skeptical (e.g. Morgan's canon over Occam's razor and despite common descent) so scientists are also likely to attribute fewer and less complex mental states to animals than I think best explains the evidence, and it's pretty clear that we've systematically underestimated their capacities, so it's likely the current state of research underestimates them overall, too.
Or, rather, researchers aren't using Bayesian reasoning in the first place, so they aren't really using priors at all in interpreting evidence; I think Morgan's canon is more like a p-value threshold than a prior.
Of course, we can just use our own priors in interpreting the evidence, and in doing so, we should take into account biases towards positive results in research.