As a rational actor with no useful information

mood

3. SEQUENCE — Sequences of actions can be assessed for rationality independently of their parts: each of "reject A" and "reject B" can be individually permissible while the sequence "reject-A-then-reject-B" is impermissible. Elga turns the same Sally argument on it. SEQUENCE makes rejecting Bet B fine when no Bet A preceded it but irrational when it would complete the bad sequence—so it, too, imposes different requirements across two situations Sally can see are identical in everything she cares about. Hence SEQUENCE fails.

I don't think this is a strong argument. There are other cases where you should make commitments that you would later be inclined to break, like Parfit's hitchhiker, and St. Petersburg lotteries with unbounded utility functions. The latter is an argument that unbounded utility functions are irrational, based on similar logic.

Furthermore, "imposes different requirements across two situations Sally can see are identical in everything she cares about". What if I do care about the differences? Or, is this any worse than picking numbers to ensure precision for no better reason than that they occured to you? Because that's what it takes to produce arbitrarily precise probabilities if you fix what information is available to you in realistic settings.

Also, here's another way someone with unsharp probabilities might handle this situation. In summary, I should accept bet A at the start to rule out the possibility of picking a dominated sequence:

1. If I accept bet A at the start, then the probability that I pick the dominated sequence (rejecting both) is 0.
2. If I reject bet A at the start and if I can't guarantee that I will accept bet B next, then there's some chance that I pick the dominated sequence.

If I compare 1 and 2 statewise, then 1 > 2 with some probability, and 1 and 2 are incomparable otherwise. In other words, either 1 beats 2, or I have no decisive reasons favouring either and I can ignore those cases. So I decide on the cases where 1 beats 2 and accept bet A at the start.

What would you recommend to handle cluelessness and unawareness about the far future and acausal influence, where contact and feedback may not be very practical or sufficiently informative? Maybe with the far future, we should just be extremely patient, possibly over 1000 years or more, and wait for that feedback anyway, but during that time, lots of confounders and clueless-making events could come up.

Or we simulate things in a lot of detail to try to get that feedback artificially, although that may mean simulating and realizing a lot of suffering.

(1) not "impartial" in the sense that some moral patients/consequences are bracketed out for not very well-motivated reasons, or (2) not action-guiding.

If you think it's fair to map the "same" moral patient across worlds to themselves, assuming at least some transworld identity, then I'd guess we could work out counterpart relations extending transworld identity to cover contingent moral patients, maybe along the lines of Thomas, 2019, pp.30-31 where, roughly, we map each contingent moral patient under one choice to the statistical average of the contingent moral patients — across all of spacetime and the whole multiverse if we're in one — under the other choice.^[1]

At first pass, if we're clueless about the total welfare effects for any action or comparing any two actions, I'd guess we'd be clueless about the average contingent moral patient, and, at least on bottom-up bracketing, just bracket them out and ignore them. This would mean the theory acts like the strict person-affecting view, where only currently existing (or at least "necessary") moral patients count, and we ignore (at least) the rest. That might sound bad, but we ignore them because we're consistently clueless about whether their counterparts are better or worse off in expectation, which seems like a good principled reason to do that.

This could be practically action-guiding. You'd probably focus on near-term human welfare, because currently existing non-human animals that you could "affect" cost-effectively don't typically live long enough for your actions today to help them (e.g. most live <2 years before slaughter, often <1 year), and you're clueless about the welfare of the counterparts of the future/contingent animals you could "affect" cost-effectively.

But maybe speculative scenarios, including how TAI might play out, also make you clueless about how to benefit currently existing humans if, before bracketing, you're aggregating and taking expectations over their entire futures or their current preferences pointing at things at least a few years away, and can't bracket out the speculative scenarios. I imagine you could get around this by using transworld identity at each point of time^[2] and do bracketing before aggregating over their futures, so you can bracket out the periods of time you're clueless about.

1. ^{^}

   Maybe with some extra accounting for the number of moral patients to do. And this could get trickier with infinities.

2. ^{^}

   Could need to account for relativity theory in some way.

Would you say bracketing isn't fully impartial? It doesn’t seem less impartial than Scanlon's Greater Burden Principle (or Tom Regan's harm principle), which roughly says to choose between two actions, you should prioritize the individual(s) with the strongest claim or who has the most to lose counterfactually between options (not necessarily the worst off, to contrast with Rawls' Difference Principle/maximin/leximin). 

I guess methods of bracketing where admissible bracketings must group certain individuals, e.g. spatiotemporally or by causal paths affecting them, are more partial. But if you go with the unconstrained version, it seems basically impartial?

It is sensitive to differences between worlds beyond the number of individuals at each welfare level, because it has to identify each individual across worlds to take individual differences.

This also applies to many comments written by people? In general, whenever something is shared, one could invest more time to make it more accurate. However, to me it seems hard to come up with general rules about how much time to invest vetting claims.

If someone had a pattern of fabrication and very poor understanding (and apparent confidence) like LLMs often do if used uncritically, I would be annoyed with them and possibly do any of the following:

1. Tell them to read and review more carefully, look for opposing arguments, etc..
2. Downvote such comments (and I very very rarely downvote).
3. Stop engaging with this person, because it wastes my time and may encourage them to waste others' time.

I think sharing long texts produced by LLMs is often fine even if they were not fully read by the person prompting, basically for the same reasons that it is fine to share long text produced by people even if the person sharing them did not fully read them.

I would request you check it yourself or at least run LLM critique passes, because of high rates of hallucination and other errors by LLMs.

Similarly, if there was an author who was just as consistently bad as LLMs are, and you shared their work uncritically like this, I'd recommend the same.

I have the impression many people, including academics, use "consciousness" and "phenomenal consciousness" interchangeably, and without wanting to take a stance on illusionism or realism.

My impression is that many people, including academics, use "consciousness" and "phenomenal consciousness" interchangeably, but they do so implicitly rejecting strong illusionism, would reject strong illusionism if asked directly, and typically don't understand strong illusionism. Maybe many are open, though, I'm not sure.

Re AI, it's a special case of Brandolini's law:

The amount of energy needed to refute bullshit is an order of magnitude bigger than that needed to produce it.

LLMs make it easy to produce a lot of plausible-looking but wrong or misleading claims quickly, and sharing them without checking yourself (or at least running another LLM to do that) puts the burden on others to correct.

I think you should make sure the claims are checked before sharing.

You could do this yourself, or run another critical LLM pass on the answer, e.g. a prompt like "Exhaustively verify and ruthlessly critique the above answer."

The critique pass itself may turn up false positive critiques, and you might want to run a third pass to verify the critiques and write a new balanced answer.

(I agree that it's also important to tag that it’s LLM output, and at least read it over yourself, which you did.)

However, illusionists acknowledge that phenomenal consciousness exists. They just argue it is not real in the sense it does not have the classic qualia properties. I seem to agree with this.

This is true of weak illusionists, but not strong illusionists. Strong illusionists reject phenomenal consciousness. Specifically they reject phenomenal properties; they accept that consciousness exists.

Would you do anything differently if you endorsed weak illusionism/realism as defined in Frankish (2016) instead of (strong) illusionism?

Other than argue about it, probably not much, assuming functionalism and materialism/physicalism of some kind that's compatible with artificial sentience.

FWIW, my impression is that there are ways to interpret ‘what-it’s-like’ or 'subjective' consistently with strong illusionism, as, e.g. the specific patterns of discriminations a brain makes or what information is accessible to it (or its attention or its global workspace). I think Frankish himself has done this later. 

But 'phenomenal' and 'qualitative' seem harder to pin down precisely in purely physical, discriminatory and informational terms in a way that I think would be broadly acceptable after critique and reflection to realists (including weak illusionists).

First, before responding to the specific claims in the Gemini output:

1. The Gemini output seems like AI slop, and I would recommend against passing along LLM outputs uncritically like this. The answer seems quite bad, most of what it said seems wrong or misguided. You could run another pass of an LLM on the answer to critique it to help a bit. I hope this doesn't accurately represent Lau's views. Illusionism is often misunderstood and strawmanned, including by academics, so there's a lot of garbage out there for LLMs to have been trained on. Some of the problem with misunderstanding and strawmanning is probably due to its unfortunate name. (I'll flag what looks bad below.)
2. I don't see a plausible path for Lau's view to solve the hard problem or meta problem of consciousness. Does Lau explain why people so often regard consciousness as non-physical, ethereal, having classic qualia properties, or some qualitative/subjective character beyond just the physical? Or does he reject that people often do regard consciousness this way?
   1. Specifically, I don't see how reality monitoring helps. Internally generated mental imagery is still experienced. It's just experienced differently.
   2. I'm not familiar with Lau's theory; there's problably more. But I've looked at so many theories, and my prior is to be extremely skeptical.

1. Macro-Level Summaries are Not "Illusions"

Is this section trying to say the non-physical "magic" is real? If not, how does it contradict illusionism?

Lau counters this with a functional analogy: macroscopic metrics are real, even if they ignore microscopic details. When a digital thermometer gives you a reading of 20°C, it is not giving you a detailed map of the kinetic energy of billions of individual air molecules. It is giving you a simplified, macroscopic summary. But that doesn't mean "temperature" is an illusion or a false belief.

This looks a lot like what Graziano says about his own illusionist theory here, so doesn't seem like a point against illusionism:

These models are not empty illusions: they are caricatures. They represent something physically real, but they are not accurate. Models never are. The brain’s models are useful, adaptive, simplified, and never fully accurate, yet they form the basis of our beliefs, thoughts, and claims.

That being said, I'd say the appearance of phenomenal properties is an illusion, and possibly an active misrepresentation, not merely a simplification. See Kammerer, 2019.

Similarly, Lau's Perceptual Reality Monitoring (PRM) argues that the prefrontal cortex generates a macroscopic summary of sensory reliability. It evaluates a signal and outputs a tag: This is real and present. Just because that tag lacks the metadata of neurotransmitter release doesn't make the resulting subjective experience an illusion. It is a highly accurate, biologically real computational output.

I don't see how this solves the hard problem or meta problem.

2. The Fallacy of the "Perfect" Introspection

Illusionism relies on the premise that for consciousness to be real, it must perfectly match our intuitive, pre-scientific introspective beliefs about it (i.e., that it is magical, ethereal, or non-physical).

This is false. Illusionsim does not rely on such a premise.

Lau points out that this is an unfair standard. Science routinely redefines phenomena without eliminating them. When we discovered that water is H₂O rather than an indivisible classical element, we didn't declare that water was an illusion. Lau argues we should treat consciousness the same way: it isn't the magical inner theater we intuitively thought it was, but rather a physical, metacognitive reality-monitoring system. Redefining it doesn't erase its existence.

Illusionism is (re)defining consciousness without eliminating it. What it eliminates is the "phenomenal" properties of consciousness, and classic qualia are roughly the only formalization/precisification of phenomenality so far that doesn't end up strong illusionist. See Frankish's Quining Diet Qualia. "What it feels like", "subjective", etc. are not formal/precise and don't rule out strong illusionism.

And again, I don't see how reality monitoring helps. 

3. The Evolutionary Reality of "Presence"

Seems independent of illusionism vs realism. Mental imagery is still experienced, just differently. Is Lau (or just Gemini?) saying the feeling of presence is a phenomenal property, and mental imagery lacks all phenomenal properties? Would you agree that dreams, thoughts and visualizations aren't phenomenally conscious experiences, but your externally generated experiences are phenomenally conscious? I don't think this is compatible with what most would intend by "phenomenal consciousness", and if this is what Lau (or Gemini?) wants to do, he's working on a totally different problem from pretty much everyone else.

Summary

Lau would say that strong illusionists correctly identify that the brain uses simplified representations, but they make a fatal philosophical error in concluding that simplified representations are false.

Seems like a strawman. Strong illusionists don't conclude that simplified representations are false. They conclude that there are no phenomenal properties, and any representations as if there are phenomenal properties are therefore misleading (illusions).

Yes, we could just remove it if we thought it was net negative overall.

Cage-free could turn out to be one of the best things we can do to reduce disabling pain, but slightly bad for excruciating pain, so that it's just unclear whether it's net good or bad under wide uncertainty about pain intensity tradeoffs. If it looks very good on views where disabling-excruciating pain tradeoffs are more modest and only somewhat bad and in the end outweighed on views with much more weight to excruciating pain, removing it from the portfolio could be a mistake.

See also my related post Hedging against deep and moral uncertainty.

Suppose cage-free increases excruciating pain compared to caged.

Is the total increase in excruciating pain across all (the sum total of) cage-free work from our community smaller than the total reduction in excruciating pain from all of our other work, e.g. CO2 stunning for chickens, other humane slaughter work, other welfare work, across species?

If yes, and if there are no other increases in excruciating pain (or they're small enough), then we could still say our community is preventing more excruciating pain than it's causing, overall. Our portfolio could still be robustly positive across different views on pain intensity tradeoffs. I think I'd be pretty satisfied with that, even if it is causing some excruciating pain (that is outweighed by reductions).

(Alternatively, we could try to compensate the specific animals we expect to cause more excruciating pain to, but that seems much harder on worldviews according to which excruciating pain matters way more than disabling pain.)