KyleM's Quick Takes

Summary: Thinking out loud about the J space paper’s implications on future animal welfare research (if there are any). I don’t know much about LLMs or brains or animals but I’d love to chat about this stuff with anyone at my same level of smartness, or learn from folks who know things.

It would be good to have some people thinking about the J-space paper and what, if anything, it has to do with animal welfare. A popular question about animal brains is “what’s going on in there?”. If we get some vague notions about the conditions and size ranges where neural nets act like global workspaces, it might give us some order of magnitude estimates and fuzzy intuitions about what sizes and types of animal brains exhibit those properties.

Some questions that seem interesting:

1. What model sizes and training regimes produce a recognizable J-space which is necessary for solving some tasks, measured by ablation?
2. Are there any phase changes in the structure or importance of the workspace as network and training scale increase? How about as the number of input streams (text, vision, sound tokens) increases? Perhaps formation of these mechanisms is visible as a critical point in a double-descent type capability curve.

3. Maybe effective utilization of J-space requires slack in pretraining in addition to scale. Perhaps you get room to develop this stuff from excess compute when you’ve hit diminishing returns from hardcoding more explicit methods for the tasks you handle.

   or (more likely?) the opposite is true - the need to address a broad task range with limited compute forces many workstreams to share computational resources, resulting in abstraction, resulting in segmentation of the abstract stuff from the concrete stuff. Which resource availability helps you grow a good workspace? Lots of free parameters, or not enough? First one then the other?
   
   What model organisms are the most “animal-like” if we want to vary parameters and look at their effect on the usefulness and recognizability of access consciousness? None are great analogues, but what’s the closest we can get?

   Do the ablation experiments in the J-space paper map onto lesioning experiments in different parts of animal brains?

4. Broadly, animals have similar tasks and data sources to other animals (processing stimuli into models of the world, making decisions on how to secure resources, reproduce, and avoid threats, etc.). We all live in the same environment, the physical world. LLMs don’t always share this common substrate, and their task space and input data is much weirder. Their food is mostly pre-chewed. My assumption is that a global workspace is more likely in systems which must reason on multiple input streams about the same thing (I.e. animals). Gotta combine that info somehow. So I’d assume animals are more dependent on workspace-like features than LLMs. Are multimodal LLMs more likely to depend on J space than single-input-type LLMs, for that reason? It may be worth checking.
5. Given how hard it is to tell what’s going on in animal brains, getting some fuzzy analogies between maybe-conscious LLMs and maybe-conscious animals seems high value. Maybe they will get less fuzzy over time.
6. Can we find an analogue of the J space in the recent fly brain simulation of questionable quality? What would that look like?
7. Does any of this have anything to do with welfare? How much do we care if something has these features? My gut says that if we’re going to figure out what it’s like to be a bug, this type of AI-to-animal analogy is probably how it will happen.