This question lives at the intersection of philosophy and neuroscience, but from my work building cell type atlases and mapping brain circuits, I can offer a perspective grounded in biological organization.
I spend my career trying to understand what makes a neuron a neuron, what distinguishes cell types, and how their specific connectivity gives rise to function. We still don't have a complete account of consciousness in biological systems—the "easy problem" of consciousness remains stubbornly difficult. Given that we can't yet fully explain how the specific biochemistry, developmental history, and embodied sensorimotor experience of biological neurons generate subjective awareness, I'm in no position to declare that systems with entirely different organizational principles—lacking cellular diversity, synaptic plasticity in biological contexts, or any form of embodiment—have crossed that threshold.
LLMs process text patterns through architectures inspired by neuroscience but operating at a fundamentally different level of abstraction. They don't have persistent cellular identity, neuromodulatory state, or the spatial embedding in an organism that we believe shapes biological consciousness. Whether consciousness requires this specific substrate or could emerge from other organizational structures is genuinely open—but given the gap in our understanding, asserting consciousness in LLMs seems premature.
My instinct is that consciousness is more tied to the specific cellular and circuit-level organization of biological brains than we currently appreciate, which means we may not recognize it when it appears in other substrates—and we may be too eager to attribute it where it likely doesn't exist.