“Seems” being the operating word here. But children think that Muopets are conscious. The most famous work of fantasy fiction is about a malevolent piece of jewelry. Humans are very good at attributing consciousness to non-conscious entities. We are easily fooled in this respect.

Even if some putative AI may be conscious, an LLM is just something that looks up words in a database with probability weights attached. This technology cannot lead to consciousness.