The way it does math is mostly as people have already assumed - approximating instead of doing it “the normal way”. It’s 2025 and at this point absolutely nobody should be surprised that AI “confidently describe[s] the standard grade-school method, concealing its actual, bizarre reasoning process”.

As for poetry,

Here, the model settled on the word “rabbit” as the word to rhyme with while it was processing “grab it.” Then, it appeared to construct the next line with that ending already decided, eventually spitting out the line “His hunger was like a starving rabbit.”

this is exactly how many poets write rhymed poetry too, it’s not even remotely bizarre.

Still, it is interesting and good to see some concrete advancement in the study of AI reasoning. Hopefully it will contribute towards reducing the mystification of the whole thing.