Hahah, yeah, maybe I am doing that. that’s why it is a shower thought, not a research paper proposal.

The thought comes from my (kind or recent) study of the algebra/calculus under LLMs (at least the feedforward and backpropagation part of them)

The interesting part is that my ass is non-differentiable at x=0:

Lim x→0⁺ δass/δx ≠ Lim x→0⁻ δass/δx