The exponential problem has always been there. We keep finding tricks and optimizations in hardware and software to get by it but they’re only occasional.

The pruned models keep getting better so now You’re seeing them running on local hardware and cell phones and crap like that.

I don’t think they’re out of tricks yet, but God knows when we’ll see the next advance. And I don’t think there’s anything that’ll take this current path into AGI I think that’s going to be something else.