I still remember Geoff Hinton’s criticisms of backpropagation.

IMO it is still remarkable what NNs managed to achieve: some form of emergent intelligence.