We have (at least) two fundamentally different types of knowledge—there’s our intuitive world model that does improve with repetition learning (much like a neural network) and does change with practice; and there’s rule-based knowledge that improves by eliminating possible rules/theories via observation.

My interpretation of Einstein’s quip is that insanity consists in confusing the two—thinking that rule-based knowledge can improve by performing the same tests over and over until the results match our theories, instead of modifying our theories to reflect the results.