I think the answer is somewhere in the middle, and depends on your definition of "swimming"

Can a grown adult who can stay calm in stressful situations learn swimming well enough from a book so that he doesn't drown the first time he jumps in a pool. Yes

Can everyone do that, definitely not.

It also depends on the activity, you can probably learn bowling from a book good enough to hit the pins and then get better from there.

You could definitely not learn to fly a helicopter from a book.