The wikipedia article Polydactyly in stem-tetrapods has some explanations on how we ended up with 5 fingers and toes.

The gist of it is that tetrapods (four-limbed vertebrate animals) evolved from a fish similar to a lobe-finned fish that had 5 bones in each of its fins that evolved into fingers and toes. Some tetrapods have subsequently lost digits but the basal state was five.

There’s a book, Your Inner Fish by Neil Shubin that’s full of this kind of stuff. Highly recommend.