Mouse with your off hand means your main hand can do other things - type or such. Most keyboards have a numberpad to the right, so the left is the more ergonomic place for a mouse.

I have two computers, and two keyboards and two mice. I keep a mouse on each side. (I tried a KVM and didn't like it). YMMV.