Eh, I experienced this.

What’s your upload speed, becuase this will be your max download speed as your sending data out of your network!

Try for an ISP with symmetric up/download, or just one that’s better.

You can test this with a vps with a higher upload than your current ISP, destroy it when your done. (Jupiter Broadcasting and Tux Digital podcasts have a good deal on linode atm)