You can always phone your ISP and get your connection tuned to online gaming needs.. Usually latencies are not the primary concern for the regular web users and download speed are never a problem for gamers, even 1/1mb connection is really fine.
Short spikes/connection issues don't bother when you're just surfing or watching youtube, you may get the dreaded buffering from time to time but generally speaking it works.. Try that in gaming.. One 1 second connection issue can ruin your (and possibly many others) 67 minute race that you've trained for weeks while short pause in youtube video stream doesn't do you so much harm.
If you have friends in the same area, you could try analyze their connection.. But don't use their computers for that, they may be configured differently than your setup.
Also install(
leatrix latency fix )(up to 20% improvement on latency) and if you're computer savvy,(
TCPOptimizer ) can get few ms off too while download speed are not affected at all. There are all kinds of packet queues and double confirmations that are designed for bursts of data that our softwares normally need and we want individual packets transmitting as fast as possible.