DSL or cable is generally not enough bandwith to support more than a few connections. The limiting factor is usually your upload bandwith. Go to
www.dslreports.com and do a speed check. Another way to check, is to play on a lan connection also. If your client on the lan connection is lagging then you server is the bottleneck.
Larger databases use more memory, thus you whole computer could run slower.
And the switch shouldn't have any noticable effect on performance