Upload Speeds Completely Cutting Out After Streaming Music To Others

So I have a problem that I can't find any information for on the net so here I am.

I stream music via SAM Brocaster and a shoutcast server to 30 or so people on a regular basis online. Lately though, after streaming this data (Usually at around 128 or 192 kbp/s), my upload speed will completely disappear to the point where only messages are barely able to be sent. It'll stay this way for anywhere between 5 and 30 minutes before coming back on. The weird thing that I can't get my head around is that I can stream for hours to the server when there is nobody listening in but after people start listening in after about 10 minutes or so. I should mention that even though the upload dies, the download amount stays exactly as it should.

I've had a technician come out who found no real issues with our box but changed some of the power settings for the upload to make it more inline with where it should be because it was a little bit out.

As I say, I'm stumped on this one and have no idea how to fix it and I can't find anything about the issue either.

Any help or tips would be greatly appreciated.

- Thanks

It's Virgin Media, and they throttle them to an extent. We're on a 60MB package which usually gives us around 45MB download and 3MB Upload. When stable, this is more than enough to do what I need to.