That's generally pretty normal. A PC isn't a pocket radio

Mine has always been like that.
That said, a lot depends on your sound
drivers. Some drivers may auto mute the speakers while others don't... Or there may be a setting in the
driver's control panel that allows you to turn that on or off.