Some PC systems have a control panel for the PC audio, such as the RealTek HD Audio Manager, and in that panel is an option to either mute the PC speakers when a headset is plugged in or to not mute the speakers.
I have a RealTek HD Audio Manager that is located in the Control Panel/Hardware and sound. Other audio such as IDT, Connexant, or whatever may have similar panels.
Without more information we are guessing. Please expand on the My System Specs, such as the specific HP model number you have and the sound device, if you know it.