- - - doesn't this point to a fault in the Windows 7 PC ? Have you tried a mic into the PC sound-card area? - that should show up any Windows 7 PSU/MBO issues.....
The odd thing is that PC's use switching PSU's so any noise should be chopping freq which will be many kHz - so out of audio range.... changing PC PSU seems rather drastic . . . . does OP fancy an upgrade maybe to a "quieter" PSU - might fix, can't hurt, etc.
Can OP provide external power to these interfaces? That might eliminate noise into the PC . . . and presumably you've not wrecked the input connector as that via USB...... there is no "audio" connection oherwise to the MBO...?
-and these interface appear to be noise-free on other PC's - that should be them out of the picture.... as I presume you've used the same USB connectors/wiring, etc...
Finally, has anything changed recently, or Opened the Windows 7 case ?
- & can "Drivers
" cause this?
OP's test with Guitar cable, - I suspect removing the jack puts a short on the input, so it's ignored - maybe passes a "no-signal" msg to the USB control in the PC. Inserting any lead causes the PC to open-up the audio path in the device and in the PC where it picks up any PSU generated noise, etc. By powering the Ext Devices (separate PSU) you should eliminate the suggestion the noise is from the USB-device . .. but that won't eliminate wiring issues, if say the Guitar had lost an earth...etc. although I discount this area since OP say they are OK on other PC's (but I'd separately power them anyway!).