New
#11
The vibration could be a hard drive or fan slightly loose in its mount, a fan going bad, or just flimsy case design that allows bezels or side panels to vibrate a bit.
The noise other than the vibration is likely a fan or power supply. You can temporarily stop any case fan with your fingers to see if the noise stops.
Fans spinning above 1000 rpm tend to be audible at close range, even the better ones.
Get the side panel off, get a flashlight, and put your ear in there and pay attention. Some fans are a lot quieter than others and even good fans will eventually fail and make a racket.
Quieting noisy parts is very difficult. The best move is to buy quiet components to begin with, so you don't have to worry much about making them quiet. Some cases have mesh front panels or are otherwise so compromised that it's very difficult to have a quiet system with them, regardless of the internal components.