Geforce GTX 550 ti Random FPS Drops

Ok so, i upgraded my graphics card from a Geforce 9500 GT to a Geforce GTX 550 ti. I can be on my computer for several hours (4-8) with no problem at all
then all of a sudden i get a major fps drop making games unplayable and streaming media such as Netflix lag, while also making my browser to lag big time while watching videos.

I haven't had this issue with any other streaming media such as youtube
It almost feels like the problem occurs while watching Netflix then sometimes ill open a new tab to check facebook or multitask and then notice a major lag spike.

I got into habbit of noticing this, closing netflix opening a game like Dead space 2 and seeing how ridiculous my fps has dropped.

A simple restart will solve this (temporarily) which then several hours later the problem comes back.

All my drivers are up to date, my temps seem to be fine.
Heres my specs.

Motherboard - MSI G31TM-P21 LGA 775 Intel G31 Micro ATX Intel
Hard drive - Seagate Barracuda ST500DM002 500GB 7200 RPM 16MB Cache SATA 6.0Gb/s 3.5"
GPU - Geforce GTX 550 ti
CPU - Pentium(R) Dual-Core CPU E6700 @ 3.20GHz
PSU - APEVIA WIN-500XSPX 500W ATX12V / EPS12V Power Supply
RAM - 4 Gigs

Would just like to point out that IMO you have a really small amount of RAM.
AND you shouldn't be using your computers for several hours straight.
Kind of like a car. Are you going to drive a car from Beijing to Paris without stopping? Hopefully not.