If I had to spend money on this computer I would start with a quality power supply.
Even if it doesn't solve the problem, in my opinion a quality power supply is needed for your system.

Your power supply prices out at about $30.00. I know of no quality power supply that cost just $30.00.