Since you have an old video card, which worked I assume, you need to
plug that back in and see if it "BSODs" still. If it doesn't then you need to
uninstall the drivers for your old card, and when you go to restart for that change to take effect, do a shutdown and install the new card.
If you still have problems, then maybe the card is dead, if you are out the money with no return option, try
baking it in your kitchen oven 
(yes I'm serious go google it, you'll need to get the plastic stuff off and read up).
If normal starts fail, but gets into safemode, then maybe you can do a firmware upgrade to the card or a MB bios firmware update which may alleviate the symptoms (though in my experience I have never needed to do for display, but people talk up the firmware upgrade so Im just putting it out there).
When you go into BIOS, does it display ok? Press either F2, F10, Shift+delete, or whatever it says on boot splash screen to get into SETUP.
Speaking of BIOS, you'd be amazed what
flashing your CMOS can do for the occasional system instability. Find how your motherboard wants to be flashed, some differ beyond just the jumper.
There are some PCIe settings that can be tampered with in bios, which I stay away from, but may be involved with your situation.
Can you get the code if it is a BSOD, that can help. (00x00000002B looking kind of code)
If its not a real BSOD, then maybe it is some kind of BIOS window that is flashing. This would make me think
maybe the PSU could be the culprit rather than the card.
Bottom line, test the old card with the new PSU and see if there is the issue so the video card can be isolated as the cause. Is all your RAM seated, is the card seated, is the CPU fan plugged in and spinning? Does the
Amp rating on your PSU's 12volt rails provide what the video card needs? Even though you may have shelled out some cash, go check. Are you overclocking the CPU, maybe you need to try factory default stuff to rule out extraneous variables.
Ubuntu not working makes me think it is hardware and not
driver software, though since it is booting from HDD partion maybe it is still
drivers.
How does a live CD/DVD linux distro perform?
Who made your new Graphics card? I always buy EVGA, PNY, ASUS, or GALAXY in that order of preferrance.
Was your old graphics card for a PCI slot, and now you are using the PCIe slot? Maybe the PCIe slot on your MB was dead and you are just finding out
This is plenty to work with, the ball is back in your court