Blank screen whenever i launch a game.

So I've had this problem for a couple months where as soon as i'd start a game my PC would send an error message (Can tell as the sound pops up) and the screen just goes blank. Usually random colours such as green/pink etc. It works fine outside of gaming but literally as soon as i enter a game of ANY kind it does this and i can still hear the sounds/ people talking to me and i can talk to them. But i can't see anything. I tried plugging the cable into my CPU and it stays blank. I've tried many many things re installations, gone into the BIOS, all this other things and eventually came to the solution it was my GPU as if i use the PC as my CPU it works fine and i can play the games but on low graphics as it isn't amazing. So i got my GPU sent back and they sent me a new one which finally arrived today. So i set it up re-install new drivers and that stuff so i decide to jump into a game and the exact same problem happens AGAIN with a brand new GPU. I've switched the sockets and everything and i can't find a solution. This is really irritating me now and I've spent hours trying to fix it. Someone please help me ASAP.