Haha I was wondering if you were thinking I screwed into the board or something. I'm a bit OCD like you and I was debating fixing it at first. But after reading up online and getting my friend to look at it - the consensus seemed to be to just leave it and it would be fine. Apparently, most older motherboards didn't even have a GPU latch and the screws into the case were enough to hold it. I am debating getting a a new motherboard and case soon anyways as mine are getting a bit old (motherboard just over 3 years and case is around 7 years). I was hoping to determine my problem before getting new parts though. I'm starting to think that the freezes I've gotten are just in relation to a bug in the game I'm playing - because I played last night again for around 2 hours without a problem.

Just to make sure we are on the same page I added a picture of what I'm talking about since I'm bad at explaining (this is not my computer):

Computer freezing while gaming-gpu.jpg
The area I circled in red is the screws I'm referring to that are holding my GPU in place