Randomly lose internet connection

Right, this would be my first time using this forum, but as I'm really getting tired of this problem I thought I might as well ask to see if anyone knows anything.

So, the deal is that I just randomly lose internet connection and I can only reconnect by disconnecting my network cable and plugging it in again. Thing is, while I'm not connected to the internet when this happens I still have connection to the router.

I've thought that it could either be my network cable of possibly the network card, but as I never have internet problems when I'm on a Lan at friends that shouldn't be the case.
And so I eventally concluded it should be the router that's the problem, even though no other computer on the network suffers this problem, so recently changed the router from a D-link DI-604 to a D-link Dir-655, to no avail.

And so, having gotten a bit tired of having this problem for around a year now, I'm hope you might have some insight to what the problem might be.
Thank you in advance.

Maybe phoning customer support/the help desk of your internet service provider might help - what you are experiencing could be a port problem, in which case they'd be able to hook you up to another functional port. They should also be able to see the interruptions/disconnections you've been having.

Yes, my computer is directly wired to the router, and as far as I know it is securely connected as I've tried with two cables, the one I use for lan and the one I use at home and both drop connection, but I don't when I'm somewhere else.

And about my direct connection to the modem, I haven't gotten many chances to try it as the rest of the family wants to use the internet as well, but when I have tried to make direct connection to the modem I haven't been able to access internet at all.