Memory usage goes up over time

This has started doing this a few months ago

I have 8 GB ram and when I first start my computer, only 20% of memory is used but if I keep my computer on for several days, that memory usage goes up 50%, 60%, 70%, or even 80% when I do not have any programs running except the system processes and stuff my computer needs like AV or TeamViewer. Closing all programs doesn't fix the problem, it is as if sometimes closing programs does not free all the memory and it is lost—stuck being used indefinitely until I restart my computer. It is getting rather annoying to have to constantly restart my computer just to free up memory when I used to be able to keep my computer on for a couple of months and only be around 30% (not 20%) when no programs are running. I tried cleanmem from the Tweaking.com repair but that only drops it down to maybe 60% to 70%. It hardly fixes anything with the memory.

Does anybody know what the problem could be?

Post a screenshot of Task manager - Performance tab, when the problem occurs.
This is not normal. It would normally take a heavy workload to push a 8 GB system to 80% usage. Daily restarts are not necessary. I often go several weeks between reboots without problem.