Keep a log. Every hour test the speed. Let's see if we can narrow down when the connection goes bad.
My former boss had a lamp with a build in transformer that would knock out his DSL thanks to some sort of RF interference. Once he moved the lamp or shut it off - all was well in the world.
So, if you're the only one who uses the machine just keep a notepad next to the mouse. If your whole family uses it, set the speed test site to the default page and make them record the time.
The key to solving problems is gathering quantifiable data.
Think about it, if the speed starts dropping off at 5:30pm and eventually becomes unusable by 8pm - what is happening? People are turning their lights on, doing wash, downloading a bootleg copy of Professor Schlong & the Sorority Sisters.
and give your neighbor a call - see if they're having the same issue.