To me, guiding by your words, it seems to be pretty much a joke, just a brother saying those things but with no real idea of how to do that or if that is even possible. If you haven't noticed anything other than his words its likely to be the case (brothers can be irritating at times

)
But let's back to computing for a moment. Out of box, I can think of a serious Windows vulnerability that's enabled by default in all installs, and can be used to access the whole drive without your knowledge or consent, the
administrative shares. In short, by default Windows automatically shares the
whole hard disk right from install, and anyone with an administrator password can access them remotely without any indication:
Disabling the Hidden Administrative Shares | ITworld
A second problem is if he, somehow, managed to create a second user account for his own use, bypassing your password changes. That way he can simply login with that account and if he gave himself admin permissions, access everything. Check what user accounts are in your system just for safety, and disable anything that you don't explicitly use for yourself.
Are by any chance the computer
physically accessible to him? I guess that by him being your brother, but correct me if I'm wrong. It's theoretically possible for him to boot your machine with another OS (in CD or pendrive for example) and that would give him full access to everything, no matter how much security you've placed on Windows. He needs to be a little more tech-savvy for this to work, but it's totally possible, as they say, physical access means "game over, the attacker won". The only way to stop him is by making sure he can't touch your computer.
Even after writing all this, I still consider a joke the most probable outcome

, but hope the tips help you anyway.