Might be a bit of a hassle but in lieu of anyone else posting I can suggest this, the long way.
In the folder press Alt to call up the File bar and then go to Tools (or press T to instantly expand the menu). At the bottom is the option Folder Options, click this to call up the, well Folder Options window. In this window head to the 'View' tab and in the Advance Settings box you will find "Show hidden folders, files and drives."
Once you check the box for this and apply the change, you should see a new file called "Thumbs.db". Delete that and then wait for Windows to recompile the thumbnail database!
If that's not what you meant then I do apologise if I've listed something you already knew how to do.