This is still happening in Windows 7 Ultimate (RTM), so it's not the Beta editions causing this.
I'm going to search a bit more before I install Windows 7 Ultimate (Retail) as there is no real difference, supposedly.
However, in the meantime, what I've done is create a batch file on my desktop that kills the explorer.exe task and then restarts it. If you would like to do the same, here is how I did it.
1. Open Notepad and type in these two lines.
taskkill /F /IM explorer.exe
2. Save the file as whatever.bat.
That's it. Works for me and is a lot quicker than manually closing and reopening explorer.
If you're not sure how to save a file as bat, in Notepad, Click File>Save As. Change the "Save as type:" to "All Files (*.*)", then name your file what you would like adding .bat to the name. Click Save and you're done.
Hope this helps someone.