After trying out a few things with IE9 and Firefox 8 and Windows Media Center (current version), I think I understand what is what.
Full screen mode is just that. Every single pixel is used to display content. No line of pixels is available for the task bar, thus there is no place to mouse over the task bar and un-hide it. Both IE9 and Firefox 8 have full screen modes, and so you can't mouse down and un-hide the task bar. You must fiddle with F11, the Windows Key, or minimize and restore the window.
I think applications should be able to make this configurable. It would be nice if there were an option to have full screen mode and full screen mode light, i.e. where the task bar can show a few pixels that can be moused over to un-hide it.
Windows Media Center should do this. When WMC is maximized, it is actually in full screen mode. Again I think it is bad form for WMC to not have full screen mode and maximized mode.
In conclusion, this inability to un-hide the task bar is more feature than bug. In addition there is probably room on Microsofts's part to clarify full screen mode, maximized windows and how each affects the auto hide feature.