When you click on the Internet Explorer "E" taskbar icon, it actually doesn't close the IE window, but minimizes instead.
To open a new window, right click on the "E" icon to see the jump list, and click on Internet Explorer
to open a new window.
You could also open a new tab, then drag the tab from the toolbar to have it in a new window as well.
Hope this helps,