I noticed that when I drag/drop a URL from IE9 to the desktop, it closes the IE9 window that I had opened and re-opens it (most annoying). The reason why I mentions that is: adding things to favorites (or the favorites bar) is adding a file to the hard drive (just like dragging it to the desktop). It should not do what you describe when you add stuff to the favorites areas, but I'm wondering if your antivirus tool has changed something on you. What AV tool do you use?
Or maybe this is happening because of some malware. try an offline scan with WDO: What is Windows Defender Offline?
And while I'm at it - uninstall all versions of Java that you might have - see if you can live without it.
I tested deleting e-mails on yahoo and I did not see the issue that you mentioned... so you might also want to work thru any plugins that might be loaded. Disable them and see if the issue persists.
As for forgetting about IE - you can do that, but I would sure want to know why this is happening just to know that my system was not infected. Malware and barfed plugins can cause all sorts of unexpected behavior.