Navigate to C:\Users\<Username>\AppData\Roaming\Microsoft\Internet Explorer\Quick Launch\User Pinned\TaskBar
As a shortcut for the above, you can type the following in the address bar of windows Explorer:
%appdata%\Microsoft\Internet Explorer\Quick Launch\User Pinned\TaskBar
Find the offending shortcut and delete it. Done.
You many need to refresh your taskbar (Toggling 'Lock the taskbar' worked for me) to get the change to appear.
If you don't want to type all that into the address bar in one go, but instead want to navigate step by step, then you'll have to show hidden folders, because the 'User Pinned' folder is hidden. To do this, you need to:
Organize>Folder and search options> View tab>Show hidden files, folders and drives