What we can do to resolve this problem is delete the file that caches the pinned items. You can see those files for yourself by opening up an Explorer window and pasting in the following path:
%APPDATA%\Microsoft\Windows\Recent\AutomaticDestinations
Unfortunately, all of these files are not human-readable, though you can kinda/sorta figure out what they are assigned to by opening them up in Notepad. The key thing to know is that there seems to be one of these per application.
