That didn't work. As it rebuilt the library it still would do the same crazy thing of putting the same album cover on every album that didn't have artwork.
I found a way to "reset WMP to it's initial settings. To the point where you get the set up screen. This is something I had been trying unsuccesfully to do since this thread:
How to turn back WMP
Very strange what was going on but crossing fingers that this solved it. But if it ever goes bad again I know how to reset it and that should always work.