It all depends how EXACTLY you go about to play your music. I didn't realize that at first as I only use WMP for a very short time
Example:
#1: In the library I click on Music and just pick a song from there and the whole library appears in said list, just as you mentioned.
#2: I click on Artist, open one and pick/double click any song from there, ALL songs from that specific artist are added.
#3: I click on Albums open one pick/double click any song and the whole Album gets added
#4: I click on Genre and just as with the Artist, all albums tagged with that genre get added.
Knowing Microsoft for some time now I believe that belongs to the category FEATURE rather than BUG
-DG