Hi Wicke and welcome to the Forum
If you don't manually associate a file to a program on your PC then Windows will select the default for that file type. A typical example of this is a plain txt file. Very few users would associate a specific program to open a txt file. Find one on your PC, double click on it and the chances are it will open in Notepad.
So to your problem. You say that when you download a picture file and save it Windows Photo Viewer "Pops Up". When you download the file does it not automatically go into your downloads folder or have you selected a folder to download to? At what stage does does the Photo Viewer pop up? Normally you would not expect WPV to open unless you double click on the file! If you want to change the program that opens your photo file (probably JPG files) just right click on the file click "Open With" and choose a program. Check the box to "Always use this prog" box. Click "Apply"