The easiest way to do that is just to select multiple photos from inside of windows, right click them and choose print.
Assuming your printer was made in the last 4 or 5 years it should kick up the wizard to let you arrange them how you want.
I think using 3x5 will generally allow 4 per page.