Do you have a decent set of computer speakers? You should really have some. They are "stereo analog" and powered, and would be fed from the "lime green" stereo output jack on the back of your computer, which is connected to the onboard audio chip (Realtek HD audio).
I "built" a modest Lenovo-based desktop system for a friend back in December and included
these particular Creative speakers (which fortunately were $30 cheaper back then!). But obviously you can go with less expensive speakers, depending on how important decent sound is. You may be unhappy with tinny cheap sound, so decent speakers are something to consider.
Also, since you haven't put a discrete graphics card in your x16 expansion slot, there is no HDMI or DisplayPort output which could provide both digital audio and digital video from the graphics card. There is only the VGA and DVI video connectors. So you should not think of getting sound from any speakers built into any monitor or TV that you buy.
Again, for my friend's system I included this very reasonably priced
22" Viewsonic monitor. It was perfectly adequate for his needs and supports "full HD" 1920x1080 16x9 resolution. You would connect it with a DVI-to-DVI cable to the DVI connector on the back of your computer. You don't need a discrete graphics card to support 1920x1080 since the built-in Intel HD graphics will be fine for your "modest needs".
You can of course spend much more money on a larger monitor if you want (just be sure that it has a DVI connector on it, which virtually all monitors do), and for example 24" monitors often provide 1920x1200 16x10 resolution which provides somewhat larger screen real estate in Windows desktop. But for your needs I suspect that 22" Viewsonic or similar would be fine. I do not recommend you use the VGA connection method from PC to monitor, although that will work fine. The digital DVI connection "looks better".