Windows Memory Diagnostic is just as good as MemTest - though I would suggest the newer Memtest86+ - Advanced Memory Diagnostic Tool
instead of MemTest. Or use Windows 7 internal Memory Diagnostic Tool:
Control Panel > Administrative Tools > Windows Memory Diagnostic
Is it ok if I pull out 2 sticks, (2Gb a piece), and run the test on 4 gigs? Thanks.
Actually, no test is perfect so pulling out 2 sticks then using your computer normally is probably the best test. If no problem, swap in the other pair. Just remember to unplug the computer from the wall before starting, and touch bare metal of the case to discharge any static before reaching in.