Windows 7: 8-Core, overclocked machine slow...

I would test RAM stick individually, and run the test overnight, and if an error is found with the RAM stick, place the same RAM stick in a different motherboard slot (repeat for all the slots), if an error is found in all the slots for the same RAM stick then you have a faulty RAM module.

However, if no errors are found for the RAM module in a different slot, then you have a bad motherboard DIMM slot. Remember, to make sure the contacts on the RAM modules are clean and the motherboard slots are not covered or full of dust.

Run the test for least 6-7 passes.

also note that most java and flash around are mostly single or two-threaded programs. That means all cores beyond the 4th are there just to keep time because there are only two "streams" of instructions to process. Having more cores means that each core taken alone is weaker than a processor with less cores but still running at the same frequency. Given modern needs, it's not a so good idea, as lots of programs (and games) aren't going to multithread over 4 cores, go figure over 8. So you have half of your processing power that is basically wasted.

AMD stuff shines when you are doing heavily multithreaded things (basically workstation or server stuff), but for gaming and personal use the best one is around the i5 3750k, performance-wise.

Also, now that you mention overclocking, AMD cores do a trick when reaching dangerous temp, and this could cause the CPU to run like a crappy i3 as overheated cores downclock to 1 Ghz or so. Your processor is a Bulldozer series, which is known for heavy power use (and high thermal output).

Check if your cores are cooled well enough to keep their stated max frequency even under real loads with programs like CPUz. This page tells more details, although it's for a slightly more modern processor of the same family.
It could be likely that you need to decrease your overclock or get better cooling to keep them from auto-downclocking.

Keep testing in all the slots, the tests will take time, but least you will sure what RAM stick or DIMM slot is causing the problem.

I started testing before I read your post. I pulled two, moved my SSD, and added another HD to my system for VMs. Ran the test, and almost INSTANTLY, I got an error. Shutdown, swapped out the first stick and running again. 49% thru the test without errors. I'm guessing it's safe to say it's the stick no the slot...

Once I positiviely identify the defective stick [I think I have] I will work on getting a replacement,but will move that stick to the very back to limit the number of crashes since my replacement will travel by mail.

The first stick is now in the back. [All in reference to the front of the case.] I am re-running Memtest86+ and the error came up almost instantly. When I swapped out the one stick, it went thru the entire test without errors.

400 MHz DDR-800, while the forth stick said 667 MHz, DDR-1334

I am undecided on 1 or 2 cores for the VM, but I'm going to lock it into Core 8, and maybe core 7.

If you have 16 GB in 4x4GB banks you can safely live without 4GB for even a month. Unless you do stuff that specifically demands it, you won't be able to fill 8 GB of ram.

The position of the memory stick is irrelevant unless you were testing the slots themselves, as all ram is going to be used regardless of the slot.
Ram tests are usually done on a single ram bank, just to make sure that if there are errors they 100% come from that bank.

400 mhz ram? Isn't it a bit outdated for a Bulldozer processor?

And yes, mixing rams working at different speeds could result in issues. if the stick giving errors was the 667Mz one, try removing all the others and placing it in the first slot, then repeat the test.
If it comes out clean it was one of such "mixing ram issues", not a defective stick.

I'd give it more, 3 or 4 cores if it can use them. Remember that each core is around half the power of a good quad-core's core.