Different components show different signs when they start to fail.
For Power supplies, the amount of power put out will degrade over time. Hence its a good idea to get a PSU with more overhead than you expect to use and not to run the PSU at full capacity full time. When PSU's start to go bad, you will get random restarts or lockups or error messages on startup or during heavy or light loads. This is usually the first thing that will die in a computer over time.
You can test the output of the PSU with a multimeter and by looking at the specifications of the PSU to make sure that the 12v rail and 5v rail are supplying the correct voltage. Here is a good guide for that:
How To Manually Test a Power Supply With a Multimeter
There are also devices you can buy to test the PSU that have all the connectors setup so you just plug it in:
How To Test a Power Supply Using a Power Supply Tester
For motherboards, components on the mobo can fail. Since the motherboard is a group of components, this can be seen by many different problems. Usually, on the motherboard, this will be a complete failure or a partial failure. In a complete failure, the motherboard will not go to POST. with this, a component essential to functionality has failed and will usualy do so without warning. With a partial failure, the individual components will start to not work. The USB drives might start failing or not recognizing external drives or mice if the USB interface starts to fail. Problems with Hard-drive SATA connections can also occur causing error messages that would look like hard-drive failures. the northbridge or southbdrige could also fail. Northbridge causeing video and memory errors, the southbridge causing problems with USB, audio and IDE drives.Here is a good synopsis on the two bridges:
http://en.wikipedia.org/wiki/Northbridge_(computing) (click link for southbridge on same page)
A partial failure is usually a warning sign of complete failure. The motherboard can still function with a partial failure, but a complete failure of the motherboard is usually expected. could be months, days or years though depending on the component that failed.
For the CPU (FYI: CPU's usually now include the Northbridge like Sandy-bridge and AMD Fusion) CPU's fail in one way. they burn out... The CPU is pushed to highest capacity before shipping to make sure the chip won't fail. if a chip fails at this time or shortly after being used new, it is called a Burn In Failure. if it passes this torture test, the component will usually last for quite a long time. The only thing to kill it would be heat. The symptoms of the CPU failing due to heat would be: system errors and system crashes, failure to POST, and the system starting but then shutting off. Sometimes, by the system starting up but with high fan usuage and won't load.
When Hard Drives start to fail, they can make noise like clicking or whirling sounds. This is not good. Also sometime before failure, a drive will start to be very sluggish. When it fails, the HDD could be not recognized in the BIOS or reported incorrectly in the BIOS. Usually the HDD maker will have a tool to check your hard-drives like Seatools:
SeaTools | Seagate
When RAM starts to die, you can have single chips on the RAM Stick or the whole stick fail. this can be seen in the system not seeing the RAM at all or if you have an error message or crash that references the same location or error code repeatedly. You can use Memtest+ to test the RAM to see if it has any issues. Info on this:
RAM Failure: Symptoms and Diagnosis
Edit: if I have made any errors in here, please correct me. I still have much to learn