While it is true overclocking will give you faster performance, the question should be do you really need it? Usually, the only people who really need the increased speed (note that I said need, not want) are serious gamers. The need the fastest machines possible to stay competitive. People who use their computers for web surfing, emails, social media, or office type applications will not significantly benefit from overclocking.
Overclocking is not without its risks. It puts more stress on your components (hence the need for more cooling that others have mentioned in this thread) which can cause shorter component life. It's trickier than using the stock settings because it is easy to push your components too far and frying them so you have to pay attention to what you are doing unless you have a MOBO that watches it for you. Also, many CPUs are locked to prevent overclocking (mostly to ensure longer life) so, unless your CPU and GPU are unlocked (with Intel CPUs, the part number will end with a K), you wouldn't be able to overclock anyway.
If all you need is to boot up your computer and programs faster, adding an SSD and putting only the OS and programs on it and keeping data on a conventional HDD will give you much faster results than overclocking without the risks associated with overclocking.