TL;DR? PCs have vastly superior visual capabilities and customization at the cost of reliability, easy of use, price, and often optimization.
Consoles are basically just custom PCs with very specific hardware/software devoted to gaming. Because of this they can provide several advantages that a less focused PC build might not be capable of with the same level of hardware. One advantage is optimized visuals. Take for example the recent sequels such as in the Halo or Skyrim franchise. They're essentially running on the same computer but developers have found was to use the hardware more efficiently and produce visuals far greater than an equivalent PC. They can do this because of a constant in the market in terms of hardware and software. A few more advantages that this constant provides is a much more efficient, hence cheaper, method of production and an easier to maintain, reliable, user friendly system.
PCs on the other hand, as already pointed out, provide a lot more freedom. You don't have to stick with the same level of hardware for 7+ years. You can upgrade small amounts at a time without having to replace the entire system, which might end up costing you less than having to buy a new system down the road. This will change for a short while when/if the new consoles are released, but PCs currently offer vastly superior visuals. You can also, very often, find games a lot cheaper and play a lot of games that aren't easily run on consoles controllers such as MMOs, RTS, etc. You can also customize a PC a lot more than a console with extra screens, gaming keyboards and mice, controllers and other peripherals but this "might" also change in the next few generations as well.
Another thing to consider is the online community. You'll get advocates on both sides claiming to be superior, but in my experience they both pretty much even out in their own respects.
In short, consoles can be made cheaper and its easier to develop more reliable and optimized games for them. PCs can be upgraded cheaper, provide better visuals, and games can potentially cost a lot less and be more varied. They, currently, can also provide a lot more customization in terms of displays and peripherals.