First of all you need to think about what type of website you want? Will it be just a blog, or will it require a store/eCommerce software and/or it's own forum, etc?
Generally speaking if you are not a coder, and it will be a modest site with or without perhaps a small shop element then I'd suggest WordPress
. If you want a more solid and capable eCommerce solution then Drupal
would likely be your best bet currently. If it's a community forum site then I'd probably go with phpBB
. These are all free and each powerful in their own ways.
Unless you intend your website to only be used by friends and family- avoid one-stop free website and hosting packages where you will not have your own independent domain name. For example- mysite.cheapaschipsweb.blah. Because as soon as you outgrow this you will pretty much have to start over, and probably have a good deal of hassle into the bargain.
Next as Jimbo says you should do a test build. The last time I looked all of the above mentioned solutions can be installed and developed on your local machine via installing only Xampp for Windows.
When you are clear about the design and have mastered the basic building elements, only then think about buying a name and any hosting.
At this juncture it's pointless thinking about hosting as it will depend on the type of site you will be building, as to what you will need and where best to get it.
One thing you should never do is buy/host a domain name with the same company your website is hosted with. Keep them separate and you will potentially avoid a lot of grief if/when you need to change your website hosting.
You can build a very credible site for free with minimal coding knowledge. Setting up your domain shouldn't cost more than £10-£20max, the hosting depends on the type of site you want/build.
If you need any detailed help as you progress please don't hesitate to ask...