Ok - that's the overview. Lets look at things in more detail.
The PCThere are lots of things to consider for anyone starting out but before you actually get started think carefully about where you are going to put your PC. For example, if you end up putting the PC in the lounge, you might want something that blends in with the rest of your lounge. A beige box in amongst a black tv/stereo may not be best. However if it is being stored away in a cupboard, and you are just streaming to the tellie, then you can probably repurpose an old pc
The second thing you need to consider is the FAF (Family acceptance factor, also called the wife acceptance factor). You will want something that is reliable, easy to use, and easy to explain to people with not too many remotes.
You can buy, build, or buy a specific device eg boxee, chromecast, appletv. There are pluses and minuses for each solution, so take time to consider each and pick a solution that will work best for you. So if you consider that, and still want to go down the PC path, you can buy or build. I was definitely build, because I've bought too many PCs that were constrained in some way by the manufacturer, and couldn't be upgraded. Or after 2 years, I needed to upgrade because it no longer worked the way I needed my pc to. It needn't be too expensive. Mine was $1300, but as you will see I could have done it for less. However after my first pc build, Ive ended up with a pc that still meets my needs and then some, 2 years later
Here are some things you need to consider:
The CPUIn most cases an i5 type chip will be perfect for an HTPC. I went with an i7 chip with video on board. In the short term that meant I didn't need a separate video card and the i7 extended the life of my PC a bit (generally people don't upgrade their CPU's). If I needed a video card I could buy one later. The cooling fan I used came with the chip.
The MotherboardThis is the main board on the PC. I looked through a few computer magazines and found one recommended for a HTPC build. It was blue!. It had 6GB/s SATA interface on it (basically means a fast interface between the board and the drives). I also wanted a USB3 external connection, for moving files about quickly, and obviously an HDMI port - again audio on board saved me some money but I could buy a separate audio card later if I wished. Audio needs to at lease=t be 5.1 channels.
You can choose different sized boards to fit in different sized PC's so consider the case along with the board. I put mine in a full sized PC, in retrospect though I maybe should have gone with something a little smaller ;-)
Hard Drives I ended up dropping in a few hard drives I had from other PC's. I think I also bought a 2TB Western Digital green. The storage drives should be large. You will need them to store all the recorded content you will have, particularly if you plan to move your dvd's and video onto the PC. If you only plan on streaming, then you may not need as much space - however 2 TB isn't that much these days
The operating system (OS) on my build sits on a 180GB SSD drive. These things are amazing and I'd recommend at least dropping your OS onto one. They run fast, so the PC is always zippy. Boot up time for Windows 10.1 is about 20 seconds from start, and I'm never left waiting for the operating system to catch up. Putting the OS onto a separate drive also means you can reload it at any point without destroying all your files. It's worth spending the extra on one of these if you have it.
The CD / DVD / Blu Ray PlayerNot much to say here. Get one that will burn disks - at least DVD's, and plays Blu Ray. Think I paid about $60 for mine. When Ultra HD comes in I can replace it with a better model. Make sure it is reasonably quiet, so do a search online and read some reviews
MemoryI put in 2x 4gb memory sticks in mine (I was planning on running 64bit windows. 32 bit windows will read up to 3.5GB only, which isn't enough). There are 2 extra slots so I can put up to 32 GB in it if I need to. THere is a point where if you add memory it will cost you more, but not actually give you much benefit. I'm happy with my 8GB
CaseThis is where you should spend a bit more. It needs to look reasonable, but it also needs to be quiet. You don't want to be listening to a whirring fan or a rattling case half way through your movie (believe me I have a noisy PS3 I never use because of the fan on it). The case I have has very little in the way of connections on the front. Inside the case has a silencing compound on the inside, and all the drives are suspended so they don't cause vibrations on the case. Don't compromise here it is worthwhile paying extra to get what you want.
Additional FansThe case came with a fan in the front, I put a fan in the back. It needs to be a silent or near silent fan, so should be large which will mean it will run at a low speed. I forget what size it was, but it wasn't expensive - just a silent fan
Power supplyA near silent power supply is a must. Just get what you need power wise, but make sure it is a good brand. Cheaper brands won't last and won't be quiet. I think mine is a 600w power supply which is heaps
Minimising noiseAs you have probably guessed minimising noise is key. It's why I don't have any video cards in the PC as it's just one less fan to make noise. Through careful selection of components I ended up with a PC that I can only just hear about 10cm away from it, so I'm pretty happy with that.
You can vary the formula depending on what you want to do - save costs in some places and spend more in others to get exactly what you need, but that's the basics to get you started.
Any questions, comments on component choice, shout out.