I just built a machine about a month ago. Some tips:
Mushkin has 512MB PC2700 DDR RAM on Sale for $59, even with their shipping that's pretty cheap for decent RAM.
http://www.mushkin.com
I second that it's hard to go wrong buying gear from egghead. Prices are a bit higher than some other places, but you always get reasonable shipping charges, and good service.
I'm not an overclocker, but I have to recommend picking up an OEM Athlon XP 2100+ from egghead. It will save you about $90, and you should be able to get 2600+ speeds out of it with a normal fan. I got a $30 zalmann fan that is very quite, and am running my 2100+ at ~2266 Mhz, which is faster than any AMD release. I bought the EPOX 8RDA+ board which makes the overclocking very easy.
You should be able to find plenty of info on setting it up by searching on google. I was looking at the 2600+ myself for quite awhile, but I kept seeing report after report of people doing this with no problems, so I had to try it. I'm a month into it stable and w/o problems, heck even if the CPU burns up in a year, I can just buy a 2800+ or 3000+ then for $90 and still come out ahead.
[/url]