I haven't logged in on this site for a very long time, but after reading this topic I figured I'd offer a bit of assistance.
First some background:
I build and test supercomputers for a living. We have contracts with NASA, MIT, the Army, Northrop-Grumman... you get the idea. I also used to do support for EVGA.
Let's rule out a couple things right away:
Hard locks like what you're describing are often caused by the CPU overheating. The temperatures you posted aren't anything I'd worry about, though, and since you just built it, chances are that the heatsink isn't completely full of dust yet so it's likely not that. Also, since it can happen before you even get to the login screen, it's unlikely that this is related to heat.
Another component that can cause locks like that is the graphics card overheating. However, in those circumstances, the lock is almost unconditionally preceded by artifacting (visual anomalies usually taking the form of screen flickering, strange triangles, corrupted textues... that sort of thing--You'd know it if you saw it). While it definitely COULD be the video card, I'm REALLY not leaning in that direction, especially since you already tried a different one.
That being said, let's get down to actually doing something.
When troubleshooting, always try the easiest, least invasive things first. Let's make sure it's not driver-related. When you first turn on your computer you'll either see an ASUS splash screen or some information about your CPU and memory that's printed out to the screen during POST (power-on self-test). When this goes away, before the Windows loading screen comes up, start mashing F8 on your keyboard. If you timed it right, you'll be presented with the advanced boot options menu. If you don't see this menu and instead just simply go to the loading screen as usual, no big deal, just hit reset and try again. From the advanced boot options menu, select "safe mode" and hit Enter. This will load Windows with minimal drivers and no startup programs. You won't be able to access the internet, everything will look like shit, and it will be a general unpleasant computing experience. Do random things or just walk away for a while--we want to see if it freezes in this state. If it does, it's the problem is probably not software-related (I can tell you right now that it's probably not software-related, but we should definitely rule that out before dicking around with hardware). If it doesn't freeze, we haven't REALLY learned much, but this test is worth doing nonetheless.
I'm just going to assume that it's going to freeze in safe mode so let's continue.
Disconnect all non-essential devices. Disconnect you optical drive(s), remove any add-in cards (except your video card, you'll need that), disconnect any USB devices... All you need is your power supply, motherboard, CPU, memory, video card (and monitor), and your hard drive. If this is a new mouse/keyboard, borrow them from another computer that you know works fine and use them instead. I KNOW it probably has absolutely nothing to do with that but just do it--every time we eliminate the possibility of it being any particular component, we're one step closer to finding the culprit.
It'll probably still freeze which will bring us to memory.
I know you ran memtest (it's great that you tried this), but something to note is that memtest isn't very good at finding problems. It never hurts to run it--we run it on every computer we build--but it almost
never finds anything. We use a Linux tool called Linpack to test for memory problems but that's no trivial matter by any stretch--I'm mentioning this purely to illustrate that memtest is not to be relied on for anything. I'm really leaning toward this being a memory-related issue and strongly suggest you try to rule that out before looking at other hardware.
The first thing I would do would be to move that 2GB DIMM over to the slot furthest away from the CPU (forget what the manual says, with a single DIMM it won't make any discernible difference whatsoever which slot it's in) and give that a test. If it works, awesome! Just leave it there. If it still fails, get your hands on another DDR2 DIMM (shouldn't be too hard, try to borrow one) and test with that. If possible, also try your DIMM in a known-good system. Like I said, I'm pretty confident that this is your problem.
If swapping DIMMs has no effect, it's obviously got to be something else. My guess would be the motherboard but let's keep going because that's a pain in the ass. CPUs are almost NEVER faulty so let's just forget about that component for now. All that's really left untested are the power supply, the motherboard, and the hard drive.
I doubt it's your hard drive but I'd advise throwing a different one in and seeing what happens. There are hard drive testing utilities like smartctl and vendor-specific utilities (in your case, you can grab Western Digital's diagnostics tool from their website) but I don't think that's your problem and it will be easier to just try a different hard drive. I REALLY doubt it's your drive.
Regarding your power supply, if you go into your motherboard's CMOS (press delete during POST to enter setup) and go take a look at the voltages you're getting (usually on ASUS boards you go over to the "Advanced" screen and then go down to "Hardware Monitor"), you'll be able to see if any are way off. Chances are great that you won't learn anything from doing this so if you happen to have another power supply that has PCI-Express power (for your GTX 260), you can try swapping it out. The power supply
could be causing this, I guess, but I'd honestly be pretty surprised. Definitely worth a shot though.
Finally, we get to the motherboard and CPU. Hopefully you've found the problem before this point but if not, we're going down this road. If you can get your hands on another LGA 775 CPU, give it a whirl. It might make a difference but, honestly, if it wasn't the memory, it's probably the motherboard.
Good luck and keep us posted. Don't worry, we'll get that fucker running one way or another.
