Well, I'm really starting to wonder about the raves that Eclipse gets. They don't include a Windows installer (heck, even I include one for BottomFeeder) - IntelliJ and NetBeans both do. I'd suggest they run off to find NSIS and have a look at that - it's simple, and it's free. Doing so would probably get rid of the problems I had.
So anyway, after a brief look at IntelliJ, I'm grabbing NetBeans. First set of points over Eclipse - they ship an installer for Windows. Heck, it's smarter than the IntelliJ installer - rather than trusting me to find a JDK, it searched for a suitable one at install time. Which limits the number of setup mistakes I could make.
Well, that was shortlived. On trying to run, it tells me that it can't find java.exe in a tiny dialog box. I think I've found the source of the odd Eclipse issue, and of many of these other ones - my path has a 1.3.1 JRE in it, which was installed by Oracle's client. So it looks like I need to muck with the OS level path. Joy.