I normally try to steer clear of system utility tools such as registry cleaners, generic Windows clean-up suites and so on, because in my experience they can do more harm than good.  But a simple, free program called Tweak Me seems to work rather well, and it's easy to reverse any changes because you can keep track of the selections you make.

Tweak Me is a 1 MB download, runs on anything from Windows XP to 8, and is free.  You can get it from http://www.wecode.biz/p/tweak-me.html.  Download, unzip, run the installer, and you're ready to roll.  I've checked the installer with VirusTotal and it's clean.  

Incidentally, the installer asks if you want to install a certain browser toolbar along with the program.  If you don't, remember to untick the box when you're asked. Warning: This program is also bundled with OpenCandy. See our information page here for more details.

Using the program is easy.  Just tick or untick the box which corresponds to the setting you want to change, then click Save.  Your tweak is now done.  For example, I used the screen shown in the picture below to turn off tabbed browsing in Internet Explorer and it worked perfectly.

Helpfully, the tweaks are colour-coded as red, amber or green, according to those that are suitable for novices and those that are best done by experienced users only.  Plus you can also use the program to clean up your PC or manage the startup options too.  

For safety, the program offers you the option to set a Windows restore point each time you run it.  I personally suggest that you should also keep a note of which boxes you've ticked, so that you can go back and un-tick them if things don't work as you expected.  

Also, it always makes sense to have a recent backup of your valuable data files, regardless of whether or not you're trying a program for the first time.

 

Thanks to Panzer for the recommendation