I’ve recently installed Ubuntu 12.04 Long Term Support (LTS). The main advantage of a LTS distribution is that once you clean it up, it stays like that for two years.
First step is to switch your desktop to LXDE, unless you like Unity or Gnome the Third. The recommended way is to install Lubuntu, in my case I will install LXDE on top of regular Unity. If you already have regular Ubuntu installed, it would be too much trouble to start downloading and installing everything. Also keep in mind that Lubuntu is not a LTS release, and the applications installed are different, for example Goolge Chromium instead of Mozilla Firefox, or Gnumeric instead of LibreOffice Calc. LXDE is such a small desktop component, it might be a better idea to chose your distro based on the applications it provides and switch the default desktop environment to LXDE.
$ sudo apt-get install lxde