Painless multi-booting with multiple UNIX systems

If you have a desktop, or laptop, or server, and want to boot multiple UNIX (or other) systems using GRUB, here are some things to keep in mind, let us say if you have one partitioned disk. Whenever I ignored my own advice, it has given me headaches:

  • Have a common /boot directory for all systems. I have a 1.5TB desktop and have given the shared /boot partition 1 gig. It uses 129 MB of it - much of it for the 7 initrd images, then some for the 7 compressed kernels etc. I have it as ext3 instead of ext4 due to some of the older systems.
  • Sometimes systems update their grub, or their kernels, re-run grub setup and installation, and this can mess up your other kernels. It is a good idea to copy relevant menuentry's from /boot/grub/grub.cfg to /etc/grub.d/40_custom for *each* of the systems that you do system updates on, and which might re-run grub setup. Standardizing this file across distributions means running grub setup should have a standard behavior, no matter under which system things are run.
  • I share swap across all the systems. Sharing /home depends - I use one system as my primary, and it has my big /home partition - on other systems I mount a local /home off of root (/), but don't have much there. There is another directory to consider sharing, which I'll mention later.
  • Have the Master Boot Record (MBR) be on top of the whole disk, not one of its partitions
  • I have a Ubuntu 10.04 partition which I do not use much, but leave there so as to boot from. If I have problems with the two systems I use the most (right now Ubuntu 10.10 and Debian sid), I can go back to that old reliable one
  • I grab a copy of the grub.cfg or /etc/grub.d/40_custom files once in a while and put them on another machine. In case I need to by hand type in which kernel I want grub to load, if grub setup has some problem
  • Remember when partitioning the disk initially that, for many cases, a disk can only have four real ("primary") partitions. I usually designate the first one the active partition, and the last one the extended partition. I then sub-divided the last partition into logical partitions.
  • My disk usage of root partitions for each system range from less than 4 gigs (Gnewsense, which I unfortunately do not use much) to 10 gigs (Ubuntu 10.10). Since I have 1.5TB of space, I give them all enough extra space to the point where I don't have to worry or think about it.
  • The final thing to think about is KVM - kernel-based virtual machines. I have Ubuntu 11.04, which is currently (as of November 2010) in alpha, in a KVM, and I have other OS's in KVMs as well. The images are in image files, which are located in /var/lib/libvirt/images. If you're going to use KVM's across different OS's, this directory can be shared. The existence of KVM's lessens some of the need to install OS's to existing partitions.