Downtown Doug Brown Wiki

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
===== Cloning an Ubuntu server to a virtual machine for testing =====

I recently did this to test a server's upgrade from 12.04 to 14.04 to make sure it would be safe to do a remote upgrade over SSH. I'm glad I did, because I found a few config file changes I needed to make.

A lot of the instructions came from [[https://help.ubuntu.com/community/BackupYourSystem/TAR|this Ubuntu wiki page]]. I also had to scrounge around for some instructions about LVM.

This may not work for certain types of virtual servers. I've only tested this process by dumping the data from an actual dedicated server. It would probably work with a KVM VPS too. If you don't have a kernel and grub installed, it's probably not going to work.

==== Make a backup of the existing server ====

This isn't too difficult. Here's what I did to create a .tar.xz file of everything (as root):

**Option 1:**

''cd /\\
sudo tar ––exclude=/backup.tar.xz ––exclude=/dev ––exclude=/proc ––exclude=/sys ––exclude=/run -Jcvpf backup.tar.xz /''

This may not be the cleanest or safest way to do it, but I was able to do it while the system was live and it was only for testing. Note that I excluded some system directories that don't matter.

The /proc, /sys, and /run directories are usually empty on a root filesystem, so you will just need to recreate them on the other end after extracting them. However, the /dev directory may actually contain some device nodes underneath the mounted version of it that you typically see. To be safe, save its actual contents:

''mkdir /tmp/blah\\
sudo mount ––bind / /tmp/blah\\
cd /tmp/blah\\
sudo tar -cf dev.tar dev\\
sudo umount /tmp/blah\\
rmdir /tmp/blah''

When it's finished, copy these backup files (backup.tar.xz, dev.tar) onto a local computer that will be accessible from the virtual machine over SSH.

**Option 2:**

If you can directly SSH in as a user who has enough permissions to make a tarball of everything, do something like this:

On the server:

''mkdir /tmp/root\\
sudo mount ––bind / /tmp/root''

On the computer where the backup will be stored:

''ssh username@yourserver "cd /tmp/root ; tar -czvf - *" > root.tar.gz''

Finally when you're done, back on the server:

''sudo umount /tmp/root\\
rmdir /tmp/root''

==== Create a virtual machine ====

I created a virtual machine with sufficient disk space to contain everything on my server. I grabbed a standard Ubuntu live CD image and booted the newly-created virtual machine from it. Do **not** install Ubuntu. There's no need, because we will be extracting your entire server onto it.
==== Set up partitions ====

Figure out the partition layout (on your server):

''sudo fdisk -l''

Partition your VM's hard drive with gparted and create appropriate filesystems based on the existing filesystem types on your server. Try to set it up so that the partition ordering is exactly the same as it was on the server, although this is probably not totally critical. I do it as closely as possible just to make sure any actions I do in the VM will mimic what actually happens on the server.

Check the ''/etc/fstab'' file on your server. See what it mounts and how – does it use UUIDs? Does it use /dev/mapper (which indicates LVM)? You may have to do some special setup.

For non-LVM mounts, set up your newly-created filesystems so they have the same UUID as your old filesystems to ensure that fstab will find them. To get the old UUID, either look in ''/etc/fstab'', or run the ''blkid'' command. To change the UUID of a partition (/dev/sda1 in this example):

''sudo tune2fs -U <new UUID> /dev/sda1''

If you have LVM (/dev/mapper/<nowiki>*</nowiki> mounts and fdisk indicates a Linux LVM partition), you will need to set up LVM volumes on your virtual machine. See [[https://wiki.ubuntu.com/Lvm|this Ubuntu wiki page]] for info on LVM. Basically, you will need to create a Linux LVM partition on the virtual machine, then run ''pvcreate'', ''vgcreate'', and ''lvcreate'' to set up the volumes on it, followed by creating the filesystems or swap on them. You should be able to see the syntax of the commands on the wiki page I linked. You can use the ''pvdisplay'', ''vgdisplay'', and ''lvdisplay'' commands on your server to see the existing setup so you'll know what to create on the virtual machine.
==== Mount the hard drive partitions in the virtual machine ====

While booted from the live CD, mount the new partitions/volumes in their target locations in a subdirectory. In this example, I'm going to mount the root of the hard drive, which is an LVM volume named "root" in a volume group named "server", as /tmp/root. Then, I'll mount the boot partition (/dev/sda1) on top of it:

''sudo mkdir /tmp/root\\
sudo mount /dev/mapper/server-root /tmp/root\\
sudo mkdir /tmp/root/boot\\
sudo mount /dev/sda1 /tmp/root/boot''

This creates a perfect place where we can extract the tarball to fill the virtual hard drive with everything from the server.

==== Extract the tarball in the virtual machine ====

OK, let's assume your tarball is on another computer available over SSH. Here's the command to use (replace backup.tar.xz with whatever the name is of the backup file):<file>
cd /tmp/root
ssh other-server.local "cat backup.tar.xz" | sudo tar -Jxvf -
</file>

==== Create the directories we skipped earlier (only if we did Option 1 above) ====

We need to create the directories that we skipped during the tarball. They only need to be empty directories; filesystems will be mounted on them later.

''sudo mkdir dev sudo mkdir proc sudo mkdir sys sudo mkdir run sudo chmod 555 proc''

If you want to extract the saved dev.tar file from earlier (probably a good idea to be safe), do that:<file>
ssh other-server.local "cat dev.tar" | sudo tar -xvf -
</file>

==== Create a chroot environment in the VM to install the bootloader ====

Now we need to chroot into this server setup and install fdisk onto the VM:

''sudo -s\\
for f in dev dev/pts proc sys ; do mount ––bind /$f /tmp/root/$f ; done\\
chroot /tmp/root\\
dpkg-reconfigure grub-pc''

It will ask you where to install the bootloader. Pick the main hard drive of the VM (for example, /dev/sda).

==== Clean up and reboot ====

Exit out of the chroot environment:

''exit''

Then, unmount the things you just mounted

''umount /tmp/root/sys\\
umount /tmp/root/proc\\
umount /tmp/root/dev/pts\\
umount /tmp/root/dev\\
umount /tmp/root/boot (if necessary)\\
exit''

Your earlier session is probably inside of /tmp/root, so change out of that directory, and then unmount /tmp/root:

''cd\\
sudo umount /tmp/root''

You may need to create swap on the swap partition. In the example command below, the swap is supposed to be on an LVM volume called "swap" in a volume group named "server":

''mkswap /dev/mapper/server-swap''

Reboot, and your VM should boot up just like it's your server!

==== Other smart things to do ====

Before rebooting, it may be wise to change your virtual machine's networking setup in ''/etc/network/interfaces'' so that it uses an IP address local to your network instead of your server's actual IP address.

Remember that it's going to do everything your server would otherwise do. If this would have bad consequences, be sure to disable anything you don't want it to do, such as automated backups or maintenance. In my case, I turned off some cron jobs I didn't want to run. Keep networking in the VM disabled until you're satisfied that it's not going to interfere with your actual server's operations.

You now have a usable clone of your server. Test out a release upgrade, test out your new config changes, whatever. It's a very convenient way to test things out before you deploy them.