Madhav Kobal's Blog

This blog will be dedicated to Linux, Open Source and Technology news, affairs, how-tos and virtually EVERYTHING in these domains.

Posts Tagged ‘Virtualization’

How to control virtual machines (Virtualbox) using VBoxManage

Posted by madhavkobal on 26/08/2009

VBoxManage, a command-line utility that allows you to control all of VirtualBox’s powerful features.In essence, VBoxManage supports everything that our graphical user interface allows you to do with the click of a button. VBoxManage supports a lot more than that,however. It exposes really all the features of the virtualization engine, even those that cannot (yet) be accessed from the GUI.

There are two main things to keep in mind when using VBoxManage

  • First,VBoxManage must always be used with a specific “subcommand”, such as “list vms” or “createvm” or “startvm”.
  • Second, most of these subcommands require that you specify a particular virtual machine after the subcommand. There are two ways you can do this:

You can specify the VM name, as it is shown in the VirtualBox GUI. Note that if that name contains spaces, then you must enclose the entire name in double quotes (as it is always required with command line arguments that contain spaces).

Example

VBoxManage startvm “Windows XP”

You can specify the UUID, which is the internal unique identifier that VirtualBox uses to refer to the virtual machine. Assuming that the aforementioned VM called “Windows XP” has the UUID shown below, the following command has the same effect as the previous:

VBoxManage startvm 670e746d-abea-4ba6-ad02-2a3b043810a5

You can type VBoxManage list vms to have all currently registered VMs listed with all their settings, including their respective names and UUIDs.

Create a new virtual machine using VBoxManage

To create a new virtual machine from the command line and immediately register it with VirtualBox, use VBoxManage createvm with the -register option.

Example

VBoxManage createvm -name “SUSE 10.2″ -register

VirtualBox Command Line Management Interface Version 2.1.0
(C) 2005-2008 Sun Microsystems, Inc.
All rights reserved.
Virtual machine ’SUSE 10.2’ is created.
UUID: c89fc351-8ec6-4f02-a048-57f4d25288e5
Settings file: ’/home/username/.VirtualBox/Machines/SUSE 10.2/SUSE 10.2.xml’

As can be seen from the above output, a new virtual machine has been created with a new UUID and a new XML settings file.

Show configuration of a particular VM

The showvminfo command shows information about a particular virtual machine.This is the same information as VBoxManage list vms would show for all virtual machines.

VBoxManage showvminfo “Windows XP”

You will get information similar to the following:

VirtualBox Command Line Management Interface Version 2.1.0
(C) 2005-2008 Sun Microsystems, Inc.
All rights reserved.
Name: Windows XP
Guest OS: Other/Unknown
UUID: 1bf3464d-57c6-4d49-92a9-a5cc3816b7e7
Config file: /home/username/.VirtualBox/Machines/Windows XP/Windows XP.xml
Memory size: 128MB
VRAM size: 8MB
Boot menu mode: message and menu
ACPI: on
IOAPIC: off
Hardw. virt.ext: off
State: powered off
Floppy: empty
DVD: empty
NIC 1: disabled
NIC 2: disabled
NIC 3: disabled
NIC 4: disabled
Audio: disabled (Driver: Unknown)
VRDP: disabled
USB: disabled
USB Device Filters:
<none>

How to Change Virtual Machine Settings

VBoxManage modifyvm this command changes the properties of a registered virtual machine. Most of the properties that this command makes available correspond to the VM settings that VirtualBox graphical user interface displays in each VM’s “Settings” dialog.

Changing memory settings using the folowing command

VBoxManage modifyvm “Windows XP” -memory “512MB”

One more example to set memory, operating system type, pae settings, monitor quantity, hardware inventory as well as snapshot configuration. Here is a sample command that sets a memory amount, makes the CD-ROM disk the first boot device and disables USB support

vboxmanage modifyvm XP-TestSystem -memory 512 -boot1 dvd -usb off

The modifyvm parameter also has extended options such as BIOS display time, network interface driver type, host network interface assigned to the VM and enabling or disabling of the clipboard. Overall, modifyvm has over 50 parameters for an individual VM.

If you want more options check virtualbox usermanual

How to start virtual machine from command line

VBoxManage startvm This command starts a virtual machine that is currently in the “Powered off” or “Saved” states. This is provided for backwards compatibility only.The optional -type specifier determines whether the machine will be started in a
window (GUI mode, which is the default) or whether the output should go through VBoxHeadless, the VRDP-only server.

This is a very quick sample of what is capable with the VBoxManage command.

Advertisements

Posted in Uncategorized | Tagged: | Leave a Comment »

Convert VMware .vmdk to KVM .qcow2 or Virtualbox .vdi

Posted by madhavkobal on 21/08/2009

I wrote this how to as I was having problems converting a VMware image to KVM. The existing tutorials all suggest using qemu-img to convert the .vmdk, however it was not working as qemu-img only supports VMware 3 and 4 compatible image formats.

At least that is what Google searching and reading the qemu man pages yielded after I got this error message :

qemu-img convert Ubuntu.vmdk -O qcow2 Ubuntu.qcow
qemu-img: Could not open ‘Ubuntu.qcow’

And to make matters worse, it was difficult to find any information on converting the .vmdk if it was split into multiple files.

This is how I ended up converting. You can convert from a .vdi or a “flat” .vmdk


First convert the .vmdk to a format compatible with qemu-img.

Turns out this can be done with vmware-vdiskmanager.

1. Converting from .vmdk

Use vmware-vdiskmanager to create a copy. This works with a single or multiple disks.


ls
Ubuntu.vmdk
Ubuntu-f001.vmdk
Ubuntu-f002.vmdk
Ubuntu-f003.vmdk ...

vmware-vdiskmanager -r Ubuntu.vmdk -t 0 Ubuntu-copy.vmdk

Note: If you have multiple disks, use Ubuntu.vmdk as well (you do not need to convert each Ubuntu-f001.vmdk).

Note: That is a -t Zero not a capital O. see man vmware-vdiskmanager.

Note: vmware-vdiskmanager is part of vmware server (and workstation, not sure about player).

2. Alternate – Converting “flat files”.

Flat files are used by vmware if you create a virtual disk (vmdk) with the “Allocate all disk space now” option (you have this option when creating disks for use with vmware). Flat files contain all the data from your .vmdk and can (usually) be converted directly.

Notice, flat files can be directly converted to .qcow. If you wish to convert to .vdi (VirtualBox) convert flat to raw.

KVM :

qemu-img convert Ubuntu-flat.vmdk -O qcow2 Ubuntu-copy.qemu

Note: That is a capital O qcow2

RAW (for VirtualBox)

qemu-img convert Ubuntu-copy.vmdk -O qcow2 Ubuntu-copy.qemu

Note: That is a capital O qcow2

3. Boot the image with KVM

kvm -hda Ubuntu-copy.qcow -net nic -net user -m 512

Caveats :

  1. If you have vmware-tools installed, you will have mouse integration.
  2. If you have vmware-tools installed, the guest desktop may well be larger then the kvm window. You will need to resize the guest display to 800×600 .
  3. I could not convert a .vmdk which was using LVM (Fedora).

VirtualBox – Convert to .vdi

1. First use qemu-img to convert the copy .vmdk to raw.

qemu-img convert Ubuntu-copy.vmdk Ubuntu-copy.img

qemu-img with no options will make a raw image. If you prefer you can specify

qemu-img convert Ubuntu-flat.vmdk -O raw Ubuntu-copy.img

2. Then convert the raw image with VBoxManage

VBoxManage convertfromraw – -format VDI Ubuntu-copy.img Ubuntu-copy.vdi

Note: Two – – in front of “format” (WordPress converts two – – to one long one).

3. Start VirtualBox, make a new machine or add the Ubuntu.vdi to an existing machine.

Posted in Uncategorized | Tagged: | Leave a Comment »

How to create VirtualBox virtual machines from Command line

Posted by madhavkobal on 20/08/2009

You can create VirtualBox virtual machines from command line in 3 easy steps. We will assume Ubuntu 9.10 in the following example.

  1. Create new virtual machine:$ VBoxManage createvm --name "Ubuntu 9.10" --register
  2. Create virtual hard disk (5 GB):$ VBoxManage createhd --filename "Ubuntu.vdi" --size 5000 --remember
  3. Modify virtual machine:$ VBoxManage modifyvm "Ubuntu 9.10" --memory "512MB" --hda "Ubuntu.vdi" --dvd /home/toor/karmic-desktop-i386.iso --acpi on --boot1 dvd --nic1 nat
    • –memory – memory size
    • –hda – specify virtual hard disk
    • –dvd – specify ISO image file
    • –acpi on – enable ACPI support
    • –boot1 – specify boot order
    • –nic1 – network setting

Now you can start virtual machine by typing this command:

$ VBoxManage startvm "Ubuntu 9.10"

Posted in Uncategorized | Tagged: | Leave a Comment »

Migrate to a virtual Linux environment with Clonezilla

Posted by madhavkobal on 20/08/2009

Thanks to two concepts driving the data center today—economy and green computing—server virtualization is a hot topic in the IT world and a lot of hardware and software vendors are offering solutions. Virtualization solutions can have several benefits: they can let you easily migrate a virtual machine from one server to another, and let you provision a virtual as a reproducible clone image for, say,software development and testing.

The process for achieving virtualized server consolidation depends on your starting point:

  • If you are designing your system from scratch, you will likely implement virtualization from the ground up.
  • If your system already exists in the physical state, you will probably want to migrate from the physical environment to the virtual one.

I refer to the second process as physical-to-virtual migration, or P2V, and it’s the subject of this article. Several methods, manual and automatic, are generally available to facilitate P2V migration. I’ll talk about those methods and show you how to convert a physical server to a virtual machine using Clonezilla Live, a cloning-based solution.

With Clonezilla Live, you first create a system image from the physical server, then use the boot CD to restore the image onto the virtual machine’s hard disk. It’s simple and fast, and it should be useful for software developers, system administrators, and support engineers who want to analyze a system copy while maintaining the original one unchanged (and avoid possible hardware mismatch challenges).

Note: Virtual and physical systems hardware and software span a broad range, so the procedures in this article are meant as a reference guide. You may have to do some things differently to get them to work for your migration project.

Introducing Clonezilla Live

Clonezilla is an open source (GPL) Norton Ghost-like duplication and clone solution that you can use to clone a particular partition or entire disk. There are two releases: Clonezilla SE (server edition) and Clonezilla Live. Clonezilla SE is best suited for backup and restoring multiple servers simultaneously across the network. Clonezilla Live is a more lightweight build for single-machine cloning.

Clonezilla Live is a combination of Debian Live and Clonezilla; it has the following features and benefits:

  • It clones only the used blocks on the hard disk.
  • It provides multiple file system and even LVM support, including ext2, ext3, xfs, jfs, and LVM2 under GNU/Linux; FAT, NTFS under Microsoft Windows; and HFS+ under Mac OS.
  • You don’t need a diskless remote boot server (Diskless Remote Boot in Linux, DRBL) in Linux to set it up, as you do with Clonezilla SE.
  • CD/DVD, USB flash/hard drives, and PXE boots are supported.
  • It has a customized capability for boot and recovery procedure.

Clonezilla Live uses such existing tools as Partition Image, ntfsclone, partclone, and dd to clone the partition or disk. For unlisted file systems, Clonezilla uses dd to copy all used and unused blocks.

_____________________________________________________________________________________

Virtualization and system migration tools

When migrating a physical server to a virtual machine, you may run into the same limitations as when migrating one physical sever to another. For example, you can not move a 64-bit system image to a 32-bit platform. You are also most likely to encounter Hardware Abstract Layer (HAL) issues when restoring a Windows image onto the virtual machine.

Many free and commercial tools can help with system migration between the physical and virtual worlds (PlateSpin PowerConvert, VMware Convert, Microsoft® Virtual Server Migration Tookit and the cloning software); these migration tools can also solve the potential problems brought by the hardware mismatch between the physical server and the virtual machine. One of their roles is to pass the necessary drivers to the operating system kernel and initialize the drivers properly during the system boot phase. Of course, most of these P2V tools are more than you need for single-machine migration.

VMware Server is a free proprietary platform that provides support for multiple guest operating system types (from Windows, Linux, Solaris, etc.) on both Linux and Windows hosts. As a full virtualization solution, the virtual machine monitors (VMMs) running on the hypervisor provide the virtual machines with all the available types of hardware resources, including the virtual BIOS, virtual memory, virtual SCSI controller, virtual network card, etc. Because the guest operating systems are fully separated from the underlying hardware, they can run unmodified.

Now let’s look at P2V migration using Clonezilla Live to clone a Windows system image onto the virtual machine hosted by VMware Server.

_____________________________________________________________________________________

Step 1. Cloning a system partition

First, boot the physical server from the Clonezilla Live CD.

Clonezilla provides several boot choices. For example, you can select the To RAM option to load Debian Live with Clonezilla into memory and leave the CD-ROM available for other uses. Select the default, and you will get the language and keyboard layout settings. Leave the default as is.

Next, choose whether to use a GUI-based backup and restore wizard or a Debian shell for advanced access and operation. Use the Start_Clonezilla option unless you are an experienced user. You’ll see how to use the shell under Restoring the image onto the virtual machine.

Now, there are two options listed regarding the cloning types:

  • device-device is for device-to-device direct cloning.
  • device-image is for device-to-image backup.

Because you are converting a physical system to a virtual machine, the device-image type is the better choice.

Next, you have to select the image directory where the backup image writes to or restores from, as shown in Figure 1. The ssh-server, samba-server, and nfs-server options are useful when you have a network connected, and local disk free space or a backup image is not available. In our case, we will clone the system partition and save it onto another partition within the same hard disk. If you select the local_dev, you have the opportunity to choose the USB device as the Clonezilla image repository.

Figure 1. Image repository choice
Image repository choice

Depending on the internal and external (USB) inserted disks, you are now asked to select the place that will be linked to the image repository, /home/partimag. An image actually is a directory under /home/partimag with all the image-related information and data files. In this example, the system partition /dev/sda1 will be saved as the image put onto /dev/sda5. Thus, sda5 is selected to be mounted as /home/partimag/ as shown in Figure 2.
Figure 2. Image partition
Image partition

Figure 2 also shows additional command information to mount the image device and the current file system layout with the free space usage report on /home/partimag. Make sure that the image repository has enough free space to store the data within the system partition.

As shown in Figure 3, you can confirm the home directory for the Clonezilla image within the image device.
Figure 3. Clonezilla mode
Clonezilla mode

After this step, choose the mode that you are planning to use in Clonezilla Live. You can choose to save or restore the whole local disk or an individual partition. Select the third item, saveparts, for the partition backup.

After selecting the mode, you are asked to enter the image name (Sys01-2009-02-23-img) and the source device (/dev/sda1) to back up. The next several figures show advanced options for the image-creation process. Because Clonezilla supports various file system formats and is integrated with partition cloning tools such as ntfsclone, partclone, partimage, and dd, the priority order of the tools to use can be set for the current file system. Leave the default settings if you are not familiar with the file system to be cloned. The default setting of -q (ntfsclone > partimage > dd) is best suited to our NTFS-based backup.

Figure 4 shows other parameters that you can set to optimize cloning.
Figure 4. Clonezilla advanced extra parameters: clone
Clonezilla advanced extra parameters: clone

For a Windows system image, the option -rm-win-swap-hib saves backup and restore time by excluding the swap and hibernation files. The -gm or -gs options are recommended if you want to transfer the saved image to another place and need to check image integrity when you restore the image.

Clonezilla Live also allows you to choose the compression method with efficiency and duration considerations. Some file systems do not support files larger than 2GB, so there is an option to set the maximum file size by splitting the image into multiple volume files.

Finally, you can designate an action with the -p parameter to indicate what to do after the cloning process finishes.

With your settings chosen, it’s now time to review the operation and confirm your choices. The backup and restore processes both use the ocs-sr tool; after completing the settings through the Clonezilla GUI wizard, the tool shows you the ocs-sr command with the necessary parameters for achieving the same result from the command line. It also creates a temporary file containing the command, saved under the /tmp directory.

Now, sit back and wait for the backup to be completed. After it’s done and the server boots up, transfer the image directory from the physical server to the place you will use as the restoration source for the virtual machine.



Step 2. Preparing the virtual machine

From here on, we’ll be creating a virtual machine within VMware Server as the destination for system migration. Make sure that the host environment complies with the VMware Server and guest operating system requirements and limitations. Because the virtual machine will use the same processor as the host as a baseline, it requires that the host environment and the cloned physical server have compatible types of processors.

First, use the VMware New Virtual Machine Wizard to create the virtual machine. During the process, select the operating system version that matches the cloned one from the physical server. Also, you have to create a virtual disk equal to or larger in size than the partition where the original cloned system resides; Clonezilla does not support restoring an image from a larger hard disk or partition to a smaller one. During the Clonezilla Live restoration process, however, you are able to restore the image to a large hard disk according to the original disk layout.
Figure 5. Specifying the virtual machine’s disk capacity
Specifying virtual machine's disk capacity

Note: The Clonezilla Live kernel might not support the SCSI disk for earlier VMware Server versions. In this case, when you create the virtual machine, use the IDE type for the virtual disks.

Second, change the virtual CD-ROM device to use the Clonezilla Live ISO image for the virtual machine, as shown in Figure 6.
Figure 6. Using Clonezilla Live ISO image in VM
Using Clonezilla Live ISO image in VM

Third, put the system image files from the previous section under the host server’s second disk, Partition 0 of PhysicalDrive 1. Because VMware Server supports pass-through SCSI drive access on the host system, add another hard disk for the virtual machine with Partition 0 of PhysicalDrive 1 directly attached in VMware. At the restore phase, this device will be presented to /home/partimag.
Figure 7. Attaching the partition with system image
Attaching the partition with system image

Now that you have a compact virtual machine created, it’s time to boot the virtual machine from Clonezilla Live.



Step 3. Restoring the image onto the virtual machine

After the virtual machine boots up from the Clonezilla Live image, Clonezilla Live has the same GUI-based wizard interface for restoration as for backup until you choose the mode, as shown in Figure 8. In this example, we will enter the Debian-based Clonezilla Live shell directly to select a manual restoration.
Figure 8. Clonezilla Live shell
Clonezilla Live shell

To restore the image under the Clonezilla Live shell, you must log on as root, as in Listing 1; this gives you full access to Clonezilla.
Listing 1. Becoming root user

user@debian:~$ sudo su -
debian:~#

Now you need to determine the destination disk for restoration and the disk where the source image is to be placed. Listing 2 shows two local hard disks.
Listing 2. Viewing disk information

debian:~# fdisk -l
Disk /dev/sda: 42.9 GB, 42949672960 bytes
255 heads, 63 sectors/track, 5221 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0x00000000

Disk /dev/sda doesn't contain a valid partition table

Disk /dev/sdb: 160.0 GB, 160039272960 bytes
255 heads, 63 sectors/track, 19457 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Disk identifier: 0xa0bea0be

Device Boot         Start       End      Blocks       Id    System
/dev/sdb1            1            19457   156288321   7      HPFS/NTFS

/dev/sda is the virtual disk we created as the restoration destination; it is unformatted. /dev/sdb is PhysicalDrive 1 of the host server that we attached to the virtual machine directly in the form of a pass-through SCSI device; we have the cloned system image on it.

To restore the cloned partition image, the destination virtual disk has to be presented to Clonezilla as formatted. You can use the fdisk tool to write the label onto the virtual disk. This procedure is shown in Listing 3.
Listing 3. Formatting the destination disk

# fdisk /dev/sda
Device contains neither a valid DOS partition table, nor Sun, SGI or OSF disklabel
Building a new DOS disklabel with disk identifier 0xee2955bc.
Changes will remain in memory only, until you decide to write them.
After that, of course, the previous content won't be recoverable.

The number of cylinders for this disk is set to 5221.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
  (e.g., DOS FDISK, OS/2 FDISK)
Warning: invalid flag 0x0000 of partition table 4 will be corrected by w(rite)

Command (m for help):
Command action
  e   extended
  p   primary partition (1-4)
p
Partition number (1-4): 1
First cylinder (1-5221, default 1):
Using default value 1
Last cylinder or +size or +sizeM or +sizeK (1-5221, default 5221):
Using default value 5221

Command (m for help):
Command (m for help): w
The partition table has been altered!

Calling ioctl() to re-read partition table.
[ 866.679048] sd 0:0:0:0: [sda] 83886080 512-byte hardware sectors (42950 MB)
[ 866.682658] sd 0:0:0:0: [sda] Write Protect is off
[ 866.683795] sd 0:0:0:0: [sda] Cache data unavailable
[ 866.683822] sd 0:0:0:0: [sda] Assuming drive cache: write through
[ 866.686443]  sda: sda1
[ 866.695530] sd 0:0:0:0: [sda] 83886080 512-byte hardware sectors (42950 MB)
[ 866.698278] sd 0:0:0:0: [sda] Write Protect is off
[ 866.699422] sd 0:0:0:0: [sda] Cache data unavailable
[ 866.699495] sd 0:0:0:0: [sda] Assuming drive cache: write through
[ 868.702058]  sda: sda1
Syncing disks.

We can accept the default values in Listing 3. The partition type is not a concern at this time because it will be re-created by Clonezilla during the image restoration.

As with the backup procedure, you must mount a writable device or space as /home/partimag, then Clonezilla will search the cloned image directory under /home/partimag. To prepare the restoration from the image on the partition /dev/sdb1 to the newly created partition /dev/sda1, first mount /dev/sdb1 to /home/partimag with the writable privilege. Note that here, /dev/sdb1 is an NTFS file system on the host server, and it may be open, so a force option might be required to mount it successfully. See Listing 4.
Listing 4. Mounting the image device to /home/partimag

debian:~# mount -t ntfs-3g /dev/sdb1 /home/partimag -o force
debian:~# ls /home/partimag
Sys01-2009-02-23-img

When /home/partimag and the destination partition are ready, you have to choose the best way to restore the image for your environment. Just as for backup, Clonezilla provides a variety of advanced and flexible options for image restoration. See Figure 9 for the restoration parameters and their meanings.
Figure 9. Clonezilla advanced extra parameters: restore
Clonezilla advanced extra parameters: restore

For Windows system image restoration, the boot header has to be written to the destination disk. To achieve this, use -j0 (use dd to create partition table) and -t1 (client restores the prebuilt MBR from syslinux). The hnX PC (X is 0 or 1) option is good for ensuring that your environment has a unique Windows machine name. In case you don’t want to keep your current destination partition layout, you also can try -k1 or another partition-relation parameter to create another partition table.

Listing 5 restores the image Sys01-2009-02-23-img from /home/partimag (/dev/sdb1) to the destination device, /dev/sda1.
Listing 5. Using the command line to restore the image

debian:~# /opt/drbl/sbin/ocs-sr -e1 auto -c -t1 -r -j3 -cm -j0 -p reboot restoreparts
            "Sys01-2009-02-23-img" "sda1"

Figure 10 shows the summary and progress updates you’ll receive after you confirm the choice. After that, you have your cloned system on a virtual machine.
Figure 10. Clonezilla restore summary and progress
Clonezilla restore summary and progress

When the restoration is done, you can watch your system boot up from the virtual machine. To get better performance, install the VMware Tools for the virtual operating system.



Conclusion

This article has shown you how to complete a physical-to-virtual system migration using an image-based method and open source tools. Remember to use the steps here as a reference; due to differences in environments, your system and migration experience may be different from what you read here.

Original document by Ting Li, Software Engineer, IBM

Posted in Uncategorized | Tagged: | Leave a Comment »