KVM
From Gentoo Linux Wiki
|
Virtualization TOC | |
|
| |
KVM (for Kernel-based Virtual Machine) is a full virtualization solution for Linux on x86 or x86_64 (AKA amd64) hardware containing virtualization extensions (Intel VT or AMD-V). It consists of a loadable kernel module, kvm.ko, that provides the core virtualization infrastructure and a processor specific module, kvm-intel.ko or kvm-amd.ko which together provide for all that is needed to get virtualized CPU support.
KVM also requires a modified QEMU or QEMU 0.11 to provide for the remaining virtualized hardware (a virtual machine) including network, disk, and video adapters as well as block devices like hard drives, cdrom or floppies.
Be aware that not all Intel or AMD CPUs have virtualization extensions and some might need to have also a BIOS setting enabled in order to be able to use it. Refer to the KVM FAQ for more information.
This howto will explain how to install kvm on your Host machine, start the installation process of your Guest OS, configure the USB and network parts.
We will show a practical example of two Guest OS with network enabled and able to talk to each other and to the Internet.
[edit] Required packages
For this howto, we will need the following packages:
- app-emulation/qemu-kvm
- sys-apps/usbutils: needed for lsusb (see #USB_support)
- net-misc/bridge-utils: needed for brctl (see #Networking)
- sys-apps/usermode-utilities: needed for tunctl (see #Networking)
- net-firewall/iptables: needed for masquerade (see #Networking)
TODO:Add information about KSM (samepage merging) which is included in kernel 2.6.32, see also the [1] website
[edit] Kernel configuration
[edit] The virtual machine
To enable the KVM, you need the kvm virtual machine, and a driver. Most likely you will want to select the Intel or AMD specific driver depending on what CPU your host computer has, but if you want to run out-of-tree modules (USE flag modules dependent package app-emulation/kvm-kmod), you may get along without either.
| Linux Kernel Configuration: Enable KVM |
[*] Virtualization --->
--- Virtualization
<M> Kernel-based Virtual Machine (KVM) support
< > KVM for Intel processors support
< > KVM for AMD processors support
|
[edit] Networking
If you want your virtual machine to access your network, you will need to support bridging, the TUN/TAP interfaces and VLAN.
| Linux Kernel Configuration: Enable bridging, TUN and VLAN |
Device Drivers --->
[*] Network device support --->
<M> Universal TUN/TAP device driver support
Networking support --->
Networking options --->
<*> 802.1d Ethernet Bridging
<*> 802.1Q VLAN Support
|
You can now compile your kernel and install the modules:
Don't forget to copy your new kernel over, replacing x86 with your arch, and kernel-2.6.31-gentoo-r6 with your kernel name and version:
[edit] Installation
Until a version is stabilized, you will need to unmask qemu-kvm as described below if you are using "stable" or move to "unstable" and help stabilizing it.
[edit] Configuring your USE flags (version: qemu-kvm-0.12.2-r1)
The ebuild has several configurable options that can be used to adjust it to your environment better. For more information about how to set the USE flags refer to HOWTO Use Portage Correctly. By default qemu-kvm has convenient USE flags enabled by its ebuild.
[edit] alsa
Adds support for media-libs/alsa-lib (Advanced Linux Sound Architecture). This used to be enabled by default for "<app-emulation/kvm-47" but was turned off because the same functionality is available by using alsa through SDL instead.
[edit] esd
Adds support for media-sound/esound (Enlightened Sound Daemon)
[edit] gnutls
Adds support for net-libs/gnutls (TLS 1.0 and SSL 3.0 support). Since >=app-emulation/kvm-45, the VNC server option available from the userspace application with the -vnc option is able to use TLS to encrypt the session established with the vnc client, this flag enables support for that functionality. If disabled vnc sessions are still possible but only using unencrypted channels.
[edit] pulseaudio
Adds support for PulseAudio sound server.
[edit] sdl
Adds support for Simple Direct Layer (media library) If enabled (recommended), will use SDL to emulate a console, so you can interact with your virtual machine through an X session.
If you are only interested on using kvm to run "headless" virtual machines then be sure to start them with either an emulated serial console going into a file (or /dev/null) or a vnc server console and disable this flag.
[edit] vde
Enable VDE-based networking. VDE is a package that provides software versions of physical networking devices (e.g. switches, cables, etc). In the same way that KVM allows you to virtualize a computer, VDE allows you to virtualize your networking gear.
[edit] Getting Started
[edit] Adding access to /dev/kvm to your user
The /dev/kvm device is owned by root but with read/write priviledges for the kvm group, so in order to be able to access it you have to add your user to it:
[edit] Create a virtual disk image
To create a virtual disk image, use the kvm-img tool. The following example will create a 10G Copy On Write disk image:
[edit] Installing the guest OS to your image
[edit] Gentoo example
To install gentoo into the disk image using an livecd ISO:
You can look at the kvm man page to see that this command says:
- run a virtual machine with:
- an emulated hard disk drive using the file gentoo-i386.img
- an emulated CDROM drive containing the emulated CDROM image livecd-i686-installer-2007.0.iso
- and boot from device "d" (i.e., the emulated CDROM drive)
[edit] Windows example
As of 2.6.29 with 84/5 I am no longer able to start windows. I had to go down to 2.6.28. --Letharion
With kvm-78 and kernel-2.6.24, nothing special was required to install Windows XP. The following command was used:
I have successfully installed Windows Vista Ultimate with kvm-84 and 2.6.28 like this:
Without the -m 512 Vista complained about to little available RAM.
I have successfully installed Windows XP with kvm-83 and 2.6.27 like this:
Once installation is complete, and you no longer need the disc, the command simply becomes
This ignores the CD, and requests boot from harddrive, instead of cd-device.
[edit] Kubuntu example
The latest ubuntu/kubuntu variant "Jaunty Jackelope" may have trouble with monitor drivers other than the default cirrus. You will probably also need to override the default boot options on its initial boot screen with "noacpi" (hit f6 for that option) and possibly "safe graphics" (f4 options) before booting the install. It also appeared to hang on its initial splash screen with -soundhw ac97 to enable audio.
This command line sets 2gb of memory, a tuntap interface for network, a name for the sdl window (kubuntu) so that more than one kvm sdl session can run without interference and the alternative grab sequence "ctrl-alt-shift" instead of "ctrl-alt". This is kvm-0.84 on a 2.6.28-r5 amd64 kernel where the tuntap and bridge interface are already plumbed up from init.d and conf.d/net settings:
After the initial installation, experiments showed that the es1370 virtual driver would allow kubuntu to boot and use audio. However the cirrus driver still appeared to be the only monitor option that would work. Thus vm display resolutions will be limited to 1024x768 until software updates from Canonical permit better driver support.
[edit] Fedora 10 example
The latest RedHat Fedora variant (at least until May 2009) does not have trouble booting the installer with acpi enabled, enhanced vga or ac97 driver support. This guest lives on tap1 alongside the kubuntu guest above:
After the install, Fedora will initially have troubles with the adjustment of the graphic display and system fonts when living in a vm. This is because it is using the new approach to Xorg which relies on the monitor's EDID information to calculate available display resolutions. You will need to use yum to install system-config-display (a sore point among the Fedora community that it was left out of the default install) in order to create the missing xorg.conf file and populate it with sane settings. You will also probably need to set a specific font resolution in kde's "system-settings -> appearances" to override the bad choices made for you.
[edit] USB support
KVM allows you to add USB support to your VM by "bridging" what the kernel of your Host sees to the Guest OS.
To do so, you need to start kvm with the -usb flag on. Moreover, you will need to tell the Guest OS which usb device is being used for it.
kvm (or more adequately qemu), can pass the USB VendorID and ProductID to the Guest OS in order for it to know that there is a USB attached. To know what VendorID and ProductID your USB device is, use the lsusb utility:
... Bus 001 Device 006: ID: 08ec:2039 M-Systems Flash Disk Pioneers ...
In this example, the VendorID is "08ec" and the ProductID "2039".
To launch kvm with USB support and this flash disk, you can use the following command:
If you have more than one USB device you would like to support, just add another "-usbdevice host:<VendorID>:<ProductID>" to the command line.
[edit] Networking
There are a few ways to allow the Guest OS to talk with each other and the outside world. This howto will explain how to set up the networking with bridging and TUN interfaces. This will allow you to communicate between Guest OS and to the Internet.
[edit] Principle
The principle is the following: you need to create an interface that will be the bridge between all the TUN/TAP interfaces (one per Guest OS) and the ethernet one of your Host OS. From there, you will be able to forward the traffic (using iptables) to the Internet.
The configuration described here after can be seen as follow:
Direct bridging (so guests use an IP address on the same subnet as the host):
HOST
+---------------+
| | KVM GUEST1
| | +--------------+
| +------+ | | |
LAN ---+--- eth0 | +--+---+---- nic0 | KVM GUEST2
| | tap0----+ | |192.168.1.13 | +--------------+
| | tap1----+ | +--------------+ | |
| +------+ | | | |
| br0 +--+----------------------+---- nic0 |
|192.168.1.12 | |192.168.1.14 |
+---------------+ +--------------+
When use NAT/Masquerading (to hide the guests behind the host):
HOST
+---------------+
| 192.168.1.12 | KVM GUEST1
LAN ---+---- eth0 | +--------------+
| ^ | | |
| | | | |
| +------+ +--+---+---- nic0 | KVM GUEST2
| | tap0----+ | |192.168.100.1 | +--------------+
| | tap1----+ | +--------------+ | |
| +------+ | | | |
| br0 +--+----------------------+---- nic0 |
|192.168.100.254| |192.168.100.2 |
+---------------+ +--------------+
[edit] Enabling the access to Internet
The most transparent option to allow your guests access to the internet is the "virtual hub". In this scheme, the bridge connects eth0 and your tuntap interfaces together, routing packets as if it were a real "old fashioned" hub (not a switch). The key to this approach is to make sure you have unique mac addresses on both the host's tuntap interface as well as the guest. The guest ip addresses are typically in the same subnet as the host, and they can ask for and receive a dhcp lease from the same dhcp server that the host might use if it used dhcp. That is because all arp traffic and other broadcasts are passed through the bridge between the eth0 interface and the guest taps. The guests can use the same default gateway as the host because of this transparent passage.
The following snippet from an /etc/conf.d/net file shows the setup of a bridge between eth0 and two tap devices for guests. Note that the dependency for eth0 is left out of the br0 config since it is always started earlier on this particular system.
Direct bridging:
bridge_br0="eth0 tap0 tap1" brctl_br0=( "setfd 0" "sethello 0" "stp off" ) rc_need_br0="net.tap0 net.tap1" # # host system is a static address at 192.168.1.12 with dns server at 34 and a router at 33 # config_br0=( "192.168.1.12/24" ) routes_br0=( "default via 192.168.1.33" ) dns_domain_br0="example.com" dns_servers_br0="192.168.1.34" dns_search_br0="example.com" config_tap0=( "null" ) tuntap_tap0="tap" tunctl_tap0="-u joeuser" mac_tap0="52:54:00:12:34:56" config_tap1=( "null" ) tuntap_tap1="tap" tunctl_tap1="-u joeuser" mac_tap1="52:54:00:12:34:59" config_eth0=( "null" )
Using NAT/Masquerading:
bridge_br0="tap0 tap1" brctl_br0=( "setfd 0" "sethello 0" "stp off" ) rc_need_br0="net.tap0 net.tap1" # # host system is a static address at 192.168.1.12 with dns server at 34 and a router at 33 # config_eth0=( "192.168.1.12/24" ) routes_eth0=( "default via 192.168.1.33" ) dns_domain_eth0="example.com" dns_servers_eth0="192.168.1.34" dns_search_eth0="example.com" config_br0=( "192.168.100.254/24" ) config_tap0=( "null" ) tuntap_tap0="tap" tunctl_tap0="-u joeuser" mac_tap0="52:54:00:12:34:56" config_tap1=( "null" ) tuntap_tap1="tap" tunctl_tap1="-u joeuser" mac_tap1="52:54:00:12:34:59"
This is essentially the conf.d/net setup for the Fedora10 and kubuntu guest examples above. Note that the bridge br0 uses a different subnet otherwise the routing table will be ambiguous. Note also that their nic definitions on the kvm command line use different mac addresses than what is set for their taps. If the same mac address had been used on both sides, the arp queries for address resolution would not work. This conf.d/net setup is also why the kvm command line says not to do anything about interface startup or takedown.
You need to create the appropriate net.br0, net.tap0 and net.tap1 initscripts. You can use ln to do this:
You also need to add net.br0 to the default runlevel:
Finally you need to make some changes to /etc/sysctl.conf and add a new init-script.
The changes in /etc/sysctl.conf are to prevent the traffic from the guests to be sent to iptables to be filtered. If you want to filter the traffic from/to the guests, you can keep the file unchanged but you will have to add the correct rules to iptables.
The addition needed in /etc/sysctl.conf is ...
The new init-script is to allow br0 to forward the traffic from your guests. This can unfortunately not be added to the /etc/sysctl.conf because the br0 interface does not exist when those changes are applied. The best way I have found to fix this is to add a file called /etc/init.d/bridge_forward with the following content ...
The alternative to the virtual hub is to enable masquerade on your machine:
The disadvantage of this approach is that the initial inbound connections to your guests will not work without dnat rules. This can become quite involved and generally will get you to look at using shorewall for the management.
[edit] Start your Guest OS
This section will show how to launch two Guest OS with kvm and the setting up. If you need the USB support, just append the USB options as shown earlier to the commands here after.
[edit] Launch KVM Guest1 with network enabled
Now that the requirements are included in the kernel or the modules compiled, we need to load the modules:
Let us create the bridge interface br0 and bring it up:
The next step is to create the TAP interface:
Now that we have the TAP interface and the bridge one, we need to link them:
And finally, we bring qtap0 up with the "promisc" mode:
Ok, you should now have br0 and qtap0 appearing with ifconfig:
br0 Link encap:Ethernet HWaddr 00:ff:18:5a:15:57
inet addr:192.168.100.254 Bcast:192.168.100.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:81 errors:0 dropped:0 overruns:0 frame:0
TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:6469 (6.3 KiB) TX bytes:410 (410.0 B)
eth0 Link encap:Ethernet HWaddr <your_hw_address>
inet addr:192.168.1.5 Bcast:0.0.0.0 Mask:255.255.255.255
UP BROADCAST MULTICAST MTU:1500 Metric:1
RX packets:0 errors:0 dropped:0 overruns:0 frame:0
TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:0 (0.0 B) TX bytes:0 (0.0 B)
Interrupt:17
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:117562 errors:0 dropped:0 overruns:0 frame:0
TX packets:117562 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:8250309 (7.8 MiB) TX bytes:8250309 (7.8 MiB)
qtap0 Link encap:Ethernet HWaddr 00:ff:18:5a:15:57
UP BROADCAST RUNNING PROMISC MULTICAST MTU:1500 Metric:1
RX packets:81 errors:0 dropped:0 overruns:0 frame:0
TX packets:6 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:500
RX bytes:7669 (7.4 KiB) TX bytes:410 (410.0 B)
Now, you can boot your VM with the following parameters:
The two "-net" switches have a different meaning but are both needed:
- nic,macaddr=52:54:00:12:34:56: we specify the options related to the network interface of the Guest OS. Please note that the macaddr needs to be specified
- tap,ifname=qtap0,script=no: we specify how kvm should work with the network on the Host side. The "script=no" means we will not use the given scripts in /etc/kvm/kvm.ifup as we already tuned everything before.
Do not forget to configure KVM Guest1 (192.168.100.1) with the br0 interface (192.168.100.254) as the default gateway. (And you probably also wanna take the DNS-settings from /etc/resolv.conf --Letharion)
[edit] Launch KVM Guest2 with network enabled
What needs to be done as we already loaded the modules and the bridge interface, we just need to create the TAP/TUN interface and link it to br0:
Now, you can boot your second VM with the following parameters:
Here, the first Guest OS has 52:54:00:12:34:56 and the second 52:54:00:12:34:57.
Do not forget to configure KVM Guest2 (192.168.100.2) with the br0 interface (192.168.100.254) as the default gateway.
Now, your two Guest OS should be able to ping each other.
[edit] Script to ease the configuration
If you want to ease the process of configuring your machine (load the modules, create the bridge interface, the TUN/TAP, ...), here is an init.d script:
#!/sbin/runscript
# Copyright 1999-2008 Gentoo Foundation
# Distributed under the terms of the GNU General Public License v2
# $Header: $
NUM_OF_DEVICES=5
USERID="<your_user>"
depend() {
need net
}
start() {
ebegin "Loading the kvm module"
/sbin/modprobe kvm
eend $? "Failed to load the kvm module"
ebegin "Loading the kvm_intel module"
/sbin/modprobe kvm_intel
eend $? "Failed to load the kvm_intel module"
ebegin "Loading the tun module"
/sbin/modprobe tun
eend $? "Failed to load the tun module"
ebegin "Setting up the bridge device (br0)"
/sbin/brctl addbr br0
/sbin/ifconfig br0 192.168.100.254 netmask 255.255.255.0 up
eend $? "Failed to create the bridge interface"
for ((i=0; i < NUM_OF_DEVICES; i++)); do
ebegin "Setting up the tap interface: qtap$i"
/usr/bin/tunctl -b -u $USERID -t qtap$i >/dev/null
eend $? "Failed to create the tap interface: qtap$i"
ebegin "Linking the bridge interface with qtap$i"
/sbin/brctl addif br0 qtap$i
eend $? "Failed to link the bridge interface to qtap$i"
ebegin "Bring qtap$i interface up"
/sbin/ifconfig qtap$i up 0.0.0.0 promisc
eend $? "Failed to bring qtap$i up"
done
ebegin "Allowing Internet access"
echo "1" > /proc/sys/net/ipv4/ip_forward
eend $? "Failed to allow forwarding"
iptables -t nat -A POSTROUTING -o eth0 -j MASQUERADE
eend $? "Failed to allow masquerade (eth0)"
eend 0
}
stop() {
for ((i=0; i < NUM_OF_DEVICES; i++)); do
ebegin "Bring qtap$i interface down"
/sbin/ifconfig qtap$i down
eend $? "Failed to bring qtap$i down"
ebegin "Unlinking the bridge interface with qtap$i"
/sbin/brctl delif br0 qtap$i
eend $? "Failed to unlink the bridge interface to qtap$i"
ebegin "Removing the tap interface: qtap$i"
/usr/bin/tunctl -d qtap$i >/dev/null
eend $? "Failed to create the tap interface: qtap$i"
done
ebegin "Stopping the bridge device (br0)"
/sbin/ifconfig br0 down
/sbin/brctl delbr br0
eend $? "Failed to stop the bridge interface"
ebegin "Unloading the tun module"
/sbin/modprobe -r tun
eend $? "Failed to unload the tun module"
ebegin "Unloading the kvm_intel module"
/sbin/modprobe -r kvm_intel
eend $? "Failed to unload the kvm_intel module"
ebegin "Unloading the kvm module"
/sbin/modprobe -r kvm
eend $? "Failed to unload the kvm module"
ebegin "Stopping Internet access"
echo "0" > /proc/sys/net/ipv4/ip_forward
eend $? "Failed to cancel forwarding"
iptables -t nat -D POSTROUTING -o eth0 -j MASQUERADE
eend $? "Failed to remove masquerade (eth0)"
eend 0
}
restart() {
stop
start
}
[edit] VDE Networking
An alternative way of networking all your virtual machines is to connect them to a virtual switch. This method is slightly easier to setup as it does not require any TUN or bridging support.
Firstly you must make sure you compiled KVM with the vde USE flag. You can then start the virtual switch by running
Now when you launch your guest OS you can tell them to automatically connect to the virtual switch
Your guests will now be able to communicate with each other.
If you want your guests to be able to communicate with the outside world (e.g. your LAN or the internet) then you need to connect your physical interface (e.g. eth0) to the virtual switch as well. This can be done by running
(Note that vde_pcapplug is only available when you are using net-misc/vde-2.2.3 (currently masked) or above with the pcap USE flag. One drawback with this vde_pcapplug approach is that the host and guest can't talk to each other directly due to limitations in linux kernel)
All incoming traffic on eth0 will now be copied to the virtual switch, and all traffic from the virtual switch will be sent out over eth0. This means your guests will appear as part of your physical LAN.
[edit] Troubleshooting
[edit] Modprobe kvm-amd gives general protection fault
If CONFIG_KVM=n, kvm-kmod cannot be built, while if CONFIG_KVM=y, there is a possible conflict between in-kernel kvm and out-of-tree modules. Personally, I get general protection fault trying
So, it should be CONFIG_KVM=m.
[edit] My x86 Gentoo Guest dies with a kernel panic
KVM doesn't emulate the MMR registers needed for performance metrics and therefore it will generate a General Protection Fault if they were used. A workaround was added to the Linux kernel 2.6.22.5 and therefore upgrading the kernel in your guest will fix this problem. If you can't upgrade your kernel try using the nolapic or nmi_watchdog=0 boot parameters instead.
[edit] But I can't boot my guest anymore
If you are using the USE-Flag qemu then you can use qemu + kqemu instead to try to fix any problems with the guest that affected kvm, if that doesn't work try using kvm -no-kvm or qemu -no-kqemu as a last resort.
[edit] modprobe kvm modules error
| Linux Kernel Configuration: KVM modules error |
Processor type and features ---> Preemption Model (Voluntary Kernel Preemption (Desktop)) |
must not been set as No Forced Preemption (Server)
When you use AMD CPU and get such an output error in:... kvm: Unknown symbol intel_iommu_domain_alloc kvm: Unknown symbol intel_iommu_detach_dev kvm: Unknown symbol intel_iommu_page_mapping kvm: Unknown symbol intel_iommu_context_mapping kvm: Unknown symbol intel_iommu_iova_to_pfn kvm: Unknown symbol intel_iommu_domain_exit ...
then turn off the experimental modules could help:
| Linux Kernel Configuration: KVM experimental modules |
PCI driver for virtio devices (EXPERIMENTAL) Virtio balloon driver (EXPERIMENTAL) |
What about when you're using an Intel CPU, these drivers are off, and you STILL get these same errors?? Or is kvm-81 just broken?
A bug on KVM's tracker indicated that CONFIG_DMAR may be to blame. ( http://sourceforge.net/tracker/?func=detail&atid=893831&aid=2405145&group_id=180599 )
[edit] Can I boot an Operating System (Windows, Linux, etc) residing on a partition instead of an image?
The key here is you must specify the hard disk (or lvm) (e.g. /dev/sda) that the OS resides on and not the specific root partition (e.g. /dev/sda3). If you boot multiple operating systems through grub normally, you should be greeted with a grub screen where you can select which OS you would like to boot.
