Xen/Troubleshooting

From Gentoo Linux Wiki

Virtualization TOC

Virtualization Paravirtualization Xen DomU creation Running Xen Troubleshooting Links and resources KVM QEmu VirtualBox VMware VMware Server VMware Workstation Gentoo guest install MS VirtualPC addons Container-based virtualization LXC OpenVZ VLAN

It can be difficult to see what's going on as the system boots. The easiest way is to use another computer connected via the serial port. Another option is to use GNU screen.

Compile support for a serial console into your Xen kernel:

Device drivers --->
  Character devices --->
    Serial drivers --->
      [*] Console on 8250/16550 and compatible serial port

Add something like the following to your grub.conf:

title  Xen 2.0.8_pre20050826 / XenLinux 2.6.12.5
root   (hd0,0)
kernel /xen.gz dom0_mem=98304 noreboot com1=9600,8n1
module /vmlinuz-2.6.12.5-xen root=/dev/md2 noreboot console=ttyS0 debug

The noreboot option tells Xen not to allow the kernel to reboot, even if you do shutdown -r. The hypervisor option com1=9600,8n1 and the kernel option console=ttyS0 tell both to output boot messages to the serial port.

[edit] Xen says Linux dom0 is not a Xen-compatible Elf image

Verify that you have a xen_guest section in the resulting dom0 vmlinux.

[edit] Required Modules

On AMD64 I found that it was necessary to add 'loop' and 'tun' to modules.autoload.d or Xen/HVM networking would not work at all. Do this by manually adding these to /etc/modules.autoload.d/kernel-2.6 (or compile them into the kernel static instead of as modules)

[edit] Compiling Xen fails on kernel.c

If you have problems compiling xen (kernel.c fails to compile), try removing 387 support from your make.conf i.e remove 387 from the -mfpmath=sse,387 flag. invalid option `no-tls-direct-seg-refs'

If GCC does not know the 'no-tls-direct-seg-refs' flag (cc1: error: invalid option `no-tls-direct-seg-refs'), you will need to upgrade to a newer version (3.4 works) as described in the Gentoo GCC Upgrading Guide

[edit] gcc & '-fno-stack-protector-all' error

Xen 3.0.2 has some problems with Hardended USE Flag. If you get error of gcc because of '-fno-stack-protector-all' option try remove hardended flag.

[edit] Warning about /lib/tls on boot

If on booting the system you see a warning similar to the following, it means you're not using a recent enough version of glibc (you need to be using atleast version 2.4). This should be fixed by upgrading your system by following the Gentoo Upgrading Guide.

  ***************************************************************
  ***************************************************************
  ** WARNING: Currently emulating unsupported memory accesses  **
  **          in /lib/tls glibc libraries. The emulation is    **
  **          slow. To ensure full performance you should      **
  **          install a 'xen-friendly' (nosegneg) version of   **
  **          the library, or disable tls support by executing **
  **          the following as root:                           **
  **          mv /lib/tls /lib/tls.disabled                    **
  ** Offending process: init (pid=1)                           **
  ***************************************************************
  ***************************************************************

[edit] System Freezing on Reboot

If you experience your system freezing when rebooting, this could be due to conflicts between the Xen and gentoo network setups. To solve the problem, edit /etc/conf.d/rc and change the RC_NET_STRICT_CHECKING setting to 'lo'.

For possible further information, see the Gentoo forum thread

[edit] log flood when using dhcpcd-3.0.16 or 17

When using dhcpcd-3.0.16 or 17 as DHCP client for domU, you might notice your messages log being flooded. Also the messages log on the dhcp server will be flooded with messages every three seconds. Typical messages are:

Jun 27 21:10:03 prime dhcpcd[3762]: eth0: bad UDP checksum, ignoring
Jun 27 21:20:03 prime dhcpcd[3762]: eth0: bad UDP checksum, ignoring
Jun 27 21:30:00 prime dhcpcd[3762]: eth0: bad UDP checksum, ignoring
Jun 27 21:30:03 prime dhcpcd[3762]: eth0: bad UDP checksum, ignoring

Jun 27 21:23:55 lucifer dhcpd: DHCPREQUEST for 192.168.0.152 from 0a:70:72:69:6d:65 (prime) via eth0
Jun 27 21:23:55 lucifer dhcpd: DHCPACK on 192.168.0.152 to 0a:70:72:69:6d:65 (prime) via eth0
Jun 27 21:23:58 lucifer dhcpd: DHCPREQUEST for 192.168.0.152 from 0a:70:72:69:6d:65 (prime) via eth0
Jun 27 21:23:58 lucifer dhcpd: DHCPACK on 192.168.0.152 to 0a:70:72:69:6d:65 (prime) via eth0

--Jhendrix 20:10, 27 June 2007 (UTC)

[edit] Host (dom0) networking fails to start

The default Xen networking scripts are not particularly resilient to differing network configurations. A number of solutions were discussed on the forums (http://forums.gentoo.org/viewtopic-t-646301.html and http://forums.gentoo.org/viewtopic-p-4799819.html#479981). Manual configuration may often be completed (as picked from preceding links) with the commands similar to the following after the Xen daemon has started (mileage may vary):

This may be hacked into startup via /etc/conf.d/local.start, or alternatively, via the "more Gentoo configuration" detailed in the next section.

[edit] Dom0 network configuration (alternative solution)

A cleaner solution for the network in dom0. This approach uses the standard gentoo networking scripts.

First, configure networking in /etc/conf.d/net as usual:

bridge_xenbr0="eth0"
brctl_xenbr0=(
 "setfd 0"
 "stp off"
)

config_xenbr0=(
   "dhcp"
)

Once all the configurations are in place link into '/etc/init.d' as normal via the following command:

but DO NOT add the interfaces to startup via 'rc-update'. Instead, allow 'xend' to start the bridge interfaces via the following.

Move the original XEN networking script out of harms way:

and put this script in it's place

#!/bin/bash

#
# Gentoo Xen network start/stop script.
#

# xenfu (get command and set variables)
XENFU_cmd="$1" && shift
while [ $# -gt 0 ]
do
        eval "$1" && shift
done

# if the 'bridge' has not been set explicitly via the configuration
# file then bring up the first bridge we find in 'init.d'
if [ -z "${bridge}" ]
then
        # get a list of gentoo configured xen bridges
        GENFU_init_xenbr='/etc/init.d/net.xenbr[0-9]'
        # pull in our net configuration (not really neeeded)
        source /etc/conf.d/net
#       for each of the bridges, run the xen command
        for GENFU_cmd in $( echo ${GENFU_init_xenbr} )
        do
                # this is redundant but thought it would be nice for consistency
                netdev="bridge_${GENFU_cmd#net.}"
                if [ "${GENFU_cmd}" == "${GENFU_expr}" ]
                then
                        echo "xenbr: no configuration found" >&2
                        exit 1
                fi
                case "${XENFU_cmd}" in
                start)
                        ${GENFU_cmd} start
                        ;;
                stop)
                        ${GENFU_cmd} stop
                        ;;
                status)
                        echo -n "WARNING: gentoo status may not be" >&2
                        echo " compatible with expected output" >&2
                        ${GENFU_cmd} status
                        ;;
                *)
                        echo "Unknown command: $xenfu_cmd" >&2
                        echo 'Valid commands are: start, stop, status' >&2
                        exit 1
                esac
        #       quit once we've performed the command for the "default bridge"
        #       nb: comment this out to bring up all bridges
                exit 0
        done
else
# ... bring up the specified bridge
        /etc/init.d/net.${bridge} $XENFU_cmd
fi

or grab a copy as follows:

Reboot and interfaces should come up as expected. One difference to note, compared to the original 'network-bridge' script is that this does not rename the interfaces and so 'eth0' continues to be the physical interface.