Build Your Own LiveCD or LiveDVD

From Gentoo Linux Wiki

[edit] Introduction

This article will show you how to create your own LiveCD. This method has the following advantages over Gentoo's own Catalyst tool:

• The build source will not be deleted between iterations of LiveCD creation. This will allow you to periodically sync, update, merge and customize your environment the same way you do it with a real system.
• The CD will boot using GRUB, not ISOLINUX, though ISOLINUX is still possible. This gives you the same flexibility you have with a real system, such as changing kernel parameters, discovering devices, et cetera.
• Keeping your build source between iterations reduces the subsequent build times enormously.
• If made correctly, the CD can boot on 32MB RAM.
• Things that are in a green box are scripts that will be used more than once, like build scripts. Those are likely to get updated once in a while too.

[edit] Requirements

You need sufficient disk space on your system. This will vary depending on which packages you include in your LiveCD (more packages increases the size). Typically 4GB free is sufficient.

You must be familiar with installing Gentoo using a stage3 tarball. If in doubt, please consult the Gentoo Handbook.

A healthy understanding of the workings of Gentoo and Linux in general is recommended. Things can get especially nasty with the new build script if you get it wrong.

You'll also need app-cdr/cdrtools and sys-fs/squashfs-tools installed to build properly.

[edit] Setting up the build environment

The build environment is a directory that will be "chrooted" into when the LiveCD is being created, much like a normal installation is from a LiveCD. The build environment will be inside another directory containing everything.

For convenience, the main directory can be assigned to a shell variable, such as LIVECD, with the build environment as the "source" directory within it.

Download the latest stage3 tarball from one of the Gentoo mirrors to your home directory and untar it in the source directory. The file will be located in /releases/<architecture>/autobuilds/current-stage3/stage3-<architecture>-<release date>.tar.bz2

Next, download the latest portage snapshot to your home directory and untar it in ${LIVECD}/source/usr.

If you need to download packages during the installation, setup the necessary files in ${LIVECD}/source/etc that are required to connect to the internet, such as resolv.conf.

[edit] Installing Gentoo

Set up necessary mount points before chrooting to your build directory.

To avoid re-downloading distfiles for the installation, bind the local distfiles directory to our chroot's. In this case it is assumed that distfiles are kept in the default directory /usr/portage/distfiles.

Now chroot to source and start the installation.

One of the first things to do is modify /etc/make.conf. A few notes on USE variables:

• The more you use, the bigger your environment will get. It might even become too big to fit on a CD!
• Consider some negatives to save space if you want to avoid pulling in extra dependencies. Flags such as -X and -doc are probably essential (unless you really want to include a GUI or man pages).
• Make sure you add livecd to your USE variables. This, along with a 'cdroot' parameter that we'll pass to the kernel at boot time, will make init scripts aware that we're booting from a CD so they don't do inappropriate things like checking the root filesystem.
• Consider adding the minimal and userlocales USE flags (don't forget to edit /etc/locales.build). If you employ the USE variable 'minimal', you will need to add an exemption for the perl package:
  echo "dev-lang/perl -minimal" >>/etc/portage/package.use
  The command above is of course for use *inside* the chroot.

Here are the installation steps roughly:

1. emerge -e system # This will take quite a while!
2. emerge -e world # If part of the systems were updated, like GCC, re-emerge the whole system and check http://www.gentoo.org/doc/en/gcc-upgrading.xml
3. ln -sf /usr/share/zoneinfo/file> /etc/localtime
4. emerge memtest86+ localepurge genkernel gentoolkit dmraid livecd-tools # (NOTE: sys-fs/fuse has been built into the kernel as of 2.6.16)
5. emerge #<required packages, as you like, e.g. logger udev eix gentoolkit>
6. rc-update add <default boot, as fits>
7. emerge scripts. You might need to add them to /etc/portage/package.keywords>
8. emerge mingetty # <Required for autologin>
9. emerge # <optional packages, as you like, e.g. kde, mc>

Before merging the kernel and boot loader, important files in /etc must be fixed first. First, /etc/fstab:

/dev/loop0              /               squashfs        defaults             0 0
none                    /proc           proc            defaults             0 0
none                    /dev/shm        tmpfs           defaults             0 0

Notice that the root is being mounted over a loop device with filesystem type squashfs. More on that later. Now, modify other /etc files as you see fit (e.g. hostname, rc.conf, conf.d/*, and so on. It is assumed that, since you have set up a Gentoo system before, you know how to do this.

It is for example a good idea to change the keyboard layout one time : edit /etc/conf.d/keymaps. For french keyboard including € character support, change to the following layout :

KEYMAP="fr-latin9"
SET_WINDOWKEYS="yes"

If you want network support, edit /etc/conf.d/net. This example sets dhcp configuration for eth0:

Notice that dhcpcd must be merged.

[edit] Kernel and initrd

Then it's time to install the kernel sources and configure them.

If you use the vanilla sources you should use >=2.6.29 kernel sources, otherwise you have to patch them with the squashfs-patch from http://squashfs.sourceforge.net.

Note however that the above target enables CONFIG_CMDLINE_BOOL in conjunction with CONFIG_CMDLINE_OVERRIDE which means that bootloader's kernel parameters will be ignored in favor of those specified in CONFIG_CMDLINE, which is empty by default. In general, it would be a good idea to review the generated kernel .config for conflicts and issues like this.

Now merge your desired kernel. We are using gentoo-sources(2.6). As we are going to use squashfs as our root filesystem, be sure to patch your kernel with it if you are not using a standard gentoo-sources / vanilla >=2.6.29.

When configuring your kernel (see below, using genkernel --menuconfig) , make sure the following is compiled in:

General setup  --->
  <*> Initial RAM filesystem and RAM disk (initramfs/initrd) support
File systems  --->
  <*> Ext3 journalling file system support
  <*> Reiserfs support
  <*> JFS filesystem support
  <*> XFS filesystem support
  Miscellaneous filesystems  --->
    <*> SquashFS 4.0 - Squashed file system support
  CD-ROM/DVD Filesystems  --->
    <*> ISO 9660 CDROM file system support
  Pseudo filesystems  --->
    [*] Virtual memory file system support (former shm fs)
Device Drivers  --->
  Block devices  --->
    <*> Loopback device support
    <*> RAM disk support
    (16)  Default number of RAM disks
    (9000) Default RAM disk size (kbytes)
  ATA/ATAPI/MFM/RLL support  --->
    <*>  Include IDE/ATAPI CDROM support

If your planning boot through usb, such as on a flash drive, you´ll need to compile the kernel with the USB Mass Storage support:

Device Drivers --->
  USB support --->
    <*> USB Mass Storage support
      [ ] USB Mass Storage verbose debug
      [*] Datafab Compact Flash Reader support (EXPERIMENTAL)
      [*] Freecom USB/ATAPI Bridge support
      [*] ISD-200 USB/ATA Bridge support
      [*] Microtech/ZiO! CompactFlash/SmartMedia support
      [ ] USBAT/USBAT02-based storage support (EXPERIMENTAL)
      [*] SanDisk SDDR-09 (and other SmartMedia) support (EXPERIMENTAL)
      [*] SanDisk SDDR-55 SmartMedia support (EXPERIMENTAL)
      [*] Lexar Jumpshot Compact Flash Reader (EXPERIMENTAL)
      [ ] Olympus MAUSB-10/Fuji DPC-R1 support (EXPERIMENTAL)

IEEE 1394 ethernet support may disturb your system : sometimes an ethernet interface is recognized first, that is to say before "Classic" ethernet interface. If you don't need IEEE ethernet support, don't activate it :

Device Drivers  --->
  IEEE 1394 (FireWire) support  --->
   < >   Ethernet over 1394

Note: If you want to let make mkxf86config.sh an usable xorg.conf at startup, you have to build usbmouse ps2mouse etc as modules (and not compiled in!) since the script is searching for them.

Optionally, you can add any convenient option you use. For example, Magic SysRq Keys, allows rebooting instantly when no file system is mounted (or you might get corrupted filesystems). Linux Kernel Configuration: kernel configuration3

Kernel hacking ->
  [*] Magic SysRq Key

We wanted something like the gentoo 2005.0 as it is much more flexible, looks 1000x better, and boots more systems. After some research, found that it was genkernel that makes such an initrd possible. (believe me, using the initrd from the 2005.0 cd is not possible today).

The old can still be used, but is not recommended. Also, this howto is now based on genkernel.

Otherwise if you have not compiled kernel, do the following:

Or leave the menuconfig bit if you have a working .config.

Then don't forget to copy/rename the file to reflect the grub configuration (remember that as stated before, iso9660 doesn't handle symbolic links). For example, if you are working with a 2.6.15-gentoo-r1 type :

We now have a suitable kernel and ramdisk.

[edit] Cleaning some files

It is a good idea to clean the kernel tree, keeping only ".config" file:

Remove all the locales you don't need. The way to do this is to emerge the localepurge package:

And specify which locales you wish to keep in /etc/locale.nopurge:

 MANDELETE
 SHOWFREEDSPACE
 VERBOSE
 en
 en_US.UTF-8
# and whatever other locales you wish to save

Also, make sure to remove the NEEDSCONFIGFIRST line from the top.

Then run localepurge and re-index:

Delete wrong symbolic links :

Finally, delete ".keep" files that are necessary to restore empty directories from stage2 and portage TAR archives :

[edit] Using "locate"

Locate is a convenient and fast tool to know if some file is present on your system. If you want to use it, you have to build the file database:

cat /proc/mounts is needed for the updatedb command. If not done you get :

Then you'll be able to search file using locate :

[edit] Installing and configuring the CD bootloader

The final step is merging and configuring the boot loader. We're going to use GRUB since it has support for booting a cdrom.

default 0
timeout 7
splashimage=/boot/grub/splash.xpm.gz

title=LiveCD
        kernel /boot/vmlinuz real_root=/dev/loop0 looptype=squashfs loop=/livecd.squashfs vga=ask initrd udev nodevfs cdroot dodmraid 
        initrd /boot/initrd

title=LiveCD NO-FB
        kernel /boot/vmlinuz real_root=/dev/loop0 looptype=squashfs loop=/livecd.squashfs initrd udev nodevfs cdroot dodmraid
        initrd /boot/initrd

title=Memtest86+
        kernel /boot/memtest86plus/memtest.bin

Also, delete the symlink /boot/grub/menu.lst and then move grub.conf to menu.lst. This is because iso9660 can not handle symlinks. If you don't grub will just give a dumb commandline!

[edit] Final cleanings

Inside the chrooted environment, don't forget to remove old kernel modules. For example :

29M     2.6.16-gentoo-r12
29M     2.6.16-gentoo-r13
29M     2.6.17-gentoo-r4
28M     2.6.17-gentoo-r7
28M     2.6.17-gentoo-r8

Here we can remove 4 obsolete modules directories (delete all directories except the kernel(s) the livecd shall be booting):

Alternative (?) : we can also use emerge to remove old source trees:

Then we can leave the chrooted environment:

Make a copy of our root sourcetree. This way we will be able to make more aggressive deletions on the copy without harming the original which can be useful for updates

If you plan to use SYSLINUX, also do this:

Copy the source tree. It will not make its way onto the image, so don't worry about the size.

The following files can be skipped. Either by adding --exclude arguments to rsync, or running rm commands afterwards:

• var/run/* (but not the directory itself)
• var/tmp/* (but not the directory itself)
• var/cache/* (but not the directory itself)
• var/log/* (but not the directory itself)
• usr/portage
• etc/portage
• usr/share/doc
• usr/src
• root/* (but not the directory itself)
• tmp/* (but not the directory itself)

[edit] More aggressive cleanings

The goal here is to remove all software that ain't mandatory. For example portage, automake and gcc will be useless on the Live CD.

Inside the chroot do :

This will provide a complete list of the desired packages.

Outside the chroot and after building the target execute a script like this :

#!/bin/bash
cd /root/livecd
for i in `cat source/root/USELESSFILELIST`
do
if [ -f target/files/source/${i} ]
then
rm target/files/source/${i} ;
fi
if [ -L target/files/source/${i} ]
then
rm target/files/source/${i}
fi
if [ -d target/files/source/${i} ]
then
rmdir target/files/source/${i}
fi
done

This will remove files, symbolic links, and empty directories (as stated, rmdir will only remove empty directories)

This saved me 80Mb before compression just removing gcc, portage, automake, rpm2targz, gentoolkit and texinfo.

devsk: Are the static libraries needed on the livecd? I saved about 85MB by doing

find ${LIVECD}/target/files/source/usr/lib -name "*.a" -print -exec rm {} \;

Better yet, you can put "*.a" in the rsync --exclude command line.

[edit] Building the LiveCD

First, using squashfs tools, convert the source directory to a squashed image. Install the sys-fs/squashfs-tools package first if you don't have it.

Next, create the required empty livecd file. This file must be on livecd root, because the init script in initramfs uses this file to identify if the CD is mounted or not.

Finally, make the ISO image.

Now you can :

1. Test livecd.iso using qemu; see section Testing the LiveCD without wasting a blank CD
2. Burn livecd.iso

FINISHED :D

[edit] Updating the CD

You can chroot again to the CD tree and make some updates :

Now you can go back to the step Cleaning some files and follow the step to create a new ISO image.

[edit] Tips

Mounting /dev is important to get things like X to work from a chroot environment.

Another thing to consider is the size of your build source. As you can see, there're things that take a lot of space that are strictly not needed on a LiveCD. Two major candidates are /usr/src and /usr/portage. If you don't want to include them on your LiveCD don't delete them, you can utilize an option in mksquashfs to exclude any directory from the squashed image. This will keep your build source directory intact.

If booting the CD fails at some point during linuxrc script execution, you can always change your grub line to init=/bin/sh instead of linuxrc. This will give you a nice little shell inside the initrd image at boot time. From there you can type the commands in the linuxrc script manually and try to find out with one is failing and why.

I hope I didn't miss anything, and hope you find this howto useful. Your corrections/feedback are always welcome.

It is not necessary to bootstrap the system; the cd wil be able to boot from non-bootstrapped or bootsrapped systems.

[edit] Generate Xorg.conf at startup

emerge -av mkxf86config hwdata-gentoo hwsetup rc-update add mkxf86config default

Note: mkxf86config realy depends on hwsetup, so you need to edit /etc/init.d/mkxf86config and add /usr/sbin/hwsetup before /usr/sbin/mkxf86config.sh . It is also recomended to use newest versions of mkxf86config and hwsetup (currently lastest is mkxf86config 0.9.8 and hwsetup 1.2, but you need to write ebuild manualy). If you want mkxf86config to detect hardware properly - you need to edit /etc/init.d/mkxf86config or it will not detect your hardware. Also it is useful to look through /usr/sbin/mkxf86config.sh... If you are building LiveCD for x86 compatible PCs - you can uncomment ddcxinfo-knoppix and emerge this tool (ddcxinfo-knoppix - it will automaticlly probe monitor for information, but it may have some troubles with old and non-x86 hardware

Comment-alchark: hwsetup will actually be called by autoconfig script from the livecd-tools package, and it will also call mkxf86config as required.

emerge -av livecd-tools rc-update add autoconfig default

--end comment

[edit] Auto Mounting partitions spiced up

I have made an automounter script that looks up which partitions are available and then tries to mount them. But it is not a stupid loop.

1. find all available partitions 2. skip unmountables such as extended and grep error's 3. use swap as swap instead of mounting 4. mount ntfs with captive-ntfs 5. if none of the above, just try to mount

Update: Use ntfs-3g to avoid licensing issues and have much faster/better ntfs support.

I do this by greping fdisk -l, and using a second loop in my loop (because of the * that fdisk puts in front of the bootable partitions).

Getting captive-ntfs to run from the live-cd is a bit of a problem most of the time. It requires the screen program, and the fuse kernel module (in the before-build script it is (re)compiled so we do not forget it). Also it needs the ntfs.sys etc, you can deliver them preinstalled on the live-cd but I do not know if this is legal or not. But it is best to edit it a bit to autodownload them when they are needed and only start this script when the user requests it when you do not want any legal problems.

I have put my script online here, do not edit it using notepad or wordpad because they put a ^M at the end of each line which will break your livecd completely!

I could use some help perfecting the 2>/dev/null. It still displays some output sometimes.

   * Try &>/dev/null, this way both STDOUT and STDERR are sent to nirvana. LX

   * Maybe this script will help to detect partition types more accurate, 

it's taken from Knoppix and changed, there is another but a little worse tool (may miss fstype for example on promise fasttrack and other raid kontrollers): guessfstype from SuSE [mf]

[edit] Auto Login

If you want root to be automatically logged in, there's nothing to do, because usually the inittab is fixed during the boot in order to autologin root on a LiveCD.

However, many LiveCDs are intended to show Linux novices the principles of a Linux system, so it might be undesired to autologin as root, which is a large security threat. So if you want to autologin as non-privileged user, here's how to do it.

First emerge mingetty. Then edit /etc/inittab on the cd (be sure to make a backup), find:

replace with

for every tty you want autologon for a user named "joe".

Then you need to manipulate /sbin/rc in order to keep it from rebuilding the inittab. This has become somewhat more difficult to do in openrc and baselayout2 but here is how to do it;

Extract ex /usr/portage/distfiles/openrc-0.2.2.tar.bz2 to a test location, then edit;

if [ -f /sbin/livecd-functions.sh -a -n "${CDBOOT}" ]; then
       ebegin "Updating inittab"
       livecd_fix_inittab
       eend $?
       telinit q &>/dev/null
fi

replace with

if [ -f /sbin/livecd-functions.sh -a -n "${CDBOOT}" ]; then
       ebegin "Updating inittab"
       /bin/true # livecd_fix_inittab
       eend $?
       /bin/true # telinit q &>/dev/null
fi

If you want to display an extra message next to "OpenRC 0.2.2 is starting up Gentoo-Linux(i686)" now is the time to edit openrc-0.2.2/src/rc/rc.c;

After this we need to get this back to the tar.bz2 and get it verified

(this makes the ~ display after automatic login, instead of /root/)

Note: if you would just like to disable autologin for all users, simply make the changes described above (for the init.sh.Linux.in file) to /sbin/rc, without making the changes to /etc/inittab. When the LiveCD finishes booting, you'll then see the normal login prompt. Make sure that you have previously set password for root with passwd, or you'll be unable to login.

[edit] No Auto Login

To enable login for something like Mini Terminal you can also edit /sbin/livecd-functions.sh in your /home/root/livecd environment:

echo "c${x}:12345:respawn:/sbin/agetty -nl /bin/bashlogin 38400 tty${x} linux" >> /etc/inittab
# change to
echo "c${x}:12345:respawn:/sbin/agetty 38400 tty${x} linux" >> /etc/inittab


echo "s0:12345:respawn:/sbin/agetty -nl /bin/bashlogin ${LIVECD_CONSOLE_BAUD} ${LIVECD_CONSOLE} vt100" >> /etc/inittab
# change to
echo "s0:12345:respawn:/sbin/agetty ${LIVECD_CONSOLE_BAUD} ${LIVECD_CONSOLE} vt100" >> /etc/inittab

[edit] Optimizing the booting process

Note: No optimization is required if app-misc/livecd-tools is installed (in the chroot environment). That package provides scripts to detect whether a live CD is running and to disable unneeded instructions like the ones described below.

I have removed these optimizations since they were obsolete for some time and the new baselayout2 and openrc made them completly obsolete.

Another neat trick: edit /etc/rc:

Change rc_parallel="no" to rc_parallel="yes"

This will start all the services at the same moment, not one after another. Some people say this has no effect yet, but it looks cool and maybe there will be a time it has a purpose. But when using autoconfig from the livecd tools it will break your boot sequence.

[edit] "Defragmenting" the LiveCD to make it faster

By default, the files inside the squashfs filesystem are stored continuously, but unordered: this means that the cdrom drive will potentially have to seek them at very distant disk offsets, leading to higher cold start read times -- this is especially apparent at boot time if the LiveCD contains desktop environments such as GNOME or KDE, or applications such as Firefox or GIMP that use large number of files.

Fortunately, mksquashfs comes with a -sort option that makes it possible to change files' order on the filesystem. The easiest way to build the needed sort file is to include openlog in your LiveCD kernel image, then run the resulting LiveCD. openlog makes the kernel able to timely log all the successfully opened files; it also includes a script, openlog.py, that makes it easy to reuse this information to create the sort file. openlog comes fully documented: see the README or README.html file provided with it.

Note: I have moved the openlog page to my own host since I have been unable to contact the author, also I have added a new patch file for newer kernel versions. The orriginal site: [2]

The quick and dirty howto:

Boot the livecd with logfiles=1 and get dmesg from it to your build environment and extract openlog.py as well. File: dmesg

And change your build script to let the ordering take effect in the next build {{File|build| mksquashfs source ${TARGET}/rescuecd.squashfs -noappend -sort /<path-to-store-order-results>/load.order.new

[edit] Checking/editing the Initramfs

If you want to edit or check the initramfs generated, you can do something like this.

or

-- Thanx to robbat2

Initrd:

[edit] USB Stick drive

(evildick 2005-08-09)

#!/bin/bash
LIVECD=~/livecd
TARGET=${LIVECD}/target
TARGET_SOURCE=${TARGET}/files/source/
SOURCE=${LIVECD}/source

#USB settings
USBDEV=/dev/sda
MNTDIR=/mnt/usbstick

#USB
echo "Warning\!  You are about to reformat your usb device on ${USBDEV}\!"
echo "The following partitions will be overwritten:"
sfdisk -l ${USBDEV}
echo "Are you certain that this is what you intend?"
echo "Press enter to continue or CTRL-C to quit."
read user_reply
echo "Please wait, preparing usb device . . ."
# if you want to completely wipe the USB drive, issue the following command :
# dd if=/dev/zero of=${USBDEV}
sfdisk -q ${USBDEV} << EOF
0,,L,*
;
;
;
EOF
mke2fs -m0 ${USBDEV}1
mkdir ${MNTDIR}
mount ${USBDEV}1 ${MNTDIR}

echo "Copying files to USB device . . ."
cp -p -R -P -d ${TARGET}/boot ${TARGET}/livecd.squashfs ${MNTDIR}

echo "Modifying ${MNTDEV}/boot/grub/menu.lst . . ."
sed 's/(cd)/(hd0,0)/g' ${MNTDIR}/boot/grub/menu.lst > ${MNTDIR}/boot/grub/menu.lst-tmp && mv -f ${MNTDIR}/boot/grub/menu.lst-tmp ${MNTDIR}/boot/grub/menu.lst
# This may need to be changed to replace the cdroot with something better, like usbroot?
#sed 's/cdroot//g' ${MNTDIR}/boot/grub/menu.lst > ${MNTDIR}/boot/grub/menu.lst-tmp && mv ${MNTDIR}/boot/grub/menu.lst-tmp ${MNTDIR}/boot/grub/menu.lst
echo "Installing Grub . . ."
grub-install --root-directory=${MNTDIR} ${USBDEV} --recheck

echo "Unmounting ${MNTDIR} . . ."
umount ${MNTDIR}

echo "Your bootable USB drive should be complete."

• For the most part this works. Problems may be experienced due to bios limitations. Problems can range but usually it will just lock up shortly around initialization of the kernel (during or on the initramfs). This so far only occurs with a Supermicro P4SCi (it is likely that it will also occur with a P4SC8 - similar boards, but I have not tested it)
• The script was designed to be used in conjuction with the above livecd scripts (this script being used last or replacing the mkiso command). The script above will format your usb device with an ext2 filesystem and write the squash image along with the grub loader files to it. After copying the grub boot loader menu it will alter it to work with the usb stick as a bootable device. Be sure to change the device settings in beginning of the script to reflect the mount directory you wish to use, as well as the usb stick device. You may also need to compile your kernel with USB devices (UHCI and OHCI) not as modules.

[edit] PXE Booting

Using the above instructions to build a squash image you can boot into the image via PXE (grub works but does not support as many network cards as pxelinux). NFS and tftp are required on the server end for this method. The following is required to be enabled in the kernel on the Livecd or client end.

Note: : Don't forget to add to your kernel built-in (no modules!)

• NFS filesystem client support as ;
• NFS ROOT support ;
• your network card(s) support ;

Copy the contents of your target directory into a your tftp/nfs directory. Use the following parameters in your PXELinux default configuration file:

label gentoo
   menu label Gentoo LivePXE 
   kernel /livecd/boot/kernel-genkernel-x86-2.6.14-gentoo-r2
   append initrd=/livecd/boot/initramfs-genkernel-x86-2.6.14-gentoo-r2 looptype=squashfs root=/dev/ram0 real_root=/dev/nfs nfsroot=172.16.4.7:/diskless/livecd ip=dhcp udev nodevfs dodmraid cdroot

[edit] Other methods to Copy files

• I have heard that tar and cpio are supposed to copy files faster than cp. I have not confirmed this yet, but here is a little bit of code to try out if you like.
• Comment out the line in your build script:

And add this above or below it:

(Comment - evermind) why don't you use tar cpf - * before the pipe? Isn't that neccesary to preserve the permissions? (End Comment)

(Comment - letum) The p option is needless. Yes, it preserves permissions, but only when you extract the archive. Tar saves permissions by default - p is necessary only during the extraction process. (End Comment)

I have not fully tested this, so be careful. You can also add; '--exclude=usr/src --exclude=var/tmp* --exclude=usr/portage exclude=etc/portage --exclude=var/db --exclude=tmp/* --exclude=var/cache/*' to the first tar if you don't mind leaving the unemerge programs in that the before-build.sh scripts removes. This cuts the time down quite a bit.

Tested it, it seemed to work, but tar does not retain the permissions correctly so in the end it is a bad choice. Which is to bad because it was fast, I am currently looking into cpio. (man tar : tar [ -p, --same-permissions, --preserve-permissions ]

Also, rsync works very well as it only updates what has been updated.

Note : the following tube should work all the time. With the first line, it makes a mini-script that is as error-safe as can be. It should preserve ownerships.

[edit] Testing the LiveCD without wasting a blank CD

If you're trying out some weird stuff with your LiveCD and don't want to waste a blank CD (and don't have a rewritable) each time you've built a new ISO-image, you can emulate the LiveCD with qemu, a very nice open-source emulator.

If you don't have qemu already, of course you need to emerge it (kqemu is an accelerator module)

  1. Notice from falcon15: Perhaps you should do 'echo "app-emulation/qemu kqemu" >>/etc/portage/package.use' before merging qemu in order to make use of kqemu

To use it, there are several possibilities:

# first, load the kqemu kernel module
# in order to get /dev/kqemu in udev on most distros you'll need to do:
# modprobe kqemu major=0
modprobe kqemu

#simple example of qemu usage
qemu -cdrom livecd.iso

#give more resources on the host systems (you need to be root for this)
nice -n 2 qemu -cdrom livecd.iso

#give more (emulated) memory, here: 512 megs
qemu -cdrom livecd.iso -m 512

It may be that some exotic functions are not supported by qemu, yet this shouldn't happen too often. If emulation starts failing at a very basic point, you might want to try emulating something like a Knoppix image in order to see if your qemu install is ok. However, booting sometimes hangs for about ten minutes or even more at

  >>> Copying read-write image contents to tmpfs

This is due to qemu's memory management. But apart from the waiting, this is a harmless effect, and it doesn't seems to occur on a burned LiveCD.

[edit] Making a bootable floppy boot from a boot option in your cd's bootloader

This is very simple after I figured this out, you just need a few basic things;

Get syslinux

Then copy the memdisk to your boot folder

Then make an image of your bootable floppy

And after all these hard steps just make an boot option (example in grub)

and addhave not compiled kernel, do the following: (End of comment)

 title=Bootable floppy from livecd
        root (cd)
        kernel (cd)/boot/memdisk
        initrd (cd)/boot/floppy

That's all there is to it!

[edit] 32bit chroot on a 64bit installation

Important Note: if you are following this guide from within a 64-bit environment, consult the "32Bit Chroot Guide for Gentoo/AMD64" at [3] for how to properly chroot.

To modify the build script change (the fast way);

to

Comment: If you don't have linux32 at hand, you could also just modify the folowing: make.conf for right architecture, run make ARCH=i386 (menuconfig) or genkernel --arch-override=x86 initrd/all, (depends on what you use)!

[edit] Floppy tips

If you have a floppy that you have made bootable, but need to make it bigger so more files will fit on it, that can be done very simple.

Create a new file as big as you need it to be

Make it a file system (dos for this instance).

Mount the new image and fill it (redo the above steps if you still have not got enough space).

Fill it with the files you need. The next step is to get the old bootsector (orriginal_floopy can be a dd image or a /dev node);

Then simply merge the bootsector and new_image;

After that you can simply start it as the old floppy!

This can also be done for hd's, I recommend using something like vmware make a small hd and install whatever you want on it (needs to be bootable). The trick is now to make a dd image of the complete hd (example: dd if=/dev/hda of=~/hd.dd).

This is now bootable as in the previous floppy example, but since it takes the whole hd the dd image can be large, luckily memdisk supports gzip. So gzip -9 your-dd-image.dd does a neat trick here.

[edit] COMPLETE BUILD SCRIPT

I had problems running this scripts : mounting/umounting different parts (sys, proc, ...), so I change to a plain text method. This way, reader can understand step by step what's happening and can check what's wrong.

[edit] build script

To help automate these steps, create a simple 'build' script inside ${LIVECD}

This builds the complete livecd from scratch at once, be careful it replaces menu.lst with grub.conf (so we do not forget!)

Then create the build script :

And here's the script

#!/bin/bash
LIVECD=~/livecd
TARGET=${LIVECD}/target
TARGET_SOURCE=${TARGET}/files/source/
SOURCE=${LIVECD}/source

# It is important, that you don't have mounted these filesystems - cp will fail in /proc when it's mounted. >>>
umount ${SOURCE}/sys
umount ${SOURCE}/dev
umount ${SOURCE}/proc
umount ${SOURCE}/usr/portage
# <<<

rm -rf ${TARGET}
mkdir ${TARGET}
cp -a ${SOURCE}/boot ${TARGET}/
mkdir -p ${TARGET}/files/source
#make sure the menu.lst is up to date (ONLY EDIT grub.conf!!)
rm ${TARGET}/boot/grub/menu.lst
cp ${TARGET}/boot/grub/grub.conf ${TARGET}/boot/grub/menu.lst

cd ${TARGET}/files
rsync -caq ${SOURCE} source
mount -o bind /sys ${TARGET_SOURCE}/sys
mount -o bind /dev ${TARGET_SOURCE}/dev
mount -o bind /proc ${TARGET_SOURCE}/proc
mount -o bind /usr/portage ${TARGET_SOURCE}/usr/portage

chroot ${TARGET_SOURCE} /bin/bash --login <<CHROOTED
env-update
/sbin/depscan.sh
update-modules
localepurge
makewhatis -u
# You can uncomment the following lines if you want, I use those so I do not forget to merge these packages (kernel updates etc).
# emerge sys-fs/fuse dmraid alsa-driver captive nvidia-kernel nvidia-glx
# emerge unmerge localepurge genkernel ccache
find / -xdev -name ".keep" -exec rm -rf {} \;
CHROOTED

umount ${TARGET_SOURCE}/sys
umount ${TARGET_SOURCE}/dev
umount ${TARGET_SOURCE}/proc
umount ${TARGET_SOURCE}/usr/portage

env-update
source /etc/profile

cd ${TARGET_SOURCE}
rm -rf var/tmp/*
#rm -rf var/run/*
rm -rf var/lock/*
rm -rf var/cache/*
mkdir -p var/lock/subsys
mkdir -p var/cache/edb/deb
mkdir var/cache/hald
rm -rf var/db
rm -rf tmp/*
rm -f etc/mtab
touch etc/mtab
rm -rf usr/portage
rm -rf etc/portage
rm -rf usr/share/doc
rm root/.bash_history
rm root/.zcompdump
rm -rf var/log
mkdir var/log
rm etc/make.profile
rm -rf root/.ccache
rm -rf root/.mc
rm -rf initrd
rm before-build

cd ${TARGET}/files
mksquashfs source/ ${TARGET}/livecd.squashfs
cd ${TARGET}
touch livecd
rm -rf ${TARGET}/files/
cd ${LIVECD}
mkisofs -R -b boot/grub/stage2_eltorito -no-emul-boot -boot-load-size 4 -boot-info-table -iso-level 4 -hide-rr-moved -c boot.catalog -o ${LIVECD}/livecd.iso ${TARGET}/

The emerge fuse and dmraid makes sure we do not forget to built these modules when we updated the kernel.

Make both executable, and be sure to read the warning message. If you do not check if it has successfully chrooted to the ~/target/files/source directory and you run the file it might DESTROY your whole system!

Squashing a large source directory takes some time, so be patient. Once done, just burn your ISO image, boot it, and enjoy :)

[edit] Work script

A basic work script which enables you to easily work in the chroot, a more advanced automated script is below. work.

cd livecd touch work chmod +x work

Here it is:

#!/bin/bash
mount -o bind /proc source/proc
mount -o bind /sys source/sys
mount -o bind /dev source/dev
mount -o bind /dev/pts source/dev/pts
mount -o bind /usr/portage source/usr/portage
chroot source/ /bin/bash --login
umount source/proc
umount source/sys
umount source/dev/pts
umount source/dev
umount source/usr/portage

Do not forget, after running this script when you get in the environment, to manually run;

Because a simple bash script is not able to give commands to the new environment, but it does unmount , run env-update and source /etc/profile after you leave the environment.

Comment: To automate this step just remove the --login part and add the folowing to source/root/.bash_rc

Here is what I use as 'work' script. This makes sure that you can invoke 'work' multiple times, quit and still have a sane environment to work in the rest of the windows.

#!/bin/bash

# change to my directory
cd /home/root/livecd

#
# mount if not already mounted
#
grep -q /home/root/livecd/source/proc /etc/mtab
if [ "$?" == "1" ]
then
       mount -o bind /proc source/proc
       mount -o bind /sys source/sys
       mount -o bind /dev source/dev
       mount -o bind /dev/pts source/dev/pts
       mount -o bind /usr/portage source/usr/portage
       mount -o bind /tmp source/tmp
       sed -i -e "s:31m\\\u:31mCHROOT-64:" source/root/.bashrc
       sed -i -e "s:31m\\\u:31mCHROOT-64:" source/root/.bash_profile
fi
chroot source/ /bin/bash

# find out if there is anybody using the chroot
count=0
for i  in `ps -aef| grep $0 | grep -v grep | awk '{print $2}'`
do
       rln=`readlink /proc/$i/cwd`
       if [ "$rln" == "/home/root/livecd" ]
       then
               count=$((count+1))
       fi
done
if [ "$count" == "1" ]
then
       sed -i -e "s:31mCHROOT-64:31m\\\u:" source/root/.bashrc
       sed -i -e "s:31mCHROOT-64:31m\\\u:" source/root/.bash_profile
       umount source/proc
       umount source/sys
       umount source/dev/pts
       umount source/dev
       umount source/usr/portage
       umount source/tmp
fi