LiveCD from scratch
[edit] Disclaimer
[edit] Introduction
This article will show you how to build your own LiveCD. This script is used to build the FLY LiveCD. The methods and script here will teach you how to make a basic bootable ISO file. From there, its up to you to add in whatever ebuilds, config files, scripts, programs, etc you so wish.
This method was adapted in April 2008 from the method described in this wiki: Build Your Own LiveCD or LiveDVD. Since then it has continued to evolve to it's current state, and will likely progress even further, at which point this article will be maintained. A few things to note that vary between these two articles:
- This article does not use Genkernel.
- This article uses a custom initramfs
- This article is current
- This article contains a complete script to make your own LiveCD
- Hopefully, this article is a little more straight forward and easier to understand.
Lets begin.
[edit] Prerequisites
[edit] System Requirements
In order to develop your own LiveCD using this article, a few things are required:
- a Linux operating system (of preference Gentoo)
- root access (or at the very least, sudo)
- SquashFS file system support (sys-fs/squashfs-tools)
- mkisofs (app-cdr/cdrtools)
- high speed internet connectivity
- At the very minimum, 3-4gb of free disk space to allocate to the build process.
Depending on how you choose to configure your LiveCD, and depending on your hardware, the compile times can be quite long. For instance, on an old Dell Inspiron 6000 laptop (Intel Pentium-M 1.6GHz and 2gb RAM) a NEW build from scratch with all ebuilds required for the FLY LiveCD takes roughly 11 straight hours of non-stop CPU torture. An UPDATE, on the other hand, can take as little as 1 hour if there are no ebuilds to update.
[edit] User Requirements
Not only must you have the required system requirements as listed above, but you also need to have certain requirements as a user:
- Linux experience
- experience installing Gentoo from a stage3 tarball following the Gentoo Handbook.
- knowledge of Bash scripting
- the ability to troubleshoot
- patience
- determination
[edit] Additional Resources
Here are some links you may want to check to help you along the way:
[edit] bash.nanorc
If you're using nano to write your Bash, you may want to add this file to enable some good Bash syntax highlighting. A little known feature of nano, but it comes with various syntax highlighting depending on what you're writing. The files can be found in /usr/share/nano/
A modified version of /usr/share/nano/sh.nanorc is used so that variables within double quotes ("") keep their proper colour, but not within single quotes (''). As well, any special characters in a comment line are not parsed.
Here is the modified file. Save it to /usr/share/nano/bash.nanorc and add include /usr/share/nano/bash.nanorc to ~/.nanorc
## Here is an example for Bourne shell scripts.
##
syntax "bash" "\.sh$"
header "^#!.*/(ba|k|pdk)?sh[-0-9_]*"
icolor brightgreen "^[0-9A-Z_]+\(\)"
color green "\<(case|do|done|elif|else|esac|exit|fi|for|function|if|in|local|read|return|select|shift|then|time|until|while)\>"
color green "(\{|\}|\(|\)|\;|\]|\[|`|\\|\$|<|>|!|=|&|\|)"
color green "-[Ldefgruwx]\>"
color green "-(eq|ne|gt|lt|ge|le|s|n|z)\>"
color brightblue "\<(cat|cd|chmod|chown|cp|echo|env|export|grep|install|let|ln|make|mkdir|mv|rm|sed|set|tar|touch|umask|unset)\>"
color brightyellow ""(\\.|[^"])*""
icolor brightred "\$\{?[0-9A-Z_!@#$*?-]+\}?"
color brightyellow "'(\\.|[^'])*'"
color cyan "(^|[[:space:]])#.*$"
color ,green "[[:space:]]+$"[edit] History
As mentioned in the introduction, the script in this article was adapted from another method of creating a LiveCD. This method used two scripts to complete the process. A primary script which ran from start to finish, and a secondary script which ran while in chroot during the middle of the primary script. Updating the LiveCD had to be done manually.
When the script in this article was first written, it too used two separate scripts. Eventually, a third script was written which had as function to update a previous version with current ebuilds, kernel, config files, etc. This script had within it the functions of the chroot script and was essentially two scripts in a single file.
Very recently, these three scripts were combined into a single file. This was made possible because the primary scripts in both a new LiveCD build and an update are similar, and likewise for the chrooted scripts. However this means this single script file is quite large and can be very confusing to those not very familiar with it.
Why merge three script files into one? Although for new users it makes it harder to read and it can make it harder to troubleshoot, having only one file means changes only have to be made to a single file. Previously, if an update was made using one script, and then another update was made from scratch it could be hard to keep all scripts in sync.
To view previous examples of this script - before it was merged into a single file - check out FLY's make knowledge page.
[edit] The Environment
[edit] Directories
The script is designed to run by a regular user in its home directory, but you can modify the script so that it can be run from another location, or even, from any location, using the proper variable usage. Just make sure the user has appropriate sudo access to the script. Some of the commands, such as mount, have to be run as root.
Let's look at the directory structure that the script is currently designed to use. Feel free to modify this if you have other preferences.
- /home/FLY/ is the main directory where all activities pertaining to building the LiveCD take place. 'FLY' is the name of my LiveCD and this is the reason for the choice.
- /home/FLY/build/ is the directory which contains the autobuild script that is part of this wiki.
- /home/FLY/build/source/ is the working directory while the LiveCD is being built.
- /home/FLY/build/FILES/ is where the Gentoo stage3 autobuild is kept.
- /home/FLY/build/FILES/MAKE/ is where all custom files for the LiveCD are kept. Within this directory, all files are kept in a new directory structure. For example, if you wanted to modify /etc/conky/conky.conf in the LiveCD, the file would be stored in /home/FLY/build/FILES/MAKE/etc/conky/
- /home/FLY/finished/ contains all finished builds of the LiveCD. A new subdirectory is created with the version number of the LiveCD, and /home/FLY/build/source/ is moved here. As well, the contents of /home/FLY/build/FILES/ is copied to the version subdirectory as well as a copy of the script used to build it.
[edit] Variables
There are two groups of variables in this script. Variables that should be configured for every new version, and variables that shouldn't.
[edit] Dynamic Variables
- VERSION is used to identify the current version of your build. You can use any format you wish, including both numerical, alphabetical, or alphanumeric. The FLY LiveCD is built using a two decimal system (x.x.x) where the first numeric is the major version, the second numeric is the minor version, and the third numeric is the patch version.
- AUTOBUILD is used to set which version of the Gentoo weekly autobuilds to use. Prior to the Gentoo Engineering team coming up with weekly builds, this value was set to the yearly stage3 releases. With weekly automated stage3 tarballs, this variable becomes much more important.
- ROOTPASSWD should be obvious. Use it to set the root password that will be set on the LiveCD.
- FLYPASSWD is used to set the password for the LiveCD's regular user account. In this case, that user is called 'fly'. If you so wish, you can change this variable to whatever name you like. Just make sure to substitute it everywhere in the script.
- TYPE is a new variable used to tell the script in which mode to run in. Two options are available: NEW and UPDATE. The former is used to set the script to build the LiveCD using the Gentoo autobuild and to emerge all your ebuilds and build the LiveCD from scratch. UPDATE is used to update a previous version of the LiveCD with new ebuilds and new files as required.
- OLDVERSION is used to list the version of the LiveCD to update when the script is set to UPDATE.
[edit] Static Variables
- BUILD is simply used to specify the location of the log file. Set this to what you want, but it shouldn't be changed after that.
- NEWKERNELMD5 When the script is running in UPDATE mode, this variable will store the MD5 checksum of the current .config file.
- OLDKERNELMD5 Similarly, this variable stores the checksum of the .config file used in the previous version of the LiveCD. If these two values are the same, it is assumed the kernel configuration hasn't changed.
- KERNEL stores the result of the above two variables. If they are equal, then this variable is set to NO, meaning no recompile is required. If the above two values are not equal, then this variable is set to YES.
- KVERSION is used to store the kernel version used in the previous version of the LiveCD. This variable is only used if KERNEL is set to NO.
- KAWK is used to store the results of awk'ing the log file to see if the kernel was updated during the build. This variable, in conjunction with KERNEL are used to decide whether or not the kernel should be recompiled.
- failure is used to count any of the $? variables for failures. If any are found, they are logged. This variable is only set at the end of the script.
[edit] Required Files
Before the script can be run, there are a few files that must exist. Depending on the ebuilds and other configuration choices you make, other files will also be required. Previous experience installing a Gentoo system will really help here.
The minimum required files are:
- /home/FLY/build/FILES/MAKE/usr/src/linux/.config: The easiest way to create the kernel config is to emerge onto your host the sources for the kernel you want in your LiveCD. Then go to it's source directory (/usr/src/) and configure it with make menuconfig. Save the config and copy over the .config file to /home/FLY/build/FILES/MAKE/usr/src/linux/.config. If your LiveCD is designed for specific hardware, you should configure it as such. However, if your LiveCD is designed to boot on as much hardware as possible, you should module as much of it as possible using make allmodconfig. From there, you can then edit the config and remove items you know you wont use. For instance, parallel port support. Note that you'll need some things to be built in to the kernel, notably, chipset drivers for booting.
- /home/FLY/build/FILES/MAKE/boot/initramfs/init: See the section 'Additional Files' further down. You'll need this file in order to boot your LiveCD.
- /home/FLY/build/FILES/MAKE/boot/grub/menu.lst: Again, see the section 'Additional Files' further down. Note that if you want to use another bootloader, you'll have to have it's file in place. For instance, if you want LILO, you'll want to have this file: /home/FLY/build/FILES/MAKE/etc/lilo.conf
- /home/FLY/build/FILES/MAKE/etc/fstab: See section 'Additional Files'.
- /home/FLY/build/FILES/MAKE/etc/make.conf: See section 'Additional Files'. Modify as required for your purposes. Note, you'll want to keep the following USE flags: livecd minimal.
- /home/FLY/build/FILES/MAKE/etc/conf.d/hostname
[edit] Explaining the Script
[edit] Part I: Setting the variables
| Code: |
# Modify the following VERSION="0.0.1" AUTOBUILD="20100518" ROOTPASSWD="toor" FLYPASSWD="sharky" TYPE="UPDATE" # Modify only if $TYPE="UPDATE" OLDVERSION="0.0.0" |
Make sure these variables are all properly configured as per #Variables above. In the case of OLDVERSION, as the comment states, you only need to configure it if the script is running in UPDATE mode. If it isn't, this value is not used.
| Code: |
# TYPE variable check if [ "$TYPE" != "UPDATE" ] && [ "$TYPE" != "NEW" ]; then echo "Wrong TYPE variable: $TYPE" echo "Quitting ..." exit fi |
The above part of the script is simply used to ensure the value of the TYPE variable is valid and the script can continue. Since an incorrect value here can royally screw things up. Aside from logging during the script, this is the only part that validates what you input. The rest of the script assumes you're entering valid data.
| Code: |
# Do not modify the following
BUILD="/home/FLY/build/source/$VERSION-AUTOBUILD.log"
if [ "$TYPE" = "UPDATE" ]; then
NEWKERNELMD5=`md5sum /home/FLY/build/FILES/MAKE/usr/src/linux/.config | awk '{print $1}'`
OLDKERNELMD5=`md5sum /home/FLY/finished/$OLDVERSION/FILES/MAKE/usr/src/linux/.config | awk '{print $1}'`
if [ "$NEWKERNELMD5" = "$OLDKERNELMD5" ]; then
KERNEL="no"
KVERSION=`ls /home/FLY/finished/$OLDVERSION/source/lib/modules/`
else
KERNEL="yes"
fi
fi
|
This next bit sets the log file location. And also checks the kernel for upgrade if the TYPE variable is set to UPDATE. Nothing here should ever be changed once properly configured.
[edit] Part II: Starting Things Off
| Code: |
# Start of FLYautobuild.sh cd /home/FLY/ mkdir finished/$VERSION 2>> tmpAUTOBUILD.log echo "==== START OF AUTOBUILD LOG ===============================================" >> tmpAUTOBUILD.log date >> tmpAUTOBUILD.log |
This is pretty basic. The script will start off in the proper directory and print a date stamp to the log file indicating the start of the build process. This is so you can keep track of how long the process takes. Another good thing to do would be running the script through time as well. This way, you'll get a report at the end of the script stating how long it ran for. Note that the result of time wont be logged, just displayed.
| Code: |
if [ "$TYPE" = "NEW" ]; then
mkdir -p build/source/{usr/portage,etc/portage,mnt/linuxfly.net} 2>> tmpAUTOBUILD.log
cd build/source 2>> tmpAUTOBUILD.log
mv ../../tmpAUTOBUILD.log $VERSION-AUTOBUILD.log
tar jxpf ../FILES/stage3-i686-$AUTOBUILD.tar.bz2 2>> $BUILD
else
mkdir build/source
mv tmpAUTOBUILD.log build/source/$VERSION-AUTOBUILD.log
rsync -aq --exclude={chroot,lib/modules/*,$VERSION-AUTOBUILD.log,date.stamp} finished/$OLDVERSION/source/ build/source/ 2>> $BUILD
cd build/source
fi
|
Throughout the script, since it is now combined with the script to update a previous version, you'll see these if statements which read the TYPE variable. Here if the LiveCD is new, the directory structure for it will be created and the Gentoo autobuild is extracted.
If in UPDATE mode, the script will copy the contents of the previous version's source directory, excluding certain files. The chroot file is created by this script with the instructions to be run while chrooted during the build process. Previously, this was a separate file. Since this file cannot be reused, it isn't copied. It isn't deleted however as it is used to help troubleshoot bad builds, and can also be used as part of logging if you so wish. The contents of /lib/modules/* is excluded, this contains kernel libraries. Previously, the kernel was always recompiled, even on UPDATE. Now, thanks to new variables and a smarter script, the kernel is only recompiled if the .config has changed, or if the version has changed. If neither of these have changed, then later /lib/modules/* will be copied over. We also don't want to copy over the previous log file or the date stamp.
[edit] Part III: Mounting
| Code: |
mount -t proc none proc/ 2>> $BUILD mount --bind /dev dev/ 2>> $BUILD mount --bind /sys sys/ 2>> $BUILD mount -o ro --bind /usr/portage/ usr/portage/ 2>> $BUILD |
Just like a new install of Gentoo on a real system, the usual /proc, /sys, and /dev have to be mounted. In addition, in order not to have to download a new Portage snapshot and then update it, and have to re-download redundant distfiles, the whole content of /usr/portage/ is mounted. Obviously if you're not running a Gentoo system you cant do this. Furthermore, the contents of /usr/portage/ are mounted read-only. This is because in previous experiences, if the script failed in progress, and it was killed, deleting the build directory would wipe your host system's entire Portage tree and distfiles. Not a huge problem but a rather large inconvenience. Having to sync a new tree and download a gig or two of distfiles isn't fun.
[edit] Part IV: Preparing for chroot
| Code: |
if [ "$TYPE" = "NEW" ]; then
cp /etc/resolv.conf etc/resolv.conf 2>> $BUILD
cp -r ../FILES/MAKE . 2>> $BUILD
mv MAKE/etc/portage/* etc/portage/ 2>> $BUILD
mv MAKE/etc/{locale.gen,make.conf} etc/ 2>> $BUILD
date +%m%d%H%M%Y > date.stamp
|
In a NEW install, resolv.conf is copied over so that once in chroot, you'll still have net access. Every customized file, which should be stored in /home/FLY/build/FILES/MAKE/ is copied over to the root of the new environment, so they'll be accessible within chroot. Its important to keep the files within the same structure as where they will ultimately be. Portage package files are moved to their proper directory, so is locale.gen and make.conf. The date command is run with specific parameters to print out the current date to a file. This is so the date can then be set from within the chroot using this file as the date parameters.
| Code: |
else
echo "==== SYNCING NEW FILES ====================================================" >> $BUILD
rsync -aq --exclude={usr/src/,etc/{locale.gen,make.conf},portage/} /home/FLY/build/FILES/MAKE/* MAKE/ 2>> $BUILD
if [ "$KERNEL" = "no" ]; then
echo "Copying old kernel to current build ..." >> $BUILD
mkdir MAKE_oldkernel 2>> $BUILD
rsync -aq /home/FLY/finished/$OLDVERSION/source/lib/modules/ MAKE_oldkernel/ 2>> $BUILD
cp -f /home/FLY/finished/$OLDVERSION/source/usr/src/linux/arch/i386/boot/bzImage MAKE_oldkernel/bzImage 2>> $BUILD
fi
rsync -aq /home/FLY/build/FILES/MAKE/etc/portage/* etc/portage/ 2>> $BUILD
rsync -aq /home/FLY/build/FILES/MAKE/etc/{locale.gen,make.conf} etc/ 2>> $BUILD
cp -f ../FILES/MAKE/usr/src/linux/.config usr/src/.config 2>> $BUILD
fi
|
In UPDATE mode, the script does pretty much the same thing, but doesn't do a date stamp, as this was done in the previous version. As well, newly featured is that if KERNEL is equal to NO, then the previous version's kernel modules and kernel image will be copied to the current build.
[edit] Part V: Writing the chroot script
Originally, this script would have been a separate file. When the first update script was created, it used line variables to contain the chroot script, and write the it to file during the execution of the update script. This method was used in this script with EOF to contain both the NEW and UPDATE chroot script.
This gets quite complex so I'll break it down in several chunks:
[edit] Setting the variables, again
| Code: |
cat > chroot << 'EOF' #!/bin/bash DATE=`cat /date.stamp` VERSION=$1 AUTOBUILD=$2 ROOTPASSWD=$3 FLYPASSWD=$4 TYPE=$5 KERNEL=$6 KVERSION=$7 BUILD=/$VERSION-AUTOBUILD.log env-update && source /etc/profile 2>> $BUILD EOF |
Previously, with the two separate scripts, variables in the chroot script had to be manually entered before running the autobuild process, and they'd have to match the variables from the first build script. This was a potential point of failure. In order to resolve this, the script will now use line arguments to pass it's variables onto the chroot script. So as can be seen here, all required variables get their data from a line argument, except for BUILD, as it's value is different from the non-chroot version.
By the way, you've probably noticed that EOF is within single quotes. This is so when the main script is run, it process any special character within EOF. These special characters will only be processed when the chroot script runs.
[edit] NEW - Setting date and time
| Code: |
if [ "$TYPE" = "NEW" ]; then
cat >> chroot << 'EOF'
echo "==== SETTING THE DATE =====================================================" >> $BUILD
cp /usr/share/zoneinfo/Canada/Eastern /etc/localtime 2>> $BUILD
if [ "$?" = "0" ]; then echo "Localtime OK!" >> $BUILD; else echo "Localtime FAIL!" >> $BUILD; fi
date $DATE 2>> $BUILD
if [ "$?" = "0" ]; then echo "Set Date OK!" >> $BUILD; else echo "Set Date FAIL!" >> $BUILD; fi
|
If the script is running in NEW mode, the correct date has to be setup in the chrooted environment. Timezone data is also setup. Obviously here you'll want to use your own zoneinfo file, not necessarily Canada/Eastern. $? checks are in place a little bit everywhere in the script to quickly confirm whether a command has executed successfully. Since this information doesn't change, it is not redone if the script runs in UPDATE mode.
[edit] NEW - Emerges
| Code: |
echo "==== EMERGING REQUIRED EBUILDS ============================================" >> $BUILD
echo "Updating autobuild $AUTOBUILD tarball" >> $BUILD
emerge -uDNvp world >> $BUILD
emerge -uDNq world 2>> $BUILD
if [ "$?" = "0" ]; then echo "Updating OK!" >> $BUILD; else echo "Updating FAIL!" >> $BUILD; fi
echo -5 | update-etc 2>> $BUILD
echo "Required merges" >> $BUILD
emerge -vp expect sysklogd memtest86+ gentoolkit gentoo-sources livecd-tools localepurge slocate >> $BUILD
emerge -q expect sysklogd memtest86+ gentoolkit gentoo-sources livecd-tools localepurge slocate 2>> $BUILD
if [ "$?" = "0" ]; then echo "Required OK!" >> $BUILD; else echo "Required FAIL!" >> $BUILD; fi
|
The first thing that needs to be done is update the Gentoo autobuild. Despite weekly (usually) releases, there will always be updates. Whats nice about the Gentoo weekly autobuilds is that you'll never be far behind on updates which means there wont be any surprises. Unlike before where the initial stage3 tarball could be months and months old and doing an update could be at best troublesome or at worst long and agonizing.
Following this the "required" ebuilds are installed. Although some of these can be considered optional, the absolute must would be:
Note that even though app-admin/localepurge and dev-tcltk/expect can be considered optional, they are used in the script and so if you choose not to emerge them, you should remove the lines that reference them. In the case of expect, this will mean you wont be able to automate the password changes in the script, and will have to manually do this. If you so wish, you can also use another kernel. You don't have to use sys-kernel/gentoo-sources. Note that if you change this, you may have to make other changes to the script as well.
| Code: |
echo "==== EMERGING NEW EBUILDS =================================================" >> $BUILD echo "0.0.1 merges" >> $BUILD emerge -vp moo >> $BUILD emerge -q moo 2>> $BUILD |
In this section, you can emerge all the ebuilds you want in the LiveCD. Every new release of a LiveCD may bring new ebuilds, so it can be useful to separate them by version with an echo and the emerge. The reason for two emerge statements is that the first will print to the log file a list of what will be emerged, and the second will only print stderr, while doing the actual merges. This way you have a complete log of the events. Obviously, for this wiki all ebuilds have been removed, but the format was kept so you get the point.
| Code: |
echo "Rebuild" >> $BUILD
revdep-rebuild -- -vp >> $BUILD
revdep-rebuild -- -q 2>> $BUILD
if [ "$?" = "0" ]; then echo "Rebuild OK!" >> $BUILD; else echo "Rebuild FAIL!" >> $BUILD; fi
|
Once all your packages are emerged, run revdep-rebuild just in case. Although it shouldn't find anything broken, it certainly doesn't hurt to check.
[edit] NEW - Post emerge
| Code: |
echo "==== FINISHING AND CLEANING UP =============================================" >> $BUILD echo -5 | etc-update 2>> $BUILD |
etc-update always has to be run. Updated ebuilds will sometimes bring new config files and so the first command here automatically copies them. Don't worry, it wont overwrite your custom files. They haven't been added yet.
| Code: |
echo "Adding / Removing startups" >> $BUILD rc-update add autoconfig default >> $BUILD rc-update del checkfs boot >> $BUILD rc-update del checkroot boot >> $BUILD |
Depending on what ebuilds you emerge, and your preferences, you'll have a few services to add to startup.
Also, checkfs and checkroot will have to be removed. Previously, there were problems with this last summer and it wasn't until these services were removed from startup, and 'read-only' was added to fstab for the root directory that the LiveCD was able to boot again. Not sure what changed, but if you set this up you're fine.
| Code: |
echo "Running eix" >> $BUILD
eix-update 2>> $BUILD
eix -Ic >> $BUILD
eix-test-obsolete brief >> $BUILD
emerge --depclean -p >> $BUILD
emerge --depclean 2>> $BUILD
if [ "$?" = "0" ]; then echo "Depclean OK!" >> $BUILD; else echo "Depclean FAIL!" >> $BUILD; fi
|
Next, we'll update eix and log each installed ebuild. Its useful information to have in release notes for your LiveCD if ever someone wants to know not only what is in your LiveCD, but also which version. Then run depclean to get rid of anything that's useless.
[edit] NEW - Installing Grub
| Code: |
echo "==== EMERGING GRUB ========================================================" >> $BUILD
emerge -vp grub >> $BUILD
emerge -q grub 2>> $BUILD
if [ "$?" = "0" ]; then echo "Grub OK!" >> $BUILD; else echo "Grub FAIL!" >> $BUILD; fi
|
This article uses the Grub boot loader, but you can use whichever boot loader you want provided you know how to set it up and make the proper modifications. Why is it emerge it separately? Not sure anymore. There was a reason to do so two years ago...
[edit] NEW - Adding a user
| Code: |
echo "==== ADDING FLY ACCOUNT ===================================================" >> $BUILD
useradd -m -G wheel,users,audio fly 2>> $BUILD
if [ "$?" = "0" ]; then echo "Useradd OK!" >> $BUILD; else echo "Useradd FAIL!" >> $BUILD; fi
|
You'll want to add a regular user to the LiveCD. Running a LiveCD as root can be dangerous. For instance, what if your LiveCD automatically starts SSHd and the default password for root is 'toor' and your attacker knew this. Not only does he gain root access to your environment, but to the box the LiveCD is running on, and he also gets access to your internal network. Fun?
You'll want to change the username. The groups you'll be adding the account to will also vary.
[edit] NEW - Building your initramfs
| Code: |
echo "==== INITRAMFS ===========================================================" >> $BUILD
mkdir -p /boot/initramfs/{sbin,bin,etc,lib,dev} 2>> $BUILD
mv /MAKE/boot/initramfs/init /boot/initramfs/init 2>> $BUILD
mv /MAKE/boot/v86d /boot/initramfs/sbin/v86d 2>> $BUILD
cp /usr/sbin/lspci /boot/initramfs/sbin/lspci 2>> $BUILD
touch /boot/initramfs/etc/{mtab,fstab} 2>> $BUILD
cd /bin
cp sh busybox /boot/initramfs/bin/ 2>> $BUILD
cp /usr/share/misc/pci.ids /boot/initramfs/bin/ 2>> $BUILD
cd /lib
#REQUIRED LIBS MAY CHANGE (use ldd <command>)
cp /usr/lib/libpci.so.3 libncurses.so.5 libdl.so.2 ld-linux.so.2 libc.so.6 libz.so.1 libresolv.so.2 /boot/initramfs/lib 2>> $BUILD
|
Whereas the LiveCD wiki this article is based on uses Genkernel which automatically builds the initramfs, this article will require you to build your own. That gives you more options. And once you've got it figured out, its not hard.
The first step is to create the directory structure. This is within /boot/initramfs/ Think of it as a miniature Linux system within your LiveCD (which is itself chrooted from within your host!) Here you'll add some basic files needed to get the initramfs to boot the LiveCD, and Grub will boot the initramfs.
The init script used is obviously a custom one that had to be written for the LiveCD. Its shown later in this wiki.
lspci is nice to have so that if your LiveCD fails to boot on some hardware, you'll at least have an output to go with to help you troubleshoot whats going on.
| Code: |
cd /boot/initramfs/dev mknod console c 5 1 2>> $BUILD mknod null c 1 3 2>> $BUILD mknod hda b 3 0 2>> $BUILD mknod hdb b 3 64 2>> $BUILD mknod hdc b 22 0 2>> $BUILD mknod hdd b 22 64 2>> $BUILD mknod sr0 b 11 0 2>> $BUILD mknod sda b 8 0 2>> $BUILD mknod sdb b 8 16 2>> $BUILD mknod sdc b 8 32 2>> $BUILD mknod sdd b 8 64 2>> $BUILD mknod tty c 4 0 2>> $BUILD mknod loop0 b 7 0 2>> $BUILD mknod tty1 c 4 1 2>> $BUILD mknod mem c 1 1 2>> $BUILD mknod zero c 1 5 2>> $BUILD mkdir loop && cd loop 2>> $BUILD ln -s ../loop0 0 2>> $BUILD |
These commands add in all the block and character devices needed to boot the LiveCD. You're probably familiar with most of them. Again last summer a problem was encountered where having a loop interface in /dev/ was causing issues and a symlink was needed in /dev/loop/ pointing back to the loop in /dev/ Redundant, but without it, the LiveCD was failing to boot.
| Code: |
cd ../.. find . | cpio -H newc -o > ../initramfs.cpio cd .. cat initramfs.cpio | gzip > initramfs.igz EOF |
Lastly, package the contents of /boot/initramfs/ into a cpio archive and then gzip it.
[edit] UPDATE - Checking for updates
Things go much quicker here. As all of the steps from the previous section have already been performed on the previous version of the LiveCD. In this case, you're merely going to update it.
| Code: |
else
cat >> chroot << 'EOF'
echo "==== UPDATING WORLD =======================================================" >> $BUILD
emerge -uDNvp world >> $BUILD
emerge -uDNq world 2>> $BUILD
if [ "$?" = "0" ]; then echo "Updating OK!" >> $BUILD; else echo "Updating FAIL!" >> $BUILD; fi
|
First thing is first. Check for updates. There will always be updates.
[edit] UPDATE - Kernel
| Code: |
echo "==== KERNEL AND MODULES ===================================================" >> $BUILD
KAWK=`awk '/sys-kernel\/gentoo-sources/ && /ebuild/' $BUILD`
if [ "$KERNEL" = "no" ] && [ "$KAWK" = "" ]; then
echo "Kernel compile not required. Using old kernel." >> $BUILD
mv /MAKE_oldkernel/bzImage /boot/vmlinuz 2>> $BUILD
mv /MAKE_oldkernel/$KVERSION /lib/modules/ 2>> $BUILD
rm -rf /MAKE_oldkernel 2>> $BUILD
else
echo "Kernel compile required. Building new kernel." >> $BUILD
cd /usr/src/linux
mv ../.config .config 2>> $BUILD
make clean && make && make modules_install 2>> $BUILD
if [ "$?" = "0" ]; then echo "Kernel make OK!" >> $BUILD; else echo "Kernel make FAIL!" >> $BUILD; fi
cp -v arch/i386/boot/bzImage /boot/vmlinuz 2>> $BUILD
fi
du -sh /lib/modules/* >> $BUILD
|
This section has just been updated! Now, the first think that happens is KAWK checks the current log file to see if the kernel was updated. Then, if KERNEL equals NO and no kernel updates were found in the log file, then the kernel from the previous LiveCD version is used. Else, the kernel is recompiled.
[edit] UPDATE - Emerges
| Code: |
echo "==== EMERGING NEW EBUILDS =================================================" >> $BUILD
emerge -vp moo >> $BUILD
emerge -q moo 2>> $BUILD
if [ "$?" = "0" ]; then echo "$VERSION OK!" >> $BUILD; else echo "$VERSION FAIL!" >> $BUILD; fi
|
Like in a NEW install, there's probably going to be new ebuilds you want to add. Do this here.
[edit] UPDATE - Post emerge
| Code: |
echo "==== FINISHING AND CLEANING UP ============================================" >> $BUILD
echo -5 | etc-update 2>> $BUILD
echo "Running eix" >> $BUILD
eix -Ic >> $BUILD
eix-test-obsolete brief >> $BUILD
emerge --depclean -p >> $BUILD
emerge --depclean 2>> $BUILD
if [ "$?" = "0" ]; then echo "Depclean OK!" >> $BUILD; else echo "Depclean FAIL!" >> $BUILD; fi
EOF
fi
|
Just like in a NEW install, update etc after the emerges, run eix and check for ebuilds that can be removed.
[edit] Setting user passwords
| Code: |
cat >> chroot << 'EOF' echo "==== USER ADMINISTRATION ==================================================" >> $BUILD echo "Changing root password ..." >> $BUILD expect << INEOF 2>> $BUILD spawn passwd expect "New password:" send "$ROOTPASSWD\r" expect "New password:" send "$ROOTPASSWD\r" expect "Re-type new password:" send "$ROOTPASSWD\r" expect eof; INEOF echo "Changing fly password ..." >> $BUILD expect << INEOF 2>> $BUILD spawn passwd fly expect "New password:" send "$FLYPASSWD\r" expect "New password:" send "$FLYPASSWD\r" expect "Re-type new password:" send "$FLYPASSWD\r" expect eof; INEOF |
Set both root and regular user passwords. Here expect is being used to run the command without user input. Unfortunately, passwd doesn't allow redirecting from a file like other commands would, so this is the only way I know of to run this command automatically. Notice it reads from the ROOTPASSWD and FLYPASSWD variables.
[edit] Moving your customized files
| Code: |
echo "==== MOVING FILES AROUND ==================================================" >> $BUILD echo "Moving etc ..." >> $BUILD mv /MAKE/etc/X11/* /etc/X11/ 2>> $BUILD mv /MAKE/etc/privoxy/* /etc/privoxy/ 2>> $BUILD mv /MAKE/etc/conky/* /etc/conky/ 2>> $BUILD mv /MAKE/etc/conf.d/* /etc/conf.d/ 2>> $BUILD mv /MAKE/etc/init.d/* /etc/init.d/ 2>> $BUILD cp /MAKE/etc/* /etc/ 2>> $BUILD |
All the files meant for /etc are moved into place. These were kept separate from other files simply to assist in troubleshooting. The rest follow:
| Code: |
echo "Moving the remainder ..." >> $BUILD mv /MAKE/boot/grub/* /boot/grub/ 2>> $BUILD mv /MAKE/home/fly/.* /home/fly/ 2>> $BUILD mv /MAKE/root/.* /root/ 2>> $BUILD mv /MAKE/sbin/* /sbin/ 2>> $BUILD mv /MAKE/usr/sbin/* /usr/sbin/ 2>> $BUILD mv /MAKE/usr/share/slim/themes/* /usr/share/slim/themes/ 2>> $BUILD |
The rest of the files are moved.
[edit] A few odds and ends
| Code: |
echo "==== MISC =================================================================" >> $BUILD
rm -r /usr/share/slim/themes/default /mnt/cdrom /etc/init.d/net.eth0 2>> $BUILD
cp /etc/tor/torrc.sample /etc/tor/torrc 2>> $BUILD
localepurge 2>> $BUILD
makewhatis -u 2>> $BUILD
find / -type f -name ".keep" -print -exec rm {} \; 2>> $BUILD
cat /proc/mounts > /etc/mtab 2>> $BUILD
updatedb 2>> $BUILD
# As of this version, and Audacious version 1.5.1 which pulls in dbus,
# This command must be run to prevent Audacious from crashing
dbus-uuidgen --ensure 2>> $BUILD
|
localepurge is then run to get rid of all locales except those defined in /etc/locale.gen This is to free up space. The whatis database is updated, then a find command is run to locate all .keep files which are used in the stage3 tarball so that when Gentoo creates it, empty directories are not deleted. These files are no longer required. Then create /etc/mtab by populating it with /proc/mounts. Finally, the slocate database is updated as well.
[edit] The installer tarball
| Code: |
# Making the tarball for the installer
cd /
tar cjfp installdata-$VERSION.tar.bz2 {var/{db,cache},usr/{lib/portage/,portage/profiles/,share/zoneinfo/},etc/portage/package.*} 2>> $BUILD
|
To keep the LiveCD to a smaller size, various files are removed from it. Such as Portage, eix databases, etc etc. During an HDD install process later on these are re-installed. Although truthfully this is somewhat beyond the scope of this article, it may be added at a later time
ADD TO THIS SECTION
[edit] Finishing up
| Code: |
echo "==== CHOWN AND CHMOD ======================================================" >> $BUILD chown fly:fly /home/fly/ -R 2>> $BUILD if [ "$?" = "0" ]; then echo "01 of 01 OK!" >> $BUILD; else echo "01 of 01 FAIL!" >> $BUILD; fi |
Depending on what ebuilds you install, and what functionalities you add, you may end up with files that need permission and/or attribute changes. This is as good a time as any to do it.
| Code: |
exit EOF |
Finally, exit chroot! Hope that wasnt too hard to follow. Note that at this point in the script, nothing has actually happened. All that has happened is that a file called chroot has been created with some of the above, either section 1 or 2 depending on the mode the autobuild script is running, and this third section. Up next, chrooting to your new LiveCD environment and this chroot script will run through.
[edit] Part VI: Chrooting
| Code: |
echo "==== START OF CHROOT ======================================================" >> $BUILD chroot . /bin/bash --login ./chroot $VERSION $AUTOBUILD $ROOTPASSWD $FLYPASSWD $TYPE $KERNEL $KVERSION |
This is where all the work that happened above pays off. Here your newly written chroot script will execute and read the line arguments into various variables as already discussed. Doing this allows the second script file to be written without any changes having to performed to variables which could cause errors.
[edit] Part VII: Post-chroot
| Code: |
echo "==== END OF CHROOT ========================================================" >> $BUILD env-update && source /etc/profile 2>> $BUILD rm -rf MAKE 2>> $BUILD umount -f sys dev usr/portage /home/FLY/build/source/proc 2>> $BUILD cd .. |
The MAKE directory which contained all your custom files (and is now empty) can be deleted. And all mounts related to the LiveCD are unmounted.
| Code: |
rsync -aq --exclude={$VERSION-AUTOBUILD.log,date.stamp,usr/{portage,src,local,i486-pc-linux-gnu,share/{doc,gtk-doc,man},lib/{portage,gcc/i486-pc-linux-gnu}},etc/portage,var/{tmp/*,run/*,lock/*,lib/clamav/*},tmp/*} source/ source.new/ 2>> $BUILD
cd source.new
|
This next step will move over all the core structure needed for the LiveCD to a new directory where it can then be worked on and molded to the final .iso file we want. The exclude list is optional, but its best to remove certain directories, such as Portage and the eix database to keep the size of the ISO down.
[edit] Part VIII: The target directory
| Code: |
touch etc/mtab 2>> $BUILD mkdir ../target/ 2>> $BUILD cp -a boot ../target/ 2>> $BUILD |
Now start building the eventual ISO. It will go in the target/ directory. Start by copying the contents of boot over, which includes the initramfs.
| Code: |
echo "==== MAKING TARBALLS ======================================================" >> $BUILD cd etc/ tar cpzf ../../target/etc.tar.gz * .[[:alnum:]]* 2>> $BUILD rm -rf * .* 2>> $BUILD cd ../home/ tar cpzf ../../target/home.tar.gz * .[[:alnum:]]* 2>> $BUILD rm -rf * 2>> $BUILD cd ../mnt/ tar cpzf ../../target/mnt.tar.gz * .[[:alnum:]]* 2>> $BUILD rm -rf * 2>> $BUILD cd ../root/ tar cpzf ../../target/root.tar.gz * .[[:alnum:]]* 2>> $BUILD rm -rf * .* 2>> $BUILD cd ../var/ tar cpzf ../../target/var.tar.gz * .[[:alnum:]]* 2>> $BUILD rm -rf * 2>> $BUILD |
A few directories will be made read-write using RAM. These are /etc, /home, /mnt, /root, and /var. In order to do this you'll have to package the contents of those directories into tarballs and use gzip to compress them. Then remove the contents of the directories to save space on the ISO. No sense having files there that use up precious disk space only to mount a tarball on top of them during the LiveCD's boot process.
[edit] Part IX: Making the SquashFS image
| Code: |
echo "==== MAKING SQUASHFS AND ISO ==============================================" >> ../finished/$VERSION/$VERSION-AUTOBUILD.log mksquashfs source.new/ target/source.img 2>> ../finished/$VERSION/$VERSION-AUTOBUILD.log |
You're nearly there. This line will take the contents of source.new/ and compress it into target/source.img using SquashFS file system. Perfect for a LiveCD. Next, make the ISO.
[edit] Part X: Making the ISO
| Code: |
mkisofs -R -b boot/grub/stage2_eltorito -no-emul-boot -boot-load-size 4 -boot-info-table -iso-level 4 -hide boot.catalog -o ../finished/$VERSION/FLY-$VERSION.iso target/ 2>> ../finished/$VERSION/$VERSION-AUTOBUILD.log e |
ADD TO THIS SECTION
[edit] Part XI: Finishing up
| Code: |
echo "==== GENERATING MD5SUM ====================================================" >> ../finished/$VERSION/$VERSION-AUTOBUILD.log cd ../finished/$VERSION/ md5sum FLY-$VERSION.iso >> $VERSION-AUTOBUILD.log md5sum FLY-$VERSION.iso > FLY-$VERSION.iso.md5 md5sum -c FLY-$VERSION.iso.md5 >> $VERSION-AUTOBUILD.log |
In order to verify the ISO during network transfers, you'll want to have an MD5 checksum file. Well, this is how to create it, verify it, and log it.
| Code: |
cp -rfp ../../build/{FILES/,FLYautobuild.sh} . 2>> $VERSION-AUTOBUILD.log
mv ../../build/{source*,target,installdata*} . 2>> $VERSION-AUTOBUILD.log
|
Next, move everything pertaining to this LiveCD to a new directory in finished/ with the version number. This way, all files pertaining to the LiveCD, from the scripts to the custom files, to log files, to the actual data is kept in one place.
| Code: |
echo "==== PASS OR FAIL =========================================================" >> $VERSION-AUTOBUILD.log failure=`grep "FAIL!" $VERSION-AUTOBUILD.log | wc -l` if [ "$failure" = "0" ]; then echo "Autobuild OK!" >> $VERSION-AUTOBUILD.log else echo "Autobuild FAIL! with $failure failure(s):" >> $VERSION-AUTOBUILD.log echo >> $VERSION-AUTOBUILD.log grep 'FAIL!' $VERSION-AUTOBUILD.log | grep -v Autobuild >> $VERSION-AUTOBUILD.log fi echo "==== END OF AUTOBUILD LOG =================================================" >> $VERSION-AUTOBUILD.log date >> $VERSION-AUTOBUILD.log |
Lastly, all those PASS or FAIL if statements are checked and if any of them returned a failure, they'll be listed here. This helps quickly identify if the autobuild script has completed correctly or not. Don't forget to check the log file for more detailed information though!
[edit] Additional Files
[edit] init
#!/bin/busybox sh
FindCD(){
CDROM=""
for x in hda hdb hdc hdd sda sdb sdc sdd sr0; do
mount -t iso9660 -r /dev/$x /cdrom > /dev/null 2>&1
if [ "$?" = "0" ]; then
echo " Press 'b' for initramfs shell"
read -n1 -t 1 b
if [ "$b" = "b" ]; then
echo "Use 'busybox --help' to view available commands."
sh
fi
id=`find /cdrom/ -name source.img`
if [ "$id" = "/cdrom/source.img" ]; then
break
else
umount /dev/$x
FindCD
fi
fi
done
}
mkdir cdrom new
FindCD
CDROM=$x
if [ "$CDROM" == "" ]; then
mkdir proc
mount -t proc none /proc
echo " This LiveCD didnt feel like booting!"
echo " Please paste the output that follows to http://bugs.linuxfly.net"
echo ""
/sbin/lspci -i /bin/pci.ids
sh
fi
mount -t squashfs -o loop /cdrom/source.img /new > /dev/null 2>&1
mount -t tmpfs -o size=8m none /new/etc > /dev/null 2>&1
mount -t tmpfs -o size=32m none /new/home > /dev/null 2>&1
mount -t tmpfs -o size=4m none /new/mnt > /dev/null 2>&1
mount -t tmpfs -o size=8m none /new/root > /dev/null 2>&1
mount -t tmpfs -o size=64m none /new/tmp > /dev/null 2>&1
mount -t tmpfs -o size=32m none /new/var > /dev/null 2>&1
cd /new/etc && tar xpf /cdrom/etc.tar.gz > /dev/null 2>&1
cd /new/home && tar xpf /cdrom/home.tar.gz > /dev/null 2>&1
cd /new/mnt && tar xpf /cdrom/mnt.tar.gz > /dev/null 2>&1
cd /new/root && tar xpf /cdrom/root.tar.gz > /dev/null 2>&1
cd /new/var && tar xpf /cdrom/var.tar.gz > /dev/null 2>&1
cd /new
mount --move . /
exec chroot . /sbin/init[edit]
default 0
timeout 5
foreground ffffff
background 0000ff
splashimage=/boot/grub/FLY.xpm.gz
# Remove initrd lines for a compile install
# Remember to modify the root= argument
title=FLY
kernel /boot/vmlinuz root=/dev/ram0 rw video=uvesafb:1024x768-32,mtrr:3,ywrap quiet CONSOLE=tty1 cdroot
initrd /boot/initramfs.igz
title=FLY (no framebuffer)
kernel /boot/vmlinuz root=/dev/ram0 rw quiet cdroot
initrd /boot/initramfs.igz
title=Memtest86+
kernel /boot/memtest86plus/memtest.bin[edit] /etc/fstab
# /etc/fstab: static file system information. # # See the manpage fstab(5) for more information. # # Uncomment and edit the lines below and remove # line /dev/loop0 for a compile install # <fs> <mountpoint> <type> <opts> <dump/pass> #/dev/BOOT /boot ext2 noauto,noatime 1 2 #/dev/SWAP none swap sw 0 0 #/dev/ROOT / ext3 noatime 0 1 /dev/loop0 / squashfs defaults,ro 0 0 none /proc proc defaults 0 0 none /dev/shm tmpfs defaults 0 0
[edit] /etc/make.conf
# These settings were set by the catalyst build script that automatically
# built this stage.
# Please consult /etc/make.conf.example for a more detailed example.
CFLAGS="-O2 -march=i686 -pipe"
CXXFLAGS="${CFLAGS}"
# This should not be changed unless you know exactly what you are doing. You
# should probably be using a different stage, instead.
CHOST="i686-pc-linux-gnu"
# During a COMPILE install, remove USE flags 'livecd' and 'minimal'
USE="X acpi alsa injection jpeg mmx mp3 pcmcia pdf png sse sse2 wifi win32codecs
-dbus -bitmap-fonts -type1-fonts -cups -pam -ipv6 livecd minimal"
MAKEOPTS="-j2"
AUTOCLEAN="yes"
PORTAGE_NICENESS="19"
INPUT_DEVICES="keyboard mouse synaptics evdev"
VIDEO_CARDS="fbdev"
LINGUAS="en"[edit] The Script
| Code: FLYautobuild.sh |
#!/bin/bash
# Copyright (c) 2010, Nicolas Roome
# All rights reserved.
#
# Redistribution and use in source and binary forms, with or without
# modification, are permitted provided that the following conditions are met:
# * Redistributions of source code must retain the above copyright
# notice, this list of conditions and the following disclaimer.
# * Redistributions in binary form must reproduce the above copyright
# notice, this list of conditions and the following disclaimer in the
# documentation and/or other materials provided with the distribution.
# * Neither the name of CriticalSecurity.NET nor the
# names of its contributors may be used to endorse or promote products
# derived from this software without specific prior written permission.
#
# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
# ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
# WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
# DISCLAIMED. IN NO EVENT SHALL NICOLAS ROOME BE LIABLE FOR ANY
# DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES
# (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES;
# LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND
# ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT
# (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS
# SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
# Instructions for use, FLYautobuild.sh script
#
# TYPE variable has two options: "NEW" or "UPDATE"
# NEW to build FLY from scratch
# UPDATE to update a previous version.
# Dont forget to edit the following file:
# /usr/share/slim/themes/FLY/slim.theme line 29
# Modify the following
VERSION="0.0.1"
AUTOBUILD="20100518"
ROOTPASSWD="toor"
FLYPASSWD="sharky"
TYPE="UPDATE"
# Modify only if $TYPE="UPDATE"
OLDVERSION="0.0.0"
# TYPE variable check
if [ "$TYPE" != "UPDATE" ] && [ "$TYPE" != "NEW" ]; then
echo "Wrong TYPE variable: $TYPE"
echo "Quitting ..."
exit
fi
# Do not modify the following
BUILD="/home/FLY/build/source/$VERSION-AUTOBUILD.log"
if [ "$TYPE" = "UPDATE" ]; then
NEWKERNELMD5=`md5sum /home/FLY/build/FILES/MAKE/usr/src/linux/.config | awk '{print $1}'`
OLDKERNELMD5=`md5sum /home/FLY/finished/$OLDVERSION/FILES/MAKE/usr/src/linux/.config | awk '{print $1}'`
if [ "$NEWKERNELMD5" = "$OLDKERNELMD5" ]; then
KERNEL="no"
KVERSION=`ls /home/FLY/finished/$OLDVERSION/source/lib/modules/`
else
KERNEL="yes"
fi
fi
# Start of FLYautobuild.sh
cd /home/FLY/
mkdir finished/$VERSION 2>> tmpAUTOBUILD.log
echo "==== START OF AUTOBUILD LOG ===============================================" >> tmpAUTOBUILD.log
date >> tmpAUTOBUILD.log
if [ "$TYPE" = "NEW" ]; then
mkdir -p build/source/{usr/portage,etc/portage,mnt/linuxfly.net} 2>> tmpAUTOBUILD.log
cd build/source 2>> tmpAUTOBUILD.log
mv ../../tmpAUTOBUILD.log $VERSION-AUTOBUILD.log
tar jxpf ../FILES/stage3-i686-$AUTOBUILD.tar.bz2 2>> $BUILD
else
mkdir build/source
mv tmpAUTOBUILD.log build/source/$VERSION-AUTOBUILD.log
rsync -aq --exclude={chroot,lib/modules/*,$VERSION-AUTOBUILD.log,date.stamp} finished/$OLDVERSION/source/ build/source/
cd build/source
fi
mount -t proc none proc/ 2>> $BUILD
mount --bind /dev dev/ 2>> $BUILD
mount --bind /sys sys/ 2>> $BUILD
mount -o ro --bind /usr/portage/ usr/portage/ 2>> $BUILD
if [ "$TYPE" = "NEW" ]; then
cp /etc/resolv.conf etc/resolv.conf 2>> $BUILD
cp -r ../FILES/MAKE . 2>> $BUILD
mv MAKE/etc/portage/* etc/portage/ 2>> $BUILD
mv MAKE/etc/{locale.gen,make.conf} etc/ 2>> $BUILD
date +%m%d%H%M%Y > date.stamp
else
echo "==== SYNCING NEW FILES ====================================================" >> $BUILD
rsync -aq --exclude={usr/src/,etc/{locale.gen,make.conf},portage/} /home/FLY/build/FILES/MAKE/* MAKE/ 2>> $BUILD
if [ "$KERNEL" = "no" ]; then
echo "Copying old kernel to current build ..." >> $BUILD
mkdir MAKE_oldkernel 2>> $BUILD
rsync -aq /home/FLY/finished/$OLDVERSION/source/lib/modules/ MAKE_oldkernel/ 2>> $BUILD
cp -f /home/FLY/finished/$OLDVERSION/source/usr/src/linux/arch/i386/boot/bzImage MAKE_oldkernel/bzImage 2>> $BUILD
fi
rsync -aq /home/FLY/build/FILES/MAKE/etc/portage/* etc/portage/ 2>> $BUILD
rsync -aq /home/FLY/build/FILES/MAKE/etc/{locale.gen,make.conf} etc/ 2>> $BUILD
cp -f ../FILES/MAKE/usr/src/linux/.config usr/src/.config 2>> $BUILD
fi
echo "==== WRITING CHROOT SCRIPT ================================================" >> $BUILD
cat > chroot << 'EOF'
#!/bin/bash
DATE=`cat /date.stamp`
VERSION=$1
AUTOBUILD=$2
ROOTPASSWD=$3
FLYPASSWD=$4
TYPE=$5
KERNEL=$6
KVERSION=$7
BUILD=/$VERSION-AUTOBUILD.log
env-update && source /etc/profile 2>> $BUILD
EOF
if [ "$TYPE" = "NEW" ]; then
cat >> chroot << 'EOF'
echo "==== SETTING THE DATE =====================================================" >> $BUILD
cp /usr/share/zoneinfo/Canada/Eastern /etc/localtime 2>> $BUILD
if [ "$?" = "0" ]; then echo "Localtime OK!" >> $BUILD; else echo "Localtime FAIL!" >> $BUILD; fi
date $DATE 2>> $BUILD
if [ "$?" = "0" ]; then echo "Set Date OK!" >> $BUILD; else echo "Set Date FAIL!" >> $BUILD; fi
echo "==== EMERGING REQUIRED EBUILDS ============================================" >> $BUILD
echo "Updating autobuild $AUTOBUILD tarball" >> $BUILD
emerge -uDNvp world >> $BUILD
emerge -uDNq world 2>> $BUILD
if [ "$?" = "0" ]; then echo "Updating OK!" >> $BUILD; else echo "Updating FAIL!" >> $BUILD; fi
echo -5 | update-etc 2>> $BUILD
echo "Required merges" >> $BUILD
emerge -vp expect sysklogd memtest86+ gentoolkit gentoo-sources livecd-tools localepurge slocate >> $BUILD
emerge -q expect sysklogd memtest86+ gentoolkit gentoo-sources livecd-tools localepurge slocate 2>> $BUILD
if [ "$?" = "0" ]; then echo "Required OK!" >> $BUILD; else echo "Required FAIL!" >> $BUILD; fi
echo "==== KERNEL AND MODULES ===================================================" >> $BUILD
mv /MAKE/usr/src/linux/.config /usr/src/linux/.config 2>> $BUILD
cd /usr/src/linux
make clean && make && make modules_install 2>> $BUILD
if [ "$?" = "0" ]; then echo "Kernel make OK!" >> $BUILD; else echo "Kernel make FAIL!" >> $BUILD; fi
cp arch/i386/boot/bzImage /boot/vmlinuz 2>> $BUILD
du -sh /lib/modules/* >> $BUILD
echo "==== EMERGING NEW EBUILDS =================================================" >> $BUILD
echo "0.0.1 merges" >> $BUILD
emerge -vp moo >> $BUILD
emerge -q moo 2>> $BUILD
if [ "$?" = "0" ]; then echo "0.0.1 OK!" >> $BUILD; else echo "0.0.1 FAIL!" >> $BUILD; fi
echo "Rebuild" >> $BUILD
revdep-rebuild -- -vp >> $BUILD
revdep-rebuild -- -q 2>> $BUILD
if [ "$?" = "0" ]; then echo "Rebuild OK!" >> $BUILD; else echo "Rebuild FAIL!" >> $BUILD; fi
echo "==== FINISHING AND CLEANING UP =============================================" >> $BUILD
echo -5 | etc-update 2>> $BUILD
echo "Adding / Removing startups" >> $BUILD
rc-update add autoconfig default >> $BUILD
rc-update del checkfs boot >> $BUILD
rc-update del checkroot boot >> $BUILD
echo "Running eix" >> $BUILD
eix-update 2>> $BUILD
eix -Ic >> $BUILD
eix-test-obsolete brief >> $BUILD
emerge --depclean -p >> $BUILD
emerge --depclean 2>> $BUILD
if [ "$?" = "0" ]; then echo "Depclean OK!" >> $BUILD; else echo "Depclean FAIL!" >> $BUILD; fi
echo "==== EMERGING GRUB ========================================================" >> $BUILD
emerge -vp grub >> $BUILD
emerge -q grub 2>> $BUILD
if [ "$?" = "0" ]; then echo "Grub OK!" >> $BUILD; else echo "Grub FAIL!" >> $BUILD; fi
echo "==== ADDING FLY ACCOUNT ===================================================" >> $BUILD
useradd -m -G wheel,users,audio fly 2>> $BUILD
if [ "$?" = "0" ]; then echo "Useradd OK!" >> $BUILD; else echo "Useradd FAIL!" >> $BUILD; fi
echo "==== INITRAMFS ===========================================================" >> $BUILD
mkdir -p /boot/initramfs/{sbin,bin,etc,lib,dev} 2>> $BUILD
mv /MAKE/boot/initramfs/init /boot/initramfs/init 2>> $BUILD
mv /MAKE/boot/v86d /boot/initramfs/sbin/v86d 2>> $BUILD
cp /usr/sbin/lspci /boot/initramfs/sbin/lspci 2>> $BUILD
touch /boot/initramfs/etc/{mtab,fstab} 2>> $BUILD
cd /bin
cp sh busybox /boot/initramfs/bin/ 2>> $BUILD
cp /usr/share/misc/pci.ids /boot/initramfs/bin/ 2>> $BUILD
cd /lib
#REQUIRED LIBS MAY CHANGE (use ldd <command>)
cp /usr/lib/libpci.so.3 libncurses.so.5 libdl.so.2 ld-linux.so.2 libc.so.6 libz.so.1 libresolv.so.2 /boot/initramfs/lib 2>> $BUILD
cd /boot/initramfs/dev
mknod console c 5 1 2>> $BUILD
mknod null c 1 3 2>> $BUILD
mknod hda b 3 0 2>> $BUILD
mknod hdb b 3 64 2>> $BUILD
mknod hdc b 22 0 2>> $BUILD
mknod hdd b 22 64 2>> $BUILD
mknod sr0 b 11 0 2>> $BUILD
mknod sda b 8 0 2>> $BUILD
mknod sdb b 8 16 2>> $BUILD
mknod sdc b 8 32 2>> $BUILD
mknod sdd b 8 64 2>> $BUILD
mknod tty c 4 0 2>> $BUILD
mknod loop0 b 7 0 2>> $BUILD
mknod tty1 c 4 1 2>> $BUILD
mknod mem c 1 1 2>> $BUILD
mknod zero c 1 5 2>> $BUILD
mkdir loop && cd loop 2>> $BUILD
ln -s ../loop0 0 2>> $BUILD
cd ../..
find . | cpio -H newc -o > ../initramfs.cpio
cd ..
cat initramfs.cpio | gzip > initramfs.igz
EOF
else
cat >> chroot << 'EOF'
echo "==== UPDATING WORLD =======================================================" >> $BUILD
emerge -uDNvp world >> $BUILD
emerge -uDNq world 2>> $BUILD
if [ "$?" = "0" ]; then echo "Updating OK!" >> $BUILD; else echo "Updating FAIL!" >> $BUILD; fi
echo "==== KERNEL AND MODULES ===================================================" >> $BUILD
KAWK=`awk '/sys-kernel\/gentoo-sources/ && /ebuild/' $BUILD`
if [ "$KERNEL" = "no" ] && [ "$KAWK" = "" ]; then
echo "Kernel compile not required. Using old kernel." >> $BUILD
mv /MAKE_oldkernel/bzImage /boot/vmlinuz 2>> $BUILD
mv /MAKE_oldkernel/$KVERSION /lib/modules/ 2>> $BUILD
rm -rf /MAKE_oldkernel 2>> $BUILD
else
echo "Kernel compile required. Building new kernel." >> $BUILD
cd /usr/src/linux
mv ../.config .config 2>> $BUILD
make clean && make && make modules_install 2>> $BUILD
if [ "$?" = "0" ]; then echo "Kernel make OK!" >> $BUILD; else echo "Kernel make FAIL!" >> $BUILD; fi
cp -v arch/i386/boot/bzImage /boot/vmlinuz 2>> $BUILD
fi
du -sh /lib/modules/* >> $BUILD
echo "==== EMERGING NEW EBUILDS =================================================" >> $BUILD
emerge -vp moo >> $BUILD
emerge -q moo 2>> $BUILD
if [ "$?" = "0" ]; then echo "$VERSION OK!" >> $BUILD; else echo "$VERSION FAIL!" >> $BUILD; fi
echo "==== FINISHING AND CLEANING UP ============================================" >> $BUILD
echo -5 | etc-update 2>> $BUILD
echo "Running eix" >> $BUILD
eix -Ic >> $BUILD
eix-test-obsolete brief >> $BUILD
emerge --depclean -p >> $BUILD
emerge --depclean 2>> $BUILD
if [ "$?" = "0" ]; then echo "Depclean OK!" >> $BUILD; else echo "Depclean FAIL!" >> $BUILD; fi
EOF
fi
cat >> chroot << 'EOF'
echo "==== USER ADMINISTRATION ==================================================" >> $BUILD
echo "Changing root password ..." >> $BUILD
expect << INEOF 2>> $BUILD
spawn passwd
expect "New password:"
send "$ROOTPASSWD\r"
expect "New password:"
send "$ROOTPASSWD\r"
expect "Re-type new password:"
send "$ROOTPASSWD\r"
expect eof;
INEOF
echo "Changing fly password ..." >> $BUILD
expect << INEOF 2>> $BUILD
spawn passwd fly
expect "New password:"
send "$FLYPASSWD\r"
expect "New password:"
send "$FLYPASSWD\r"
expect "Re-type new password:"
send "$FLYPASSWD\r"
expect eof;
INEOF
echo "==== MOVING FILES AROUND ==================================================" >> $BUILD
echo "Moving etc ..." >> $BUILD
mv /MAKE/etc/X11/* /etc/X11/ 2>> $BUILD
mv /MAKE/etc/privoxy/* /etc/privoxy/ 2>> $BUILD
mv /MAKE/etc/conky/* /etc/conky/ 2>> $BUILD
mv /MAKE/etc/conf.d/* /etc/conf.d/ 2>> $BUILD
mv /MAKE/etc/init.d/* /etc/init.d/ 2>> $BUILD
cp /MAKE/etc/* /etc/ 2>> $BUILD
echo "Moving the remainder ..." >> $BUILD
mv /MAKE/boot/grub/* /boot/grub/ 2>> $BUILD
mv /MAKE/home/fly/.* /home/fly/ 2>> $BUILD
mv /MAKE/root/.* /root/ 2>> $BUILD
mv /MAKE/sbin/* /sbin/ 2>> $BUILD
mv /MAKE/usr/sbin/* /usr/sbin/ 2>> $BUILD
mv /MAKE/usr/share/slim/themes/* /usr/share/slim/themes/ 2>> $BUILD
echo "==== MISC =================================================================" >> $BUILD
rm -r /usr/share/slim/themes/default /mnt/cdrom /etc/init.d/net.eth0 2>> $BUILD
cp /etc/tor/torrc.sample /etc/tor/torrc 2>> $BUILD
localepurge 2>> $BUILD
makewhatis -u 2>> $BUILD
find / -type f -name ".keep" -print -exec rm {} \; 2>> $BUILD
cat /proc/mounts > /etc/mtab 2>> $BUILD
updatedb 2>> $BUILD
# As of this version, and Audacious version 1.5.1 which pulls in dbus,
# This command must be run to prevent Audacious from crashing
dbus-uuidgen --ensure 2>> $BUILD
# Making the tarball for the installer
cd /
tar cjfp installdata-$VERSION.tar.bz2 {var/{db,cache},usr/{lib/portage/,portage/profiles/,share/zoneinfo/},etc/portage/package.*} 2>> $BUILD
echo "==== CHOWN AND CHMOD ======================================================" >> $BUILD
chown fly:fly /home/fly/ -R 2>> $BUILD
if [ "$?" = "0" ]; then echo "01 of 01 OK!" >> $BUILD; else echo "01 of 01 FAIL!" >> $BUILD; fi
exit
EOF
chmod 700 chroot 2>> $BUILD
echo "==== START OF CHROOT ======================================================" >> $BUILD
chroot . /bin/bash --login ./chroot $VERSION $AUTOBUILD $ROOTPASSWD $FLYPASSWD $TYPE $KERNEL $KVERSION
echo "==== END OF CHROOT ========================================================" >> $BUILD
env-update && source /etc/profile 2>> $BUILD
rm -rf MAKE 2>> $BUILD
umount -f sys dev usr/portage /home/FLY/build/source/proc 2>> $BUILD
cd ..
mv source/installdata* . 2>> $BUILD
rsync -aq --exclude={$VERSION-AUTOBUILD.log,date.stamp,usr/{portage,src,local,i486-pc-linux-gnu,share/{doc,gtk-doc,man},lib/{portage,gcc/i486-pc-linux-gnu}},etc/portage,var/{tmp/*,run/*,lock/*,lib/clamav/*},tmp/*} source/ source.new/ 2>> $BUILD
cd source.new
rm -rf boot/{initramfs,initramfs.cpio,boot} etc/mtab root/.bash_history chroot var/db var/cache 2>> $BUILD
touch etc/mtab 2>> $BUILD
mkdir ../target/ 2>> $BUILD
cp -a boot ../target/ 2>> $BUILD
echo "==== MAKING TARBALLS ======================================================" >> $BUILD
cd etc/
tar cpzf ../../target/etc.tar.gz * .[[:alnum:]]* 2>> $BUILD
rm -rf * .* 2>> $BUILD
cd ../home/
tar cpzf ../../target/home.tar.gz * .[[:alnum:]]* 2>> $BUILD
rm -rf * 2>> $BUILD
cd ../mnt/
tar cpzf ../../target/mnt.tar.gz * .[[:alnum:]]* 2>> $BUILD
rm -rf * 2>> $BUILD
cd ../root/
tar cpzf ../../target/root.tar.gz * .[[:alnum:]]* 2>> $BUILD
rm -rf * .* 2>> $BUILD
cd ../var/
tar cpzf ../../target/var.tar.gz * .[[:alnum:]]* 2>> $BUILD
rm -rf * 2>> $BUILD
cd ../..
mv source/$VERSION-AUTOBUILD.log ../finished/$VERSION/
echo "==== MAKING SQUASHFS AND ISO ==============================================" >> ../finished/$VERSION/$VERSION-AUTOBUILD.log
mksquashfs source.new/ target/source.img 2>> ../finished/$VERSION/$VERSION-AUTOBUILD.log
mkisofs -R -b boot/grub/stage2_eltorito -no-emul-boot -boot-load-size 4 -boot-info-table -iso-level 4 -hide boot.catalog -o ../finished/$VERSION/FLY-$VERSION.iso target/ 2>> ../finished/$VERSION/$VERSION-AUTOBUILD.log
echo "==== GENERATING MD5SUM ====================================================" >> ../finished/$VERSION/$VERSION-AUTOBUILD.log
cd ../finished/$VERSION/
md5sum FLY-$VERSION.iso >> $VERSION-AUTOBUILD.log
md5sum FLY-$VERSION.iso > FLY-$VERSION.iso.md5
md5sum -c FLY-$VERSION.iso.md5 >> $VERSION-AUTOBUILD.log
cp -rfp ../../build/{FILES/,FLYautobuild.sh} . 2>> $VERSION-AUTOBUILD.log
mv ../../build/{source*,target,installdata*} . 2>> $VERSION-AUTOBUILD.log
echo "==== PASS OR FAIL =========================================================" >> $VERSION-AUTOBUILD.log
failure=`grep "FAIL!" $VERSION-AUTOBUILD.log | wc -l`
if [ "$failure" = "0" ]; then
echo "Autobuild OK!" >> $VERSION-AUTOBUILD.log
else
echo "Autobuild FAIL! with $failure failure(s):" >> $VERSION-AUTOBUILD.log
echo >> $VERSION-AUTOBUILD.log
grep 'FAIL!' $VERSION-AUTOBUILD.log | grep -v Autobuild >> $VERSION-AUTOBUILD.log
fi
echo "==== END OF AUTOBUILD LOG =================================================" >> $VERSION-AUTOBUILD.log
date >> $VERSION-AUTOBUILD.log
|
