Initramfs

From Gentoo Linux Wiki

initramfs is a root filesystem which is embedded into the kernel and loaded at an early stage of the boot process. It is the successor of initrd. It provides early userspace which lets you do things that the kernel can't easily do by itself during the boot process.

Using initramfs is optional. By default, the kernel initializes hardware using built-in drivers, mounts the specified root partition, loads the init system of the installed Linux distribution. The init system then loads additional modules and starts services until it finally allows you to log in. This is a good default behaviour and sufficient for many users. initramfs is for users with advanced requirements, for users who need to do things as early as possible, before the root partition is mounted.

Here are some examples of what you can do with initramfs:

• Customize the boot process (e.g. print a welcome message, boot splash, ...)
• Load modules (e.g. a third party driver that can not be integrated into the kernel directly)
• Mount the root partition (for encrypted, logical, network and otherwise special partitions)
• Provide a minimalistic rescue shell (if something goes wrong)
• Anything the kernel can't do (as long as you can do it in user space, e.g. by executing commands)

If you don't have advanced requirements, you do not need initramfs.

[edit] Prerequisites

There are countless ways to make an initramfs. You can choose not to create an initramfs at all but let other apps, such as genkernel, do the work for you. If you are lucky, genkernel does what you want out of the box, and you don't need to bother with how initramfs works and what it does anymore. If you're unlucky, genkernel does not do what you want and you have to build an initramfs all by yourself.

The initramfs contains at the very least of one file, /init. This file is executed by the kernel as the main init process (PID 1). It has to do all the work. In addition, there can be any number of additional files and directories that are required by /init. They are usually files you will also find on any other root filesystem, such as /dev for device nodes, /proc for kernel information, /bin for addition binaries, and so on. The structure of the initramfs can be simple, or it can be complicated, depending on what you are planning to do.

When the kernel mounts the initramfs, your physical root partition is not yet mounted, so you can't access any of your files. That means there is nothing but the initramfs. So everything you need, everything you want, you need to include it in your initramfs. If you want a shell, you have to include it in your initramfs. If you want to mount something, you need a mount utility. If you need to load a module, your initramfs has to provide both the module, as well as a utility to load it. If the utility depends on libraries in order to work, you have to include the libraries as well. This seems complicated, and it is, because the initramfs has to function independently.

[edit] Kernel Configuration

You need to enable support for Initial RAM filesystem and RAM disk (initramfs/initrd) support for the initramfs functionality.

General setup  --->
    [*] Initial RAM filesystem and RAM disk (initramfs/initrd) support

[edit] Basics

In this section you will learn the easy and straightforward way to initramfs creation. You will make a functional - albeit minimalistic - initramfs which you then can extend according to your own requirements.

[edit] Directory Structure

Create the directory that will later become your initramfs root. For consistency. we'll work in /usr/src/initramfs, but any directory would do. Create initramfs root directory and cd into it.

Create a basic directory layout.

[edit] Applications

Any binary you want to execute at boot needs to be copied into your initramfs layout. You also need to copy any libraries that your binaries require. To see what libraries any particular binary requires, use the tool ldd. An example examining what libraries app-text/nano requiries:

    linux-gate.so.1 =>  (0xb7f8a000)
    libncursesw.so.5 => /lib/libncursesw.so.5 (0xb7f2e000)
    libc.so.6 => /lib/libc.so.6 (0xb7dbb000)
    /lib/ld-linux.so.2 (0xb7f8b000)

Here you see that for app-text/nano to work in our initramfs, we not only need to copy /usr/bin/nano to /usr/src/initramfs/bin, but also /lib/libncursesw.so.5, /lib/libc.so.6 and /lib/ld-linux.so.2 to /usr/src/initramfs/lib. Note that you don't need the linux-gate.so.1.

[edit] Busybox

Instead of collecting countless utilities and libraries (and never seeing the end of it), you can just use sys-apps/busybox. It's a set of utilities for rescue and embedded systems, it contains a shell, utilities like ls, mkdir, cp, mount, insmod, and many more - all in a single binary called /bin/busybox. For busybox to work properly in a initramfs you'll firstly need to emerge sys-apps/busybox with the static USE flag enabled, then copy the /bin/busybox binary into your initramfs layout as /usr/src/initramfs/bin/busybox:

[edit] Init

The file structure of your initramfs is almost complete. The only thing that is missing is /init, the executable in the root of the initramfs that is executed by the kernel once it is loaded. Because sys-apps/busybox includes a fully functional shell this means you can write your /init binary as a simple shell script (instead of making it a complicated application written in Assembler or C that you have to compile).

The following example realizes this executable as a minimalistic shell script, based on the busybox shell:

#!/bin/busybox sh

# Mount the /proc and /sys filesystems.
mount -t proc none /proc
mount -t sysfs none /sys

# Do your stuff here.
echo "This script mounts rootfs and boots it up, nothing more!"

# Mount the root filesystem.
mount -o ro /dev/sda1 /mnt/root

# Clean up.
umount /proc
umount /sys

# Boot the real thing.
exec switch_root /mnt/root /sbin/init

This example needs some device nodes to work, mainly the root block device, as exampled:

Change the script and copy the the corresponding file from /dev/ to fit your needs.

Lastly, make the /init executable:

[edit] Packaging Your Initramfs

Your initramfs now has to be packaged as a gzip compressed cpio archive. This archive is then either embedded directly into your kernel image, or stored as a separate file which can be loaded by grub during the boot process. Both methods perform equally well, simply choose the method that you find most convenient.

[edit] Embedding into the Kernel

Edit your kernel config and set Initramfs source file(s) to the root of your initramfs,(e.g /usr/src/initramfs):

General setup  --->
    (/usr/src/initramfs) Initramfs source file(s)

Now when you compile your kernel it will automatically compress the files into a cpio archive and embed it into the bzImage. You will need to recompile the kernel any time you make any changes to your initramfs.

[edit] Creating a Separate File

You can create a standalone archive file by running the following commands:

This will create a file called my-initramfs.cpio.gz in your /boot directory. You now need to instruct grub to load this file at boot time, you do this with the initrd line.

title=My Linux
root (hd0,0)
kernel /boot/my-kernel
initrd /boot/my-initramfs.cpio.gz

It is possible to specify multiple initramfs to be extracted during boot. This can be useful if you want to create a generic initramfs and then add modifications in separate files. You can specify multiple initramfs archives in your grub config by using multiple initrd lines.

title=Gentoo Linux 2.6.27-r8 with initramfs (hd0,0)
root (hd0,0)
kernel /boot/kernel-2.6.27-gentoo-r8
initrd /boot/first-initramfs.cpio.gz
initrd /boot/second-initramfs.cpio.gz
initrd /boot/third-initramfs.cpio.gz

The kernel will extract these files in the order they are specified. The files will all be extracted into the same place, this means files from later archives will overwrite former ones if they have the same filename. Also note that it is possible to have an initramfs archive embedded in the kernel as well as extra ones specified in the grub config. The archive in the kernel will be extracted first, followed by the ones in the grub config.

[edit] Extracting

If you ever need to extract the initramfs archive manually, then you can do so with the following commands. Please note that this overwrites files in the directory you're currently in:

[edit] Finalizing

You can now reboot your machine. On boot, the kernel will extract the files from your initramfs archive automatically and execute your init script, which in turn should then take care of mounting your root partition and exec the init of your installed Linux distribution.

[edit] Functionality

[edit] Rescue shell

If you want to be dropped to a rescue shell if an error occurs, you can add the following function to your /init and call it when something goes wrong.

rescue_shell() {
    echo "Something went wrong. Dropping you to a shell."
    busybox --install -s
    exec /bin/sh
}

This can, for example, be used to drop you into a shell if something goes wrong. In the Example below we drop to the rescue_shell if the root partition fails to mount:

mount -o ro /dev/sda1 /root || rescue_shell

[edit] Telnet server (remote rescue shell)

Starting a telnet server and then shelling in remotely is another good way to give you a rescue shell. This is especially useful if you don't have physical access to the machine that's booting, or if it doesn't have a monitor/keyboard.

First you will need to populate the /etc/passwd, /etc/shadow and /etc/group files in your initramfs. Then you can include the following snippet in your init script. Make sure you change the IP address to one that you can access.

remote_rescue_shell() {
    echo "Something went wrong. Starting telnet server."
    busybox --install -s

    # start network interface.
    # NOTE: Change the IP address to something you can access.
    ifconfig eth0 10.1.1.50 up

    # telnetd requires devpts to be mounted.
    mkdir /dev/pts
    mount -t devpts devpts /dev/pts
    
    telnetd

    # Start a local shell as well.
    exec /bin/sh

    # Alternatively you can start the server via netcat.
    # The following command will launch the telnet server for 300 seconds and then automatically reboot if no connection has been made.
    # nc -l -p 23 -w 300 -e /sbin/telnetd -i
    # echo "Rebooting..."
    # reboot -f
}

This can be used just like the rescue_shell if something goes wrong. In the Example below we drop to the remote_rescue_shell if the root partition fails to mount:

mount -o ro /dev/sda1 /root || remote_rescue_shell

All the commands (e.g. ifconfig, telnetd and nc) in the above snippet are included in busybox, so you don't need to add anything extra.

[edit] mdev

If you have to work with dynamic devices during init, you can initialize mdev, the udev replacement of busybox. Please note that this slows down the boot process, as mdev creates nodes for all devices, not just the ones you need for booting. In case of external devices, you may also have to add a sleep statement to your script, as the kernel can take some time detecting these devices properly.

mini_udev() {
    echo /sbin/mdev > /proc/sys/kernel/hotplug
    mdev -s
}

Example invocation, note that /proc and /sys must be mounted before mini_udev is run:

mini_udev

[edit] LVM

If your root partition is located on a logical volume, you need to include a static LVM binary in your initramfs. You can get a static binary by enabling the static USE-Flags for sys-fs/lvm2. Copy it to your initramfs sbin/ directory.

Now you can enable your LVM partitions in /init.

lvm vgscan --mknodes
lvm vgchange -a y

Your root partition may then be called /dev/lvm/root depending on the name you choose for your root volume.

[edit] DM-Crypt

If your root partition is encrypted, you need to include a static cryptsetup binary in your initramfs. You can get a static binary by setting the -dynamic USE-Flag for sys-fs/cryptsetup. Copy it to your initramfs sbin/ directory:

Now you can unlock your encrypted root partition in /init:

cryptsetup -T 5 luksOpen /dev/sda1 luks

Your root partition would then be called /dev/mapper/luks.

[edit] Software RAID

Normally the Linux kernel will automatically scan for any "Linux raid autodetect" partitions (type: fd) and start as many software RAIDs as it can find. But if you use an initramfs the kernel will not automatically scan for RAIDs until it is told to. In the following example we instruct the kernel to scan for software RAIDs and start as many as it can find ,this will actually start all arrays, not just /dev/md0:

busybox raidautorun /dev/md0

[edit] mdadm

If you are not using "Linux raid autodetect" partitions, or need to do more advanced RAID setup, then you can use mdadm instead. You will need to include a static mdadm binary in your initramfs. You can get a static binary by enabling the static USE-Flag for sys-fs/mdadm.

Copy the binary(/sbin/mdadm) and your /etc/mdadm.conf into your initramfs:

Edit your the mdadm.conf in your initramfs to your liking. An example mdadm.conf follows:

# mdadm configuration file

DEVICE /dev/sd??*

ARRAY /dev/md0 UUID=57b62805:84968c87:2222d7c7:02df0843

This mdadm.conf will scan all /dev/sd* devices and assemble the RAID device fitting the UUID 57b62805:84968c87:2222d7c7:02df0843. You get the UUID by running:

Now you can initialize your software RAIDs in /init:

mdadm --assemble --scan /dev/md0

[edit] Bootsplash

See fbsplash.

[edit] UUID/LABEL Root Mounting

If you want to be able to set root on the kernel command line with either a LABEL or the UUID you'll need to add that parsing functionality to your initramfs, note that this depends on mdev:

uuidlabel_root() {
    for cmd in $(cat /proc/cmdline) ; do
        case $cmd in
        root=*)
            type=$(echo $cmd | cut -d= -f2)
            if [ $type == "LABEL" ] || [ $type == "UUID" ] ; then
                uuid=$(echo $cmd | cut -d= -f3)
                mount -o ro $(findfs "$type"="$uuid") /mnt/root
            else
                mount -o ro $(echo $cmd | cut -d= -f2) /mnt/root
            fi
            ;;
        esac
    done
}

It's a good idea to invoke this with the rescue_shell. Also make sure you don't have any other mount lines set to mount the root filesystem in your /init:

uuidlabel_root || rescue_shell

This snippet will now allow for grub.conf entries like:

title My Linux
kernel /my-kernel root=LABEL=Gentoo

Or, using UUIDs:

title My Linux
kernel /my-kernel root=UUID=7364896a-ed4d-41ec-a08a-8e010be61446

The standard /dev/sd??* arguments will still work.

[edit] Kernel Modules

If you want to have your initramfs load kernel modules you'll need to check for modules dependencies, copy the required kernel modules to the initramfs and have the init script load them. Start with checking what dependencies the modules has, in this example we want to load the ext4.ko module(EXT4 Filesystem support). This we do with modinfo and by grepping for depends:

Take note of the $(uname -r) here, you can of course substitute this with whatever kernel version intend to use. The output should be similar to:

depends:        mbcache,jbd2

Create the /usr/src/initramfs/lib/modules directory:

Here we make our lives a little easier by using find to search and then copy the module and its dependencies to the initramfs:

Lastly we need to tell the init script to load the modules:

load_modules() {
    MODULES="mbcache.ko jbd2.ko ext4.ko"
    MOD_PATH="/lib/modules"
    for MODULE in ${MODULES} ; do
        insmod -f ${MOD_PATH}/${MODULE}
    done
}

Make sure the MOD_PATH points to the directory containing your modules and the MODULES variable contains the modules you want to load expressed by the modinfo output, in order, ending with wanted module. It's often a good idea to invoke bits like this with the rescue_shell to fall back on:

load_modules || rescue_shell

[edit] Troubleshooting

[edit] Static vs. Dynamic binaries

Any custom binaries you need to use in your Initramfs before mounting have to be fully functional, independently from any files you may have installed on your root partition. This is much easier to achieve with static binaries (which usually work as single file) than with dynamic binaries (which need any number of additional libraries to work).

Gentoo provides static binaries for some ebuilds. Check if the ebuild for your binary offers a static or -dynamic USE flag. That's by far the easiest method to get a static binary for something, but unfortunately only a select few ebuilds support it.

Many applications also offer static builds with an option in their configure scripts. There is no standard name for the option, it may be --enable-static or similar. When compiling a package manually, check the list of available options with ./configure --help to see if it supports building static binaries for you.

You can check wether or not a binary is static by using the ldd command. For static binaries, it will simply say that it's not a dynamic executable. Otherwise it will show a list of libraries, for example:

       linux-gate.so.1 =>  (0xffffe000)
       librt.so.1 => /lib/librt.so.1 (0xb7ee8000)
       libacl.so.1 => /lib/libacl.so.1 (0xb7ee0000)
       ...

Including libraries into your Initramfs in order to make a dynamic executable work is a last resort only. It makes the initramfs much larger and more complicated to maintain than necessary.

Sometimes even static binaries also have dependencies on other files or libraries that are not obvious at first glance. In those cases you can use strace to investigate in more detail why a binary is failing, when it does not give you a good error message.

[edit] Kernel panics

When working with initramfs and writing custom init scripts for it, you may experience the following kernel panic on boot:

 Kernel panic - not syncing: Attempted to kill init!

This is not an error in the kernel, but an error in your /init script. This script is executed as the init process with PID 1. Unlike other processes, the PID 1 init process is special. It is the only process that is started by the kernel on boot. It's the process that takes care of starting other processes (boot process, init scripts) which in turn start other processes (daemons, login prompts, X), which in turn start other processes (bash, window manager, browser, ...). The init process is the mother of all other processes, and therefore it mustn't be killed. On shutdown, it's again the init process that takes care of cleaning up by shutting down other processes first, then running processes that will unmount the filesystems, until it is safe to actually do a shutdown without corrupting anything.

If you have some error in your init script, that causes the init process to end, this basically means there are no processes left to run, there is nothing that could take care of cleaning up, and the kernel has no choice but to panic. For this reason there are some things in /init that you can't do like you can do them in a normal shell script, like using return or exit, or letting the script just run a series of commands and then simply end.

If you want /init to end, you have to pass the responsibility of the init process to someone else using exec. See the examples above how exec is used to either run /sbin/init of the mounted root partition or to run a rescue shell in case something went wrong.