Portage TMPDIR on tmpfs

During emerge, you can build packages in tmpfs (aka 'ram disk' or 'virtual memory file system'), instead of having massive intermediate, temporary files on an HDD. Let RAM replace HDD.

Reduction of compilation time may very well not be in any way significant. The usefulness of TMPDIR on a tmpfs is often discussed, such as on this forum thread.

This page also describes some basic aspects of tmpfs itself.

[edit] Preparation

[edit] Kernel config

You need to have a kernel with swap and Virtual memory file system support enabled. Almost all default kernel configurations have these enabled.

File systems --->
    Pseudo filesystems ---> 
        [*] Virtual memory file system support (former shm fs)

[edit] fstab

Add the tmpfs entry to /etc/fstab,

tmpfs      /var/tmp/portage   tmpfs  size=2G         0 0

[edit] Size caveat

The size is very important, and fully explained here.

Tmpfs should be big enough, because once it gets full, emerge fails at that point. Probably you can't resume it, since files in tmpfs may be killed by oom killer, just like processes.

If you don't specify the tmpfs size, it'll be half of your physical RAM. It's merely the limit; only the actually used size occupies the RAM. Tmpfs grows and shrinks dynamically, so to say.

Swap can be used, so tmpfs can be bigger than the physical RAM. However, if the use of tmpfs exceeds the half of physical RAM, it slows down considerably. [1] [2]

If you have parallel merge, take 1 or 2GB. For single merge most packages suffice with 500MB, but some packages need huge area, for example:

• mail-client/mozilla-thunderbird-3.0: More than 2GB.
• sys-devel/gcc-4.3.2-r3 or kde-base/kdelibs-4.2.4: more than 1GB to compile (1.8GB with the doc USE flag enabled).
• app-office/openoffice: assume 10G or so.

[edit] Per-package choice for tmpfs / HDD

There're two methods to switch between tmpfs and HDD per packages.

[edit] By portage configuration

In this method, the packages you don't want to use tmpfs are specified in /etc/portage/package.env. This requires portage >= 2.1.9.

First create the following file in which you tell portage where to place the temporary files directory portage instead of default /var/tmp.

PORTAGE_TMPDIR="/var/tmp/notmpfs"

Next create the file in which you will list all the packages that need some special environment variable settings:

app-office/libreoffice notmpfs.conf
mail-client/mozilla-thunderbird notmpfs.conf

And you are done! Portage will now use default settings for all packages except for ones listed in /etc/portage/package.env.

(This section is adapted from the blog post by Gentoo developer Jeremy Olexa which is available under CC-BY-SA license.)

[edit] Script to use tmpfs temporarily

This method is the opposite. You don't use tmpfs by default, but the script below automatically does all to emerge with tmpfs temporarily. This doesn't need the line in /etc/fstab. It will

1. checks if /var/tmp/portage is already mounted (then quits, because it would confuse the other emerge process)
2. mounts tmpfs to /var/tmp/portage
3. runs emerge
4. unmounts /var/tmp/portage

#!/bin/bash
MEMSIZE=512M
mounted=false

. /etc/make.conf
. /etc/init.d/functions.sh

if [ -z "$PORTAGE_TMPDIR" ]; then
  PORTAGE_TMPDIR="/var/tmp/portage"
fi


mounttmpfs() {
                mount -t tmpfs -o size=$MEMSIZE,nr_inodes=1M tmpfs ${PORTAGE_TMPDIR}
                mounted="true"
}

compile() {
        einfo "running emerge ${*}"
                emerge ${*}
}

unmount() {
        ebegin "unmounting tmpfs"
                umount -f ${PORTAGE_TMPDIR}
        eend $?
}

ebegin "Mounting $MEMSIZE of memory to ${PORTAGE_TMPDIR}"
        if [ -z "$(pgrep -f /usr/bin/emerge)" ];then
                if [ -z "$(mount | grep ${PORTAGE_TMPDIR})" ];then
                     mounttmpfs
                else
                     eerror "tmpfs already mounted!"
                     exit 0
                fi
        else
                eerror "emerge already running!"
                exit 0
        fi
eend $?

# the next line would change the cpu-governour, if available, to the highest frequency
#[ -f /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor ] && echo performance > /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor

# run emerge
compile $@

# unmount tmpfs
$mounted && umount ${PORTAGE_TMPDIR}

# and set the scheduler back to "ondemand"
#[ -f /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor ] && echo ondemand > /sys/devices/system/cpu/cpu0/cpufreq/scaling_governor

[edit] See Also

• Portage with autofs