Search:  
Gentoo Wiki

HOWTO_Small_Footprint_Gentoo_on_USB

Contents

Overview

The idea was to create a platform for image-processing; since I'm pretty original, I decided to name the platform: ipp (Image Processing Platform). The ipp needs to have USB and gphoto support to run a couple digital cameras, and ethernet so that it can talk to the outside world - but not much else. It seemed a good idea to put the whole system on a USB-stick, cause then we'll have no moving parts. To do so meant we needed to create an entire filesystem on a host and then move it over - there just wasn't enough space on the USB-stick to do the compile and install on the target. Thankfully the gentoo people have created a nice tool to do just that: Catalyst.

Catalyst is a tool provided by gentoo to make bootable CD's. It can also create any bootable file system (FS). Our USB sticks are just that: a bootable FS. So if we restrict the packages we install, we should be able to get the FS down to under 512MB, the physical size of the disk.

What I did

Most of this document was written in retrospect; this is how I would have done it had I the proper foresight. That being said: you can learn from my mistakes...

Let's start by making a little directory structure in which to store the files for the ipp. I reasoned that there are three major components to the ipp:

~/ipp_files
 +-/rootfs
 +-/kernel
 +-/configs

Create the bootable USB-stick

This was completed first because it was necessary to experiment with various kernel versions to ensure that we had a useable kernel before going ahead and creating the root FS with catalyst. One could easily leave this step until last if they knew they had a useable kernel.

When a USB stick is inserted, Linux will recognize it as a SCSI hdd. Newer kernels recognize it as a usb block device /dev/uba. However, for this tutorial we'll continue as though it were /dev/sda. Let's treat the USB stick like a regular hardrive and create two partitions on it: a boot partition and a root partition: First clear the partition table from the stick:

Warning: If you have other SCSI devices in your computer (like SATA drive), make absolutely sure that /dev/sda is actually the USB stick device (or change the shown commands accordingly). If you don't do so, you'll instantly lose all of your hd's content! time to say ***make backups***
dd if=/dev/zero of=/dev/sda bs=512 count=1

Then we'll create our partitions using #fdisk /dev/sda:

Code: fdisk commands
:n       # create new partition (for /boot)
:p       # primary
:1       # first
:        # press enter to accept the default value for the beginning of the partition
:+25M    # 25 MiB in size

:n       # another new partion (for /)
:p       # primary
:2       # second
:        # press enter to accept the default value for the beginning of the partition
:        # press enter to accept the default value for the end of the partition (= full size)
:w       # write the new partition table to the USB and exit

Pretty cryptic, but that will create a /dev/sda1 as 25MB partition and /dev/sda2 with the remaining space.

Our partitions are created. now we'll reload the table and create the filesystems:

rmmod usb-storage
modprobe usb-storage
mke2fs /dev/sda1
mkreiserfs /dev/sda2

I chose ext2 and reiserfs - though I think JFS is more appropriate for solid-state hardware.

We haven't created the final root FS yet, but we can apply a preliminary one from a stage3 tarball. Use the same stage3 that will be used in the creation of the root FS with catalyst:

mount /dev/sda2 /mnt/ipp
mkdir /mnt/ipp/boot
mount /dev/sda1 /mnt/ipp/boot
cd /mnt/ipp
tar xvjpf stage3-xxx.tar.bz2

The stage3 tarball does not contain the grub stages; luckily our target platform is compatible with the host. So we just copy the contents of our hosts boot dir with cp -pr /boot/* /mnt/ipp/boot

Then use GRUB to setup the MBR on the stick:

Code: GRUB Commands
>root (hd2,0)
>setup (hd2)

On my development system (the system I used to create the FS, not the target machine) the usb stick showed as hd2 in GRUB. Make sure you find the right hd#, you don't want to clobber your current MBR.

Then configure GRUB's menu.lst:

File: /mnt/ipp/boot/grub/menu.lst
# Boot automatically after 5 secs.
timeout 5
# By default, boot the first entry.
default 0
# Fallback to the second entry.
fallback 1

splashimage=(hd0,0)/grub/splash.xpm.gz

title=Test: IPP: 2.4.26
root (hd0,0)
kernel /test.ipp.kernel-2.4.26 root=/dev/sda2 vga=791 rootwait

So I told GRUB to use a kernel that isn't there - it doesn't matter until we try to boot with this thing. So don't try to boot with it yet.

Create a useable kernel

Hopefully you know what kernel works, or have a kernel with the features you require on a host machine. On the host, make a copy of the kernel source tree:

mkdir ~/ipp_files/kernel/linux-2.4.26
cd ~/ipp_files/kernel/linux-2.4.26
rsync -aH /usr/src/linux/ .

Then proceed to modify the kernel config to your liking by editing the .config file or using make menuconfig. A couple tips on configuring. Make sure you have SCSI disk support,USB Host Controllers and USB Storage compiled in - not as modules - otherwise you'll need to do some fancy initrd'ing. If these are not compiled in or loaded by an initrd, you are likely to get an error message like VFS: cannot open root device "801" or unknown-block(8,1). When configured to your liking, do a full recompile, just to be sure:

cp .config ../
make mrproper
cp ../.config .
make oldconfig
make dep && make bzImage modules

Note that there is no modules_install, you only want to build the tree, not install it.

That's it the kernel tree is ready to be incorporated onto your USB-stick. I wasn't lucky enough to have the kernel work right-away; unless you're sure you've got a useable kernel you should probably iteratively repeat the above commands and the following ones; followed by a reboot on the target.

cd ~/ipp_files/kernel/linux-2.4.26
cp .config /mnt/ipp/boot/test.ipp.config-2.4.26
cp System.map /mnt/ipp/boot/test.ipp.System.map-2.4.26
cp arch/i386/boot/bzImage /mnt/ipp/boot/test.ipp.kernel-2.4.26
umount /mnt/ipp/boot
umount /mnt/ipp
sync

Then unplug the usb from the host- plug it into the target - turn on the target. If success proceed to next section, otherwise remount and repeat.

Notes on 2.6 Kernel

The time it takes for the kernel to properly initialize USB devices varies. However, there is a parameter to force the kernel to wait until the root device becomes available. Simply add rootwait to your kernel parameters.

Later on, in case you get an error such as

VFS: Cannot open root device "sda3" or unknown-block(0,0)

Try using the numerical device notation instead of the /dev notation. There is usually no reason why this should fail. E.g. replace

kernel /test.ipp.kernel-2.4.26 root=/dev/sda2 vga=791

with

kernel /test.ipp.kernel-2.4.26 root=0x802 vga=791

-- Paniq 20:45, 16 Jun 2005 (GMT)


Create the root FS

We have to create a stage4 (stage3+packages) root filesystem for use on the USB-stick. This will be done with catalyst. I mostly followed the line presented in the Catalyst HOWTO as it was on July 14th 2004. The Catalyst project does not have a valid HOWTO anymore, but suggests users emerge with USE=doc and read the examples.

NOTE: Using CFLAGS="-Os" will yield much smaller binaries than CFLAGS="-O2". Keep that in mind.

First env USE=doc emerge catalyst it's important to add USE=doc since the examples are very useful. Then edit /etc/catalyst/catalyst.conf and /root/catalyst-env.sh. I ended up setting mine to this:

File: /etc/catalyst/catalyst.conf
distdir="/usr/portage/distfiles"
options="ccache pkgcache distcc"
sharedir="/usr/lib/catalyst"
envscript="/root/catalyst-env.sh"
File: /root/catalyst-env.sh
export MAKEOPTS="-j6"

I set MAKEOPTS like that because I have 4 additional hosts running distcc for compiling. If n is the number of hosts you have running distcc, use -j`(n+1)*2`.

We'll try the easy way first: get a stage3 and create a snapshot; then use both to create a livecd.

First we create a snapshot of the current source tree:

File: /root/catalyst/ipp/snapshot.spec
target: snapshot
version_stamp: 20040714

then catalyst -f snapshot.spec will create: /var/tmp/catalyst/snapshots/portage-20040714.tar.bz2

Note: the version stamp does not need to match an existing gentoo portage snapshot -- you can use an arbitrary name.

Assuming we've created portage-20040714.tar.bz2 and retrieved stage3-athlon-xp-2004.1.tar.bz2...

cp stage3-athlon-xp-2004.1.tar.bz2 /var/tmp/catalyst/builds/

We must create a specfile:

File: /root/specfiles/ipp/liveusb-stage1.spec
subarch: athlon-xp
version_stamp: 20040714
target: livecd-stage1
rel_type: default
profile: default-x86-2004.0
snapshot: 20040714
source_subpath: stage3-athlon-xp-2004.1
distcc_hosts: 1 2 3 4
livecd/use:
        -X
        -gtk
        -gtk2
        -kde
        -arts
        -alsa
        -avi
        -cdr
        -cups
        -esd
        gphoto2
        acpi
        -dvd
        -dvdr
        -emacs
        -ethereal
        f77
        -fbcon
        -ginac
        -odbc
        -plotutils
        -ppds
        -samba
        -sse
        usb
        -wmf
        -xml
        -xosd
        -gnome
        -opengl
        -oss
        -sdl
        -spell
        -truetype

livecd/packages:
        baselayout
        module-init-tools
        hotplug
        syslog-ng
        pciutils
        strace
        nfs-utils
        usbutils
        e2fsprogs
        reiserfsprogs
        nano
        less
        openssh
        dhcpcd
        mingetty
        screen
        iputils
        tcptraceroute
        ethtool
        distcc
        libusb
        grub
        gentoolkit

See: HOWTO build a LiveCD for more information on this file.

Note: the profile value: default-x86-2004.0 should refer to an existing profile in the portage tree under /usr/portage/profile - if you set it to a profile that does not exist then you will get the following error: ARCH is not set... Are you missing the /etc/make.profile symlink?


Then we tell catalyst to do the dirty work with: catalyst -f liveusb-stage1.spec After which, if there was nothing fatal, catalyst will have placed an uncompressed root filesystem at /var/tmp/catalyst/tmp/default/livecd-stage1-athlon-xp-20040714/. We should move it to our working dir before it gets overwritten by another catalyst command:

mkdir ~/ipp_files/rootfs/stage1-livecd
cd ~/ipp_files/rootfs/stage1-livecd
rsync -aH --progress /var/tmp/catalyst/tmp/default/livecd-stage1-athlon-xp-20040714/ .

Create the configs

We could have dumped that onto the stick and booted-up to start playing inside the system; but I'm betting that it doesn't fully work right away, and we'll need to re-create the root FS again and again. So we should isolate the configs to a different directory so that we can reapply them to a newly created root FS.

If this is the first time through, execute the following.

cd ~/ipp_files/configs/etc
rsync -aH /var/tmp/catalyst/tmp/default/livecd-stage1-athlon-xp-200407/etc/ .

Then we'll basically follow the gentoo handbook, chapter 8 to make sure we have useable configs. I won't cover every config, cause that would be silly.

File: /etc/fstab
/dev/sda1               /boot           ext2            noauto,noatime          1 1
/dev/sda2               /               reiserfs        noatime                 0 0
none                    /proc           proc            defaults                0 0
none                    /dev/shm        tmpfs           defaults                0 0

I also copied the following files from my host: /etc/distcc/hosts, /etc/conf.d/distcc, /etc/hosts, /etc/dnsdomainname, /etc/localtime, /etc/nanorc, and /etc/syslog-ng/syslog-ng.conf.

Putting it all together

You'd probably like to reduce the number of write-cycles to the usb-stick - depending on the hardware inside it. For this reason the rsync command and the uncompressed filesystem handed to us by catalyst are pretty useful.

Putting all together involves dumping the livecd-stage1 FS, then the kernel, then the configs all to /mnt/ipp.

cd /mnt/ipp
rsync -aH --progress --exclude 'boot/*' --exclude 'usr/portage/*' --exclude 'usr/share/doc/*' \
--exclude 'usr/share/man/*' --exclude 'usr/share/info/*' ~/ipp_files/rootfs/stage1-livecd/ .
mount /dev/sda1 boot/
cd boot/
rsync -aH --progress --exclude 'menu.lst' ~/ipp_files/rootfs/stage1-livecd/boot/ .
cd ~/ipp_files/kernel/linux-2.4.26/
env INSTALL_MOD_PATH="/mnt/ipp/" make modules_install
cp .config /mnt/ipp/boot/test.ipp.config-2.4.26
cp System.map /mnt/ipp/boot/test.ipp.System.map-2.4.26
cp arch/i386/boot/bzImage /mnt/ipp/boot/test.ipp.kernel-2.4.26
cd /mnt/ipp/etc/
rsync -aH --progress ~/ipp_files/configs/etc/ .
cd
umount /mnt/ipp/boot && umount /mnt/ipp
sync

Yibadee-Yibadee-Yibadee: That's all folks! Unplug the stick. Plug into the target. Turn on the target. Then proceed to configure things like ssh servers, nfs mounts, and whatever else you need to do dev on the target.

If you have trouble with a message like "Could not find CD to boot, something else needed!" when booting from this pendrive you will have to insert a delay on the /init script inside syslinux/gentoo.igz; to do that:

-> Mount the pendrive:

   mount /dev/sda1 /mnt2

-> Expand the initrd:

    cd /tmp; mkdir gentoo.initrd; cd gentoo.initrd
    gunzip </mnt2/isolinux/gentoo.igz >../gentoo.i
    bvi ../gentoo.i
         -> Replace all ocurrences of 'TRAILER!!!' for 'notTRAILER',
            *EXCEPT* the LAST one. Use common sense to verify if there isn't any ocurrences
                  that aren't cpio headers (DON'T change these if there are any).
               -> This is needed because the .cpio file that was compressed to produce
                  the original isolinux/gentoo.igz file is actually a concatenation of
                  *many* (I counted 10) different CPIO files; the kernel (and presumably
                  some other version of cpio) can handle this correctly, but mine (debian
                  v. 2.6-5) can't, and silently stops expanding the file on the first
                  'TRAILER!!!' header.
     cpio -idv < ../gentoo.i
         -> Expands the whole initrd tree

-> Modify the init script:

   -> Just before these lines:
        # Mount sysfs
        mount_sysfs

        # Start udev/devfs
        start_dev_mgr
   -> Add these lines:
       #2005/09/25 Durval Menezes
       #good delay after loading modules, so the system can stabilize
       #(if we don't delay, the start_dev_mgr a few lines below won't
       #find our pendrive's device and the boot will fail
       echo -n "Waiting for loaded drivers to stabilize"
       COUNTER=0
       while [ $COUNTER -lt 5 ]; do
              sleep 1
              echo -n '.'
              let COUNTER=$COUNTER+1
       done
       echo
   -> Recreating the initrd file with the modification above:
       find . | cpio --quiet --dereference -o -H newc | gzip -9 >/mnt2/isolinux/gentoo.igz
   -> Unmount the pendrive:
       umount /mnt2

The above modification should eliminate the "could not find CD to boot" message so the boot will proceed smoothly to the end.

See Also

Retrieved from "http://www.gentoo-wiki.info/HOWTO_Small_Footprint_Gentoo_on_USB"

Last modified: Thu, 02 Oct 2008 10:53:00 +0000 Hits: 87,722