Search:  
Gentoo Wiki

TIP_Imaging_with_yet_another_ghost_installer

Contents

YAGI

YAGI stands for Yet Another Ghost Installer. YAGI has quite a few advantages over other imaging systems in that it is far simpler to use from the client's point of view. YAGI really doesn't include any new technology, it's simply a collection of fairly common *nix utilities and some relatively basic concepts. The ingenuity of YAGI is in that combination.

Setup

The first step to setting up your YAGI server is to get all of the appropriate packages. Run the following commands:

emerge --sync
emerge tftp-hpa
emerge nfs-utils
emerge dhcp
emerge proftpd

Also, download the YAGI binaries Once we have all the packages, we can begin creating the directory structure.

The Directory Structure

Run the following commands as root:

cd /
wget http://dan.bayern.de/dan.deam.org/YAGI.tar.gz
tar -xvf YAGI.tar.gz
cd YAGI
mv -rf linuxroot linuxroot-old
mkdir linuxroot
cd linuxroot
mkdir home
mkdir dev
mkdir proc
mkdir tmp
mkdir mnt
chmod a+w mnt
mkdir mnt/.initd
mkdir root
mkdir var
mkdir var/empty
mkdir var/lock
mkdir var/log
mkdir var/run
mkdir var/spool
mkdir usr
mkdir opt
mknod dev/console c 5 1

Basically, you're creating a skeleton diskless root drive which will be mounted by the client(s) over NFS and from which the client(s) will boot. [Note:] It's really a bad idea to have several clients simultaneously booting from a single diskless server unless you really know what you're doing. However, all we need to do is get a simple linux boot going, just enough to suck the image off the drive and stream it to the FTP server.

Server Kernel Setup

cd /usr/src/linux
make menuconfig

Dig down into the 'File Systems' directory and make sure that your server is configured to act as an NFS server (version 3 is fine, we don't really need any of the new features for our purposes). Remember, if you have made any changes, no matter how minor, you will have to recompile and redeploy your kernel. Then you will have to restart. If you're not sure how to go about doing this, check one of the other howtos. Your configuration should include something like the following:

Code maturity level options  --->
  [*] Prompt for development and/or incomplete code/drivers

Device Drivers --->
  Networking options --->
    <*> Packet socket
    <*> Unix domain sockets
    [*] TCP/IP networking
    [*]   IP: multicasting
    [ ] Network packet filtering (replaces ipchains)

File systems --->
  Network File Systems  --->
    <*> NFS server support
    [*]   Provide NFSv3 server support

Client Kernel Setup

cd /YAGI
mkdir kernel-src
cp -r /usr/src/linux/* kernel-src
cd kernel-src
make menuconfig

The configuration for the kernel will default to whatever you have your server kernel set to. This is probably not what you want for your client(s). You will want to remove any extra options, as many modules as you can find, and all around slim the kernel down. This will make the boot up go much faster. You will want set all necessary options to be compiled statically into the kernel and not as modules. This is because the clients will have a hard time loading modules over NFS. Anyway, you're not doing much so what do you care?

A very important thing for you do to do is to ensure that the client's kernel includes the NFS v3 file system support, as well as Root as NFS (the exact wording changes weekly, don't worry about it) compiled statically into the kernel. Also, you will require the network device driver for your client(s) compiled statically into the kernel. These steps are critical, your client won't even boot without them. The configuration should include the following:

Code maturity level options  --->
  [*] Prompt for development and/or incomplete code/drivers

Device Drivers --->
  [*] Networking support
  Networking options --->
    <*> Packet socket
    <*> Unix domain sockets
    [*] TCP/IP networking
    [*]   IP: multicasting
    [*]   IP: kernel level autoconfiguration
    [*]     IP: DHCP support (NEW)

File systems --->
  Network File Systems  --->
    <*> file system support
    [*]   Provide NFSv3 client support
    [*]   Root file system on NFS

Once you're done, run the following command:

make && make modules && make bzImage

Now, you will need to drill down into your arch/i386/boot directory and grab the bzImage file. Copy the bzImage file to the /YAGI/tftboot/kernel directory and replace the existing bzImage file. Then, in the /YAGI/tftboot/kernel directory, run the following commands:

rm bzImage-2.4.19
cp bzImage bzImage-2.4.19

This replaces both images. Your client will not boot if you do not follow these steps closely.

Configuring DHCP

You will need to configure your DHCP server to provide and IP address to the client(s) as well as a TFTP address for the boot loader. Fortunately, this is fairly easy to do. Use the following dhcpd.conf file as a template:

Code: /etc/dhcp/dhcpd.conf
option domain-name "yourdomain.local";

default-lease-time 600;
max-lease-time 7200;

allow bootp;
allow booting;
boot-unknown-clients on;

ddns-update-style ad-hoc;

option space pxelinux;
option pxelinux.magic      code 208 = string;
option pxelinux.configfile code 209 = text;
option pxelinux.pathprefix code 210 = text;
option pxelinux.reboottime code 211 = unsigned integer 32;

subnet 10.152.187.0 netmask 255.255.255.0 {
}

option option-135 code 135 = text;

subnet 192.168.1.0 netmask 255.255.255.0 {
	range 192.168.1.3 192.168.1.30;
	next-server 192.168.1.2;
	option option-135 "bob/yagi-ascii";

	host client2 {
		hardware ethernet		00:12:3F:5E:8F:85;
		fixed-address			192.168.1.16;0
		option pxelinux.configfile	"yagi";
		option pxelinux.pathprefix	"/YAGI/tftpboot/pxelinux";
		option pxelinux.reboottime 	5;
		filename			"/pxelinux/pxelinux.0";
	}
}

You will have to change the various options to suit your needs. I suggest checking out HOWTO setup DHCP

Configuring TFTP

The TFTP server will provide both the boot loader and the kernel image to the client(s). Fortunately, YAGI simplifies our task a little by having a preconfigured boot loader (based on pxelinux) already included. All we have to do is configure the TFTP server.

Code: /etc/conf.d/in.tftpd
 # /etc/init.d/in.tftpd

# For more options, se tftpd(8)
INTFTPD_OPTS="-l -vvvvvv -p -c -s /YAGI/tftboot"

I know they recomend you use the INTFTPD_PATH option for modularity. Don't worry about it. If you're using TFTP for anything fancy at all, then you are doing it wrong.

Configuring NFS

Now you need to set up the NFS exports to the client. To do this, edit the /etc/exports file as root.

Code: /etc/exports
# /etc/exports: NFS file systems being exported.  See exports(5).

/YAGI/linuxroot	192.168.1.0/24(sync,rw,no_root_squash,no_all_squash)
/opt		192.168.1.0/24(sync,rw,no_root_squash,no_all_squash)
/home		192.168.1.0/24(sync,rw,no_root_squash,no_all_squash)
/usr		192.168.1.0/24(sync,rw,no_root_squash,no_all_squash)

You can export the /opt, /home and /usr directories ro (read only) if you want. It doesn't make much of a difference since the client(s) won't be doing anything on them.

Configuring ProFTP

I'm not going to go into too much detail here. There's already an excelent howto on how to setup ProFTP and customize it for your needs. I'm just going to use a simple, anonymous setup with no security. This config file has the stored images being placed in the /mnt/raid0 directory on the server. You can change this to suit your needs.

Code: /etc/proftpd/proftpd.conf
ServerName "ProFTPD Image Server"
ServerType standalone
DefaultServer on
RequireValidShell off
AuthPAM off
AUthPAMConfig ftp

Port 21

Umask 022

MaxInstances 30

User proftpd
Group proftpd

<Anonymous /mnt/raid0>
    User ftp
    Group ftp

    UserAlias anonymous ftp

    AllowOverwrite on
</Anonymous>

Note: It really isn't recommended that you not have any authentication on your FTP server. I simply did that because it's easier. I'm assuming that you're not exposing this to the internet. Thus, it shouldn't be that much of a problem.

Configuring Client Root

Now we need to create all the binaries and configuration files for the client's network boot. This is important since the client needs an OS to boot to. :-) Run the next series of commands:

cp -r /etc /YAGI/linuxroot/etc
rsync -avz /bin /YAGI/linuxroot
rsync -avz /sbin /YAGI/linuxroot
rsync -avz /lib /YAGI/linuxroot

We also want the client to boot without running fschk since the file system it would be checking is an NFS mount. So, we disable fschck by adding the /fastboot file:

touch /YAGI/linuxroot/fastboot
echo "touch /fastboot" >> /YAGI/linuxroot/etc/conf.d/local.start

The second line is to make sure that the /fastboot file is recreated on boot time. (this is necessary because fschk removes the /fastboot file after it has determined that it exists). Now our client(s) will boot without trying to check the root drive for errors.

You will also need to setup the client's /etc/fstab file to mount the filesystems from the NFS server:

Code: /etc/fstab
192.168.1.2:/YAGI/linuxroot   /         nfs     sync,hard,intr,rw,nolock,rsize=8192,wsize=8192    0 0
192.168.1.2:/opt                     /opt      nfs     sync,hard,intr,ro,nolock,rsize=8192,wsize=8192    0 0
192.168.1.2:/usr                     /usr      nfs     sync,hard,intr,ro,nolock,rsize=8192,wsize=8192    0 0
192.168.1.2:/home                    /home     nfs     sync,hard,intr,rw,nolock,rsize=8192,wsize=8192    0 0
none                            /proc     proc    defaults                                     0 0

Copying the YAGI Scripts

YAGI comes with two scripts, eatdisk and cookdisk, which are really kind of nice. They automate the creation of the images and the streaming over to the ftp server.

cd /YAGI
cp linuxroot-old/bin/eatdisk linuxroot/bin
cp linuxroot-old/bin/cookdisk linuxroot/bin
rm -rf linuxroot-old

Now would also be a good time to open those scripts up and take a look inside. The meat of the whole operation is in the following lines, respectively:

Code: /YAGI/linuxroot/bin/eatdisk
( dd if=/dev/$device | gzip -9 ) | ncftpput -d /var/log/ftp.log -c $server $imagename
Code: /YAGI/linuxroot/bin/cookdisk
ftp - o "|gunzip|dd of=/dev/${device}" ftp://${server}/${image}

I changed the cookdisk line to:

Code: /YAGI/linuxroot/bin/cookdisk2
wget -O - ftp://${server}/${image} | gunzip | dd of=/dev/${device}

This is because gentoo doesn't include the ftp utility by default. Speaking of default inclusion, you will have to emerge the ncftpput and the wget utilities as well:

emerge ncftp
cp /usr/bin/ncftpput /YAGI/linuxroot/bin
emerge wget
cp /usr/bin/wget /YAGI/linuxroot/bin

Now, the forthcoming explanation of what those magic script lines do:

eatdisk runs dd, a command which grabs the bit by bit data off the specified device and spits it out to the console (by default, you can route it elsewhere). The output from dd is routed to gzip which compresses the new image and then pipes its output to ncftpput. ncftpput streams the image over to the ftp server as the image is being created and compressed. This means no temporary files to clutter up the NFS mount root.

cookdisk runs wget, which pulls the image file down from the ftp server and spits it out to the console. The output from wget is piped to gunzip which decompresses the file and pipes its output to dd again. dd then takes the input from the console pipe and writes the data bit by bit to the specified device.

Starting the Needed Services

/etc/init.d/in.tftpd start
/etc/init.d/proftpd start
/etc/init.d/dhcp start
/etc/init.d/nfs start
rc-update add in.tftpd default
rc-update add proftpd default
rc-update add dhcp default
rc-update add nfs default

Now you should be all set. It's time to cross your fingers (or rub your prayer beads, whichever makes you feel better) and try it out.

Network Boot Testing

Make sure your client is connected to the same network as your server (I know, difficult step, you'd be amazed how many times people forget this one). Now, start your client booting. In the POST, make sure you enter SETUP and ensure that Network Boot (over PXE) is the first choice for a boot device. This step varies from machine to machine, so you'll have to dig a bit. Save and Exit from SETUP and watch and wait.

The client should get an IP address from the server within 30 seconds or so. Immediately, you should see a simple ASCII art of fancy text saying 'YAGI' This is the splash screen for the pxe linux boot loader. A few seconds later, you should see a standard linux kernel decompression and boot. After you get past the 'Mounting root as read/write' stage, you're probably home free. Once in a while, I had some trouble with the loading of the modules. (ok, it always has trouble) Give it some time. If your client boot actually fails, go back and check your kernel configuration for the client (check the server too, can't hurt...) Nine out of ten times, your problem is there. In the client boot, ignore any warnings and errors that don't actually prevent boot up. You don't need much, just a simple environment.

If your client booted successfully congratulations are in order. You are now the proud owner of a very simplistic diskless node. The hard part is over.

Post Network Boot Configuration

Your client will boot by default with the same set of services and configuration (including hostname and network config) that your server has. We'll fix that right away. Once you get to the login (X11 doesn't work over network boot so you need to be comfortable logging in as a text only logon). Enter the root username and password from your server.

Important Note: When you shutdown or reboot the client, use the power-off button on the front of the box. Do not use the shutdown now or reboot commands. If you do use those, once in a while the network boot becomes unusable, and you have to start from scratch.

Configuring Client Services

You can run the rc-update command as much as you want in the diskless client environment. Remove as many of the services as you can. Remember, you really don't need much on your client. Once you're satisfied that everything is up to spec (you might want to reboot the client once or twice to make sure), it's time to start imaging.

Client Imaging

Run the following command:

eatdisk 192.168.1.2 imagename hda

Replace the IP address with the IP of your server, the imagename with the desired saved image name, and the 'hda' with the device id of your hard drive (i.e. hda, sda, etc... No device number). This should take about an hour to complete. Go catch up on your reading or better yet, those Star Trek reruns that you have yet to finish.

If it completes successfully, you should be able to put a new OS install (even Windows or MacOSx86) and configuration on your client, run the command again (with a different image name) and swap between the images. To write and image to the disk, you use the following command:

cookdisk 192.168.1.2 imagename hda

Basically, the same options as with eatdisk. Once again, this command will take about an hour to run. However, this time you get a progress bar (and a very inaccurate time projection).

If everything worked properly, you should be able to switch between the two images and boot to them seemlessly. If everything did not work well, check that you have all the necessary utilities (especially ncftpput and wget) in the /YAGI/linuxroot/bin directory.

Imaging Different Hardware

I don't recommend that you try to place an image from one machine on another with drastically different hardware. However, between to machines with the same hardware spec (i.e. two Dell Optiplex GX520s) you shouldn't have any problems. For more info on the actual imaging process and its limitations, check out the dd manual. Enjoy!

Sources

Retrieved from "http://www.gentoo-wiki.info/TIP_Imaging_with_yet_another_ghost_installer"

Last modified: Mon, 22 Sep 2008 03:48:00 +0000 Hits: 14,059