Search:  
Gentoo Wiki

OpenAFS

Contents

Purpose

This howto is just a beginning, but it ultimately aims to provide a complete guide on how to properly setup the current versions of OpenAFS with MIT KerberosV (and who knows, maybe I'll figure out how to setup Kerberos to behave nicely with OpenLDAP too... someday).

Subpages

Requirements

You will need a Gentoo system. Other distributions may (and often do) place some files in different places so you would have to look elsewhere.

You will have to set your hostname to something that resolves to your external ip (ie something different from 127.0.0.1 or the ipv6 equivalent - if this works with ipv6 at all). Note that X requires that your hostname never changes during an X session because it is embedded in its magic cookies (that are needed to open new connections - start new clients).

Ideally you would set up a dedicated machine and register it properly in dns to run an afs server. But testing purposes setting up an entry in /etc/hosts should be good enough.

It is necessary to reserve a partittion for AFS (unlike samba the filesystem is only accessible through the server, and uses special on-disk format). For testing a file (at least several GB in size) that can be mounted as loop device should suffice.

Same old, same old... emerge

If you are here, then you are probably using Gentoo, well, that's what I'm going to assume from this point on. As the good gentooer that you are, you know what the next lines are going to be, but it's no reason for me not to say it anyways.

First we tune up our USE flags. For this, as always I recommend using flagedit (emerge flagedit). Here, we need to add the kerberos and afs USE flags.

flagedit +afs +kerberos +ssl +ldap

Now we emerge all that we need. (Now here, if someone tries these steps and find out that I missed something out, please let me know, or better edit this wiki, as I already had a working SSL and Kerberos setups at the time of this writing.)

emerge openafs mit-krb5 openssl

Now get yourself a coffee, go take a shower, watch TV, sleep, eat, sleep again, go to the toilet to empty your colon, and it should be almost over... seriously, except X11 and KDE, gnome and openoffice, very few things take that long to compile.

Now, you should have OpenAFS, MIT KerberosV (for those Hamdal lovers please feel free to complete this guide) and even OpenSSL.

FIXME: ...for the moment being, I'll skip kerberos setup (hopefully I'll come back to it later) and assume from here that you have a working Kerberos realm...

Some more planning

Now you should have everything to start an afs server.

Note: It is suggested that you name your afs cell something like yourcell.your.dns.domain, and the kerberos realm the same, just in uppercase.

But you can name the cell anything you like (it only has to be lowercase), and the realm anything uppercase.

For the purpose of shortening the examples I will use cell name "acell" and realm name "ACELL".

The kerberos comes with nice documentation that should get you started.

Be sure you got the kerberos setup right. You should be able to authenticate using kinit, and see a ticket with klist. If your administrative user is admin you can test like this:

klog admin@ACELL

klist

However, the default would be typically root/admin or yourusername/admin instead of just admin.

Setting up basic OpenAFS

Let's start with some theory, OpenAFS is the result of IBM branching it's distributed file system and making it open source. It was previously developed by Transarc which was later bought by IBM, and is now being used in Tivoli server systems suite.

The OpenAFS software suite has a neat modular approach. It is comprised of the following services :

  1. The Authentication Server (the kaserver process) maintains the Authentication Database
  2. The Backup Server (the buserver process) maintains the Backup Database
  3. The Protection Server (the ptserver process) maintains the Protection Database
  4. The Volume Location (VL) Server (the vlserver process) maintains the Volume Location Database (VLDB)

This is a great feature, since most users/admins will not want to use the integrated authentication server, but rather use an already existing KerberosV setup.

You might have noticed that there is also an article about OpenAFS in the Gentoo Handbook. I felt the need to create this one because the official article is getting old. Since it was written, the official Gentoo OpenAFS package structure has changed and the files, paths and configuration settings specified are mostly outdated and it doesn't accurately explain how to integrate OpenAFS with KerberosV.

First we need to start the Basic OverSeer (BOS) Server, which monitors and controls other AFS server processes on the server machine. The -noauth settings are used to do initial setup since we didn't get the chance to setup authentication and authorization yet. The official documentation states this : "Disabling authorization checking gravely compromises cell security. You must complete all subsequent steps in one uninterrupted pass and must not leave the machine unattended until you restart the BOS Server with authorization checking enabled". All I can say is... good luck with the one step thing... and I beg you to do the initial steps offline, or at least on a screened subnet if this system is to be used seriously/securely (I'll even add, just for the record, that I'd recommend trying this extensively on a development system before moving on to production, now don't say I didn't warned you!).

Now on with the command lines to start our basic services, and remember we do not start kaserver, the authentication server since we want to replace it with our own MIT KerberosV server.

First, let bosserver do the initial setup job (creating directories, services, cell config and such). From this point, I will be using <machine name> and <cell name>, <machine name> being the hostname of the AFS server machine, and <cell name> being... well... the cell name you want to create.


Tip: Many commands refer to a specific AFS fileserver (most likely the one you're setting up!): <machine-name>. To properly set up a cell, bosserver needs an externally available address for that machine. It won't do much good to set up an AFS fileserver that can't be accessed from another machine. 127.0.0.1 is the localhost loopback regardless of the machine and cannot be used to address the fileserver from another machine. Make sure $HOSTNAME resolves to an IP of your box other than 127.0.0.1. Typically this is verified with 'host $HOSTNAME' or 'nslookup $HOSTNAME'. Go ahead and delete previous CellServDB and ThisCell files in /etc/openafs and /etc/openafs/server. They should be recreated by bosserver. They will contain the IP $HOSTNAME resolves to. Note that host and nslookup only use DNS to resolve names while bosserver may use other sources such as /etc/hosts. Make sure that /etc/hosts references your '$HOSTNAME ONLY to an externally available address and not 127.0.0.1. If in doubt, comment out $EXTIP $HOSTNAME altogether and force the DNS lookup for your external IP.


bosserver -noauth

bos setcellname <machine name> <cell name> -noauth

Next, we need the buserver, ptserver and vlserver services. To do so, we need to tell the bos server where to find the required services. That's one place the official OpenAFS documentation and Gentoo documentation doesn't quite match the actual Gentoo package structure. Our services binaries were created in the /usr/libexec/openafs directory by the emerge compile process. Now let's create those services within our current bosserver session.

bos create <machine name> buserver simple /usr/libexec/openafs/buserver -cell <cell name> -noauth

bos create <machine name> ptserver simple /usr/libexec/openafs/ptserver -cell <cell name> -noauth

bos create <machine name> vlserver simple /usr/libexec/openafs/vlserver -cell <cell name> -noauth


Tip: Now you messed up and mistyped that service path and bosserver won't let you 'delete' it? Don't worry, just remember to issue a 'bos shutdown <machine name> -noauth' to stop all services and then 'bos delete <machine name> <service name> -noauth'

Now that you have created the services you can check they are running.


Shell: # bos status <server name> -noauth

Instance buserver, currently running normally.

Instance ptserver, currently running normally.

Instance vlserver, currently running normally.


Tip: To shut down afs you can use bos <hostname> -shutdown -noauth and kill the bos server once no services are running killall bosserver.

To restart in noauth mode use again bosserver -noauth.

To start with authentication use /etc/init.d/openafs-server start and later /etc/init.d/openafs-server stop to stop the server.

It should work even if you set up the authentication incorrectly, you can still stop the server this way.

Setting up OpenAFS to communicate securely with MIT KerberosV

It is neccessary to create an user in the afs protection database. The steps as listed at http://www.gentoo.org/doc/en/openafs.xml#doc_chap5_sect5:

# bos adduser <server name> admin -cell <cell name> -noauth

# pts createuser -name admin -cell <cell name> [-id <AFS UID>] -noauth

# pts adduser admin system:administrators -cell <cell name> -noauth

Shell: # pts membership admin -cell <cell name> -noauth

Groups admin (id: 1) is a member of:

system:administrators


Now add a principal for the afs server into kerberos (kadmin prompt in bold, what follows the prompt are the commands):

Shell: kadmin -r ACELL -p admin

Password for admin@ACELL:

kadmin: addprinc -randkey afs/acell

NOTICE: no policy specified for afs/acell@ACELL; assigning "default"

Principal "afs/acell@ACELL" created.

kadmin: ktadd -k /etc/afs.keytab -e "des-cbc-crc:normal" afs/acell

Entry for principal afs/acell with kvno 3, encryption type DES cbc mode with CRC-32 added to keytab WRFILE:/etc/afs.keytab.

kadmin: getprinc afs/acell

Principal: afs/acell@ACELL

Expiration date: [never]

Last password change: ...

Password expiration date: [none]

Maximum ticket life: 1 day 00:00:00

Maximum renewable life: 0 days 00:00:00

Last modified: ... (admin@ACELL)

Last successful authentication: [never]

Last failed authentication: [never]

Failed password attempts: 0

Number of keys: 1

Key: vno 3, DES cbc mode with CRC-32, no salt

Attributes:

Policy: default


Use asetkey to add the key to afs (has to be done on the machine, needs direct access to some files storing the keys). Note the kvno (in italics) which has to be copied from the kadmin output.

asetkey add 3 /etc/afs.keytab afs/acell

Shell: # asetkey list

kvno 3: key is: ...

All done.


Now restart the bos server with authentication, and get a token for admin with kinit.

you should be able to authenticate to afs with aklog.


Shell: # aklog -cell acell -k ACELL -d

Authenticating to cell acell (server <hostname>).

We were told to authenticate to realm ACELL.

Getting tickets: afs/acell@ACELL

Using Kerberos V5 ticket natively

About to resolve name admin to id in cell acell.

Id 1

Set username to AFS ID 1

Setting tokens. AFS ID 1 / @ ACELL

Storing data in the filesystem

Refer to http://www.gentoo.org/doc/en/openafs.xml#doc_chap5_sect6 for information on starting the fileserver services.


If you start openafs-client as well, and login with aklog you should be able to access /afs.

Shell: # ls -A /afs

acell .acell

Shell: # fs lsmount /afs/.acell /afs/acell

'/afs/.acell' is a mount point for volume '%acell:root.cell'

'/afs/acell' is a mount point for volume '#acell:root.cell'


As you see, there is a volume root.cell mounted at /afs/acell. However, the volume does not exist.

Shell: # ls /afs/acell/

ls: cannot open directory /afs/acell/: No such device


To create the volume you will need the dedicated partition mentioned in the requirements.

Note: The partitions used for afs are traditionally mounted in /vicepn where n is from a to z.
If not already done, initialize the partition with a filesystem (such as ext3), mount it somewhere, and restart afs-server (so that it sees newly mounted partitions).


Shell: # vos create `hostname` <partition mountpoint> root.cell

Volume ... created on partition <partition mountpoint> of <hostname>


Test with touch /afs/acell/test


Note: unless you set up some more advanced authentication method or tweak some options the tickets obtained with kinit/aklog expire. When they do (usually after a few minutes) you will no longer be able to access afs until you get new tickets.
Shell: # ls /afs/acell/

ls: cannot open directory /afs/acell/: Permission denied

Retrieved from "http://www.gentoo-wiki.info/OpenAFS"

Last modified: Fri, 15 Aug 2008 13:10:00 +0000 Hits: 4,648