Search:  
Gentoo Wiki

OpenVZ


This article is part of the HOWTO series.
Installation Kernel & Hardware Networks Portage Software System X Server Gaming Non-x86 Emulators Misc

Contents

Introduction

OpenVZ is a virtualization solution. It allows the creation of multiple isolated virtual environments (VEs), each one of which looks and feels like a separate physical server. This approach is called OS-level virtualization and means that partitioning is done on the kernel level: there is a single kernel hosting all the VEs.

OS-level virtualization can be limiting because you can not run different operating systems on your physical server. But the upside features native speed (no emulation or hypervisor overhead; virtualization overhead is negligible: up to 3%), high density (up to several hundreds of VEs on a single box), high scalability (up to 64GB of RAM, up to 64 CPUs), dynamic resource management (everything can be changed on-the-fly, including memory and disk space for a VE), and other goodies.

Virtual Private Server - Future of Web Hosting

Prerequisites

An installed Gentoo system, an Internet connection.

Installation

This is what I did on my system to get it work. This is not well-tested, so YMMV.

Get your Sources

Code: Emerge the OpenVZ kernel sources
# USE=symlink emerge openvz-sources

Prepare Your kernel

  • cd into openvz kernel source
Code: Traverse into the new source
# cd /usr/src/linux
  • The gentoo-supplied openvz-sources does not produce a good .config by itself (Single CPU Kernel)
Code: Download a good .config from http://openvz.org/download/kernel
# wget http://download.openvz.org/kernel/branches/2.6.18/current/configs/kernel-2.6.18-i686.config.ovz
  • The gentoo-supplied openvz-sources does not produce a good .config by itself (SMP Enabled CPU / Hyperthreaded Kernel)
Code: Download a good .config from http://openvz.org/download/kernel
# wget http://download.openvz.org/kernel/branches/2.6.18/current/configs/kernel-2.6.18-i686-smp.config.ovz
  • The gentoo-supplied openvz-sources does not produce a good .config by itself (RAM >4GB, Enterprise)
Code: Download a good .config from http://openvz.org/download/kernel
# wget http://download.openvz.org/kernel/branches/2.6.18/current/configs/kernel-2.6.18-i686-enterprise.config.ovz
  • Configure your new kernel (be sure to select your necessary drivers)

- Now run "make menuconfig", select "Load an alternate configuration file" and select the file "/usr/src/linux-2.6.18/kernel-2.6.18-i686.config.ovz".

- Be sure to enable any system-specific drivers or modules that you need.

Code: Configure your kernel parameters
# make menuconfig

- Be sure that all OpenVZ parts compiled as modules.

Build Your kernel

  • Compile your kernel
Code: Compile the kernel
# mount /boot # make && make modules_install
  • Prepare your kernel to boot
Code: For the Grub Bootloader
# cp arch/i386/boot/bzImage /boot/linux-2.6.18-openvz-028.053.5 # cp .config /boot/.config-linux-2.6.18-openvz-028.053.5
  • Prepare your bootloader
File: /boot/grub/grub.conf
title OpenVZ 2.6.18-openvz-028stab053.5
root (hd0,0)
kernel /linux-2.6.18-openvz-028.053.5 ro root=/dev/sda3

Configure your kernel parameters

Edit /etc/sysctl.conf and make it include the following.


File: /etc/sysctl.conf
...
# On Hardware Node we generally need
# packet forwarding enabled and proxy arp disabled
net.ipv4.ip_forward = 1
net.ipv4.conf.default.proxy_arp = 0
# Enables source route verification
net.ipv4.conf.all.rp_filter = 1
# Enables the magic-sysrq key
kernel.sysrq = 1
# TCP Explict Congestion Notification
#net.ipv4.tcp_ecn = 0
# we do not want all our interfaces to send redirects
net.ipv4.conf.default.send_redirects = 1
net.ipv4.conf.all.send_redirects = 0
...
Note: You can load the parameters immediately into your running kernel with the command sysctl -p; be forwarned that you might get an error regarding "error: "kernel.sysrq" is an unknown key" since you have not yet booted the new kernel

Bring up the new Kernel

Reboot your system into the new kernel

Emerge the Components

Now that the new kernel has booted, it is time to install the necessary utilities. However, before we can do this, there is a necessary warning.

Code: Install the components
# emerge vzctl vzquota

Start VZ Service

Code: Start VZ
# /etc/init.d/vz start
Code: Configure VZ to start at boot
# rc-update add vz default

OS Template Cache Preparation

Note: An OS template cache is an OS template installed into a VPS and then packed into a gzipped tarball. Using a pre-built cache, a new VPS can be created in a matter of minutes, if not seconds.
  • For the purpose of this exercise, we will use a pre-built gentoo stage3 OS Template Cache.
Code: Download the Gentoo Template Cache
wget http://download.openvz.org/template/precreated/gentoo-20060317-i686-stage3.tar.gz
  • If you would like specific information in creating this yourself, please see the official site and OpenVZ wiki on Gentoo template creation.
  • Place your OS Template Cache in the directory /vz/template/cache/
Code: Mv your Template
# mv gentoo-20060317-i686-stage3.tar.gz /vz/template/cache/.

VPS Creation

Please see the openvz.org site on VPS-Creation for specific questions. Here, I will merely cover what I did to get it working.

Configure your Global Values

When you run a command such as

Code: Create your VPS
# vzctl create 101 --ostemplate gentoo-20060317-i686-stage3 --config vps.basic
  • It creates a VPS with an ID of 101 in a private area with the gentoo template, and the configuration parameters taken from ve-vps.basic.conf-sample.
  • If you do not specify the --config parameter, vzctl will take the configuration sample specified by CONFIGFILE parameter in /etc/conf.d/vz file. By default this is vps.basic.
  • If you do not specify the --ostemplate parameter, vzctl will take the OS template specified by the DEF_OSTEMPLATE parameter in /etc/conf.d/vz file.

If you want your default OS template to be gentoo, modify the /etc/conf.d/vz accordingly:

File: /etc/conf.d/vz
...
DEF_OSTEMPLATE="gentoo-20060317-i686-stage3"
CONFIGFILE="vps.basic"
...
  • Once you have taken the advice and put in the configs above, you could create the same VZ by simply typing the following, though more specifics are to follow.
Code: Create your VPS Shortcut

# vzctl create 101

Defining a VPS

       ** Setting Virtual Private Server startup parameters;
       ** Setting Virtual Private Server network parameters;
       ** Setting Virtual Private Server user passwords;
       ** Configurating Quality of Service (Service Level) parameters.
       ** vzctl set 101 --onboot yes --save
       ** vzctl start 101
       ** vzctl set 101 --hostname test101.my.org --save
       ** vzctl set 101 --ipadd 10.0.186.1 --save
       ** vzctl set 101 --nameserver 192.168.1.165 --save
       ** If you enter the above commands while the VPS is running and omit the --save option, they will only apply to the currently running session.

Accessing a VPS

       ** vzctl start 101
       ** vzctl exec 101 /etc/init.d/sshd status
       ** vzctl exec 101 /etc/init.d/sshd start
       ** vzctl exec 101 /etc/init.d/sshd status
       ** vzctl start 101
       ** vzctl exec 101 passwd
Note: You will have to enter the root password when prompted. This will appera in clear-text on your screen for now. Once you ssh/connect to the virtual host, you can change this password again.
Code: My VPS Creation
SYSTEM # vzctl create 101 --ostemplate gentoo-20060317-i686-stage3
Creating VPS private area: /vz/private/101
Performing postcreate actions
VPS private area was created

SYSTEM # vzctl set 101 --hostname virtualbox --save
Set hostname: virtualbox

SYSTEM # vzctl set 101 --ipadd 192.168.37.10 --save
Saved parameters for VPS 101

SYSTEM # ping 192.168.37.10
PING 192.168.37.10 (192.168.37.10) 56(84) bytes of data.
64 bytes from 192.168.37.10: icmp_seq=1 ttl=64 time=0.105 ms

SYSTEM # vzctl start 101
Starting VPS ...
VPS is mounted
Adding IP address(es): 192.168.37.10
Setting CPU units: 1000
VPS start in progress...

SYSTEM # vzctl exec 101 /etc/init.d/sshd start

SYSTEM # vzctl exec 101 passwd

SYSTEM # ssh 192.168.37.10

VPS Control

       ** vzctl start 101   - Start the VPS
       ** vzctl status 101  - Check the status of the VPS
       ** vzlist 101        - Query info on the VPS
       ** cat /proc/vz/veinfo   - See inf on the host box abt the VPS
       ** vzctl stop 101    - Stop the VPS
       ** vzctl restart 101 - Restart the VPS
       ** vzctl destroy 101 - Destroys VPS (VPS must be stopped first)
       ** vzctl exec 101 /etc/init.d/sshd status
       ** OpenVZ allows you to execute commands to the VPS directly through vzctl, however, it is recommended that you connect to the VPS via the network (e.g. ssh) and perform maintenance that way.
       ** for i in `vzlist -o veid -H`; do echo "VPS $i"; vzctl exec $i <COMMAND>; done
               *** This will pass the command to all running VPSs

VPS Discovery

       ** vzlist -a         - Does a listing of all VPS (running and stopped)
       ** vzlist -o veid,diskinodes.s -s diskinodes.s (e.g. Custom list)

VPS Maintenance

Tips and Tricks

Sharing the portage-tree

WARNING, if one of your hosts is compromized then _EVERY_ host _AND_ your server are in danger, if someone adjust your portage and you install one of those programs (that appear to be safe) then you're in big trouble, the attacker could just install and run software on your system.

To share the portage tree and your distfiles directory you can bindmount /usr/portage to the VPS:

Code: mounting /usr/portage in the VPS
# mount --bind /usr/portage /vz/root/<VPS-ID>/usr/portage

This command must be run on the host, and the VPS must be running.

To do this automatically, you can create mount and umount scripts in /etc/vz/conf. If this should be done for a specific VPS create a <VPS-ID>.mount and <VPS-ID>.umount script. If all of your VPS are running Gentoo, you probably want this for all VPS. So name the scripts vps.mount and vps.umount .

File: /etc/vz/conf/vps.mount
#!/bin/bash
mount --bind /usr/portage /vz/root/${VEID}/usr/portage
exit ${?}

File: /etc/vz/conf/vps.umount
#!/bin/bash
umount /vz/root/${VEID}/usr/portage
exit 0 # ignore errors

OR, to automatically do this for only gentoo VEs try this:

File: /etc/vz/conf/vps.mount
#!/bin/bash
# if one of these files does not exist then something is 
# really broken 
[ -f /etc/vz/vz.conf ] || exit 1 
[ -f $VE_CONFFILE ] || exit 1 

. /etc/vz/vz.conf
. $VE_CONFFILE

if [ "x$DISTRIBUTION" == "xgentoo" ]
then
	mount -o bind /usr/portage $VE_ROOT/usr/portage
fi

exit ${?}

File: /etc/vz/conf/vps.umount
#!/bin/bash
# if one of these files does not exist then something is 
# really broken 
[ -f /etc/vz/vz.conf ] || exit 1 
[ -f $VE_CONFFILE ] || exit 1 

. /etc/vz/vz.conf
. $VE_CONFFILE

if [ "x$DISTRIBUTION" == "xgentoo" ]
then
	umount $VE_ROOT/usr/portage &> /dev/null
fi

exit 0

This requires that you have DISTRIBUTION="gentoo" in each of the Gentoo VE configuration files (ie: /etc/vz/conf/101.conf). Also, it will complain for some strange reason that /usr/portage is not mounted when you stop the VE, as umount is apparently called multiple times by vz; make sure it exits with 0 and it will be fine. This "noise" is hidden by redirecting the umount output to /dev/null.

Don't forget to make these scripts executable.

Virtual memory exhausted / Memory allocation error

If you get an error like "virtual memory exhausted" or "Memory allocation error" in a VPS, your limits for that VPS are to small.

Code: Show the actual limits
# cat /proc/user_beancounters

You should have a look at privvmpages for the VPS and the last column 'failcnt', if there is a number greater the zero, you should set greater values.

Code: increase privvmpages
# vzctl set <VPS-ID> --privvmpages 73728:80362 --save

I've had the values 49152 and 53575 before, and have multiplied them by 1.5. Maybe, you will have to try a little bit to find out the best values for your system.

FYI: I got this error while emerging dev-libs/boost.It looks that emerging big packages (gcc & glibc) also needs more resources, so you may need to tweak other limits, ie kmemsize.

Retrieved from "http://www.gentoo-wiki.info/OpenVZ"

Last modified: Fri, 05 Sep 2008 20:34:00 +0000 Hits: 25,592