Search:  
Gentoo Wiki

HOWTO_Install_VMWare_Server


Please improve it in any way that you see fit, and remove this notice {{Cleanup}} from the article. For tips on cleaning and formatting see Cleanup process


Contents

VMWare Server

VMWare brings free virtualization to everyone. From their site:

'Begin enjoying the benefits of server virtualization with the free VMware Server. VMware Server installs on any existing server hardware and partitions a physical server into multiple virtual machines by abstracting processor, memory, storage and networking resources, giving you greater hardware utilization and flexibility. Streamline software development and testing and simplify server provisioning as you utilize the ability to "build once, deploy many times."'

In short: vmware-server allows you to run a virtual computer inside your computer which can be controlled remotely. This allows you to install multiple virtual machines running various operating systems (from DOS to Windows Vista and BSD to another Gentoo inside your Gentoo machine) on one physical computer. Lets get cracking shall we?

Kernel Configuration

Make sure you have configured your (2.6) kernel with the following options:

Linux Kernel Configuration: Enable unloading modules
Loadable module support --->
 [*] Module unloading
 [*] Automatic kernel module loading

If unsure, look for the symbols CONFIG_MODULE_UNLOAD and CONFIG_KMOD in /proc/config.gz. As vmware will need to load some custom drivers in order to run the virtual machines on your Gentoo host system.

If you don't have modules unloading support or automatic loading, it still might work but you will probably run into trouble later on.

Note: If you're running 2.6.21 x86_64, you will experience VMM faults leading to full host lockups. This is due to the revamped IRQ structure in 2.6.21 -- VMWare expects to be able to use a certain interrupt, which is being used by kernel. No fix from VMWare as of yet. Workaround is to downgrade to 2.6.20. More info: http://lkml.org/lkml/2007/5/4/175
Warning: Do NOT compile your kernel with KVM on Intel processors due it probably hangs your host system when you try to power on any virtual machine. You have been warned. Check http://communities.vmware.com/thread/112203 for more information.

Installing VMWare Server

Start by making sure you have the latest portage tree by syncing and then emerge vmware-server; this will get xinetd, vmware-modules and vmware-server for you and a load of X related packages.

Code: emerge
emerge --sync
emerge -a vmware-server

But what if you don't want a X windows system? You still need the packages because vmware-server depends on them. Just remember that its only a matter of libraries: you don't actually need to configure X or start a graphical session in order to work with vmware-server.

If emerge tells you that you need unmask ~x86, you can use this command:

  echo "app-emulation/vmware-server ~x86" >> /etc/portage/package.keywords

or

  flagedit app-emulation/vmware-server -- +~x86

then

  emerge -a vmware-server
Note: Currently version 2.10.6 of GTK+ require the Cairo library to be compiled with the 'X' flag set. Because GTK+ is required, please make sure you have that flag set for the Cairo package or emerge will fail on the gtk package.

Note: VMWare Server has some dependencies on parts of GTK and X, even on a headless system. So far all my attempts to install it on a completely empty system - aka no X11 - have failed. It might be possible to get this to work without installing the whole graphical system but so far we're stuck with it.

Note: I have successfully installed VMWare Server on several boxes, with no X11, it appears ebuilds have been fixed to allow this. --No, they haven't. Some X11 libraries are required because vmware uses them.

Next up is the configuration of the vmware package. The version in portage has a tweaked installer: the original vmware scripts don't work very well on Gentoo (location of various directories etc).

Lets fire it up:

Code: Run config script
/opt/vmware/server/bin/vmware-config.pl

I will now assume you want to be able to give your virtual machines host-only, NAT and bridged networking options - in other words: the whole shebang. This is how the installer could be run (depending on what you want ofcourse):

Code: vmware-config.pl
You must read and accept the End User License Agreement to continue.
Press enter to display it.
<<enter>>
<<q>>

Do you accept? (yes/no)
<<yes>>

Do you want networking for your virtual machines? (yes/no/help) [yes]
<<enter>>

Configuring a bridged network for vmnet0.
The following bridged networks have been defined:
. vmnet0 is bridged to eth0
<<q>>

All your ethernet interfaces are already bridged.
Do you want to be able to use NAT networking in your virtual machines? (yes/no) [yes]
<<enter>>

Configuring a NAT network for vmnet8.
Do you want this program to probe for an unused private subnet? (yes/no/help) [yes]
<<enter>>

Probing for an unused private subnet (this can take some time)...
The subnet 192.168.147.0/255.255.255.0 appears to be unused.
The following NAT networks have been defined:
. vmnet8 is a NAT network on private subnet 192.168.147.0.
<<q>>

Do you wish to configure another NAT network? (yes/no) [no]
<<enter>>

Do you want to be able to use host-only networking in your virtual machines? [yes]
<<enter>>

Configuring a host-only network for vmnet1.
Do you want this program to probe for an unused private subnet? (yes/no/help) [yes]
<<enter>>

Probing for an unused private subnet (this can take some time)...
The subnet 192.168.82.0/255.255.255.0 appears to be unused.
The following host-only networks have been defined:
. vmnet1 is a host-only network on private subnet 192.168.82.0.
<<q>>

Do you wish to configure another host-only network? (yes/no) [no]
<<enter>>

Please specify a port for remote console connections to use [902]
<<enter>>

Configuring the VMware VmPerl Scripting API.
Building the VMware VmPerl Scripting API.
Using compiler "/usr/bin/gcc". Use environment variable CC to override.
Installing the VMware VmPerl Scripting API.
The installation of the VMware VmPerl Scripting API succeeded.
Generating SSL Server Certificate
In which directory do you want to keep your virtual machine files? 
[/var/lib/vmware/Virtual Machines]
<<enter>>

The path "/var/lib/vmware/Virtual Machines" does not exist currently. This
program is going to create it, including needed parent directories. Is this
what you want? [yes]
<<enter>>

Please enter your 20-character serial number.
Type XXXXX-XXXXX-XXXXX-XXXXX or 'Enter' to cancel:
<<enter serial number>>
<<enter>>

* Starting VMware services:                                              [ ok ]
*   Virtual machine monitor                                              [ ok ]
*   Virtual ethernet                                                     [ ok ]
*   Bridged networking on /dev/vmnet0                                    [ ok ]
*   Host-only networking on /dev/vmnet1 (background)                     [ ok ]
*   Host-only networking on /dev/vmnet8 (background)                     [ ok ]
*   NAT service on /dev/vmnet8                                           [ ok ]

The configuration of VMware Server 1.0.1 build-29996 for Linux for this running
kernel completed successfully.

Note that this is an example of how the configuration could be. If you tweak it, you will ofcourse get different messages.

Base rule: almost everyone could use the defaults. This boils down to hit 'q' or 'enter' where appropriate and insert the serial number when asked for.

Note: The serial number can be obtained for free after registration on the vmware website. Without a valid serial you won't be able to boot a virtual machine. You do not actually need to hand over your details to VMWare though - just fill in the form with random data and you will get a serial number after submitting it.


Add yourself to the vmware group:

Code:

sudo gpasswd -a yourusername vmware

Now recompile your kernel, install and reboot. VMWare server should work fine now.


VMWare server and PaX (Hardened kernel) troubleshooting

If you are running hardened, you may have to disable PaX. Problem will manifest at start of guest OS as a black screen and grayed out power buttons.

paxctl -pemrxs /opt/vmware/server/sbin/vmware-serverd
paxctl -pemrxs /opt/vmware/server/bin/vmnet-bridge
paxctl -pemrxs /opt/vmware/server/lib/bin/vmware-vmx

I did not have time to figured out what specifically cause it to not work, but this will at least get you running. When I have more time or if a kind person please narrow it down.

If you run PaX and a recent version of VMWare (which you almost automatically will be doing if you are reading this) you might not be able to start VMWare virtual machines while getting all kinds of strange errors (not enough memory, modules not configured etc) and you know for sure that you followed this guide to the letter, check the out the output of dmesg for something like this:

PAX: vmware-vmx:6590, uid/euid: 0/0, attempted to modify kernel code at virtual address c0696085
 printing eip:
0036e076
*pde = 004001e1
Oops: 0003 [#1]
PREEMPT
Modules linked in: tun rtc vmnet(P) vmmon(P) parport_pc sata_nv libata parport
CPU:    0
EIP:    0060:[<0036e076>]    Tainted: P       VLI
EFLAGS: 00013002   (2.6.22-hardened-r8 #1)
EIP is at Task_Switch_S1B1+0x236/0x9f0 [vmmon]
eax: 00000089   ebx: 00000080   ecx: c0696080   edx: 0000008b
esi: f881b000   edi: 000006d0   ebp: 00000080   esp: f73f59f4
ds: 0068   es: 0068   fs: 0000  gs: 0033  ss: 0068
Process vmware-vmx (pid: 6590, ti=f73f4000 task=f7216a90 task.ti=f73f4000)

The culprit here is not vmware-vmx (note that you *should* disable PaX on vmware-vmx) but in fact the module vmmon is. This is because it attempts to alter kernel memory which is not allowed in most hardened kernels.

Warning: If you have the CONFIG_PAX_KERNEXEC and vmmon issue, note that disabling the option to get vmware to work *will* downgrade your security (if only by a little). If you find security more important than getting vmware to work you should probably wait for a future version of the vmware-modules package that works fine with all PaX features.

Make sure you disable the CONFIG_PAX_KERNEXEC option in your kernel (2.6.22-hardened in this case):

 Security options  --->
    PaX  --->
       Non-executable pages  --->
          [ ] Enforce non-executable kernel pages       <-- Make sure this is disabled!

Getting ready to connect

If everything went as planned, you will now have a number of vmware modules loaded into your kernel and a working configuration. Before we can actually enjoy our new marvel, we need to set up xinetd so we can connect to the management interface.

By default, xinetd allows only connections from localhost, so unless you are using a SSH tunnel to get to the server I suggest you open it up to your internal lan by adding a only_from line to it.

File: /etc/xinetd.d/vmware-authd
# default: on
# description: The VMware remote access authentification daemon
service vmware-authd
{
   disable         = no
   port            = 902
   socket_type     = stream
   protocol        = tcp
   wait            = no
   user            = root
   server          = /opt/vmware/server/sbin/vmware-authd
   type            = unlisted
   only_from       = 192.168.0.0/24 127.0.0.1
}

If your network has a different range you would ofcourse specify it here.

Warning: If you forget this step or make a mistake you won't be able to connect to your VMware Server, you will notice vmware-authd generates fail messages in /var/log/messages and your vmware console will hang forever instead of logging in.

Finally we are ready to rumble to lets start all the services and add then to the bootup.

Code: Console
/etc/init.d/xinetd start
/etc/init.d/vmware start
rc-update add xinetd default
rc-update add vmware default

One last thing that needs to be done is granting the user you wish to connect with the rights to do so by adding him/her to the vmware group.

Code: Adding root to vmware group
 gpasswd -a root vmware
Note: If you do not add a user to the vmware group you will get a message saying your credentials aren't right - this is merely because you have no users for vmware

Conclusion

There you have it. Ten minutes worth of installation and configuration to get yourself a working vmware-server install. Of course if you had to find everything out from scratch or had to install the entire X system (like yours truly) you would probably still be at it...

Now all you need is the vmware-console to actually use your new toy. Gentoo has a package for it:

# emerge -a vmware-server-console

Note that this is done on the client machine that you are going to connect to the server with. The server does not need vmware-server-console to run.

Windows users should grab it from the vmware website. (If you use vmware server only locally, there is no need to install the separate package with console! Simply run vmware after configuration of the server and xinetd.) You are going to have to create and interact with the virtual machine images somehow. You can do that remotely from a Windows box using its edition of vmware-server-console, but otherwise, I'm not sure how you can get around not needing it. And installing it requires even more parts of X including fonts which emerge doesn't currently include.

Below are some sections of the old guide which might help people with troubleshooting. Both sections are probably useless as they are broken or incomplete. If you made it this far, congratulations! You are done! Otherwise: happy hunting :)

Web Interface (MUI)

'The instructions have been tested with VMware server 1.0.1 and 1.0.3 (so I suspect they will work with 1.0.2)

Download the .tar.gz file from the VMware website into /usr/local/src

  tar -xvf VMware-mui-1.0.3-44356.tar.gz

Create some dirs for the vmware init scripts to be stored in (I store most my scripts in /usr/scripts so)

  mkdir –p /usr/scripts/vmware/rc{0,1,2,3,4,5,6}.d

Find out the path of the vmware executable (and make a note of it, copy it to your clipboard)

  which vmware
  /opt/vmware/server/bin/vmware
Warning: If you don’t have X Windows installed then you need to also install libXi
  emerge libXi
  cd /usr/local/src/vmware-mui-distrib

Now run the install script. If starts saying that you must have VMware installed and you do check that the MUI version matches you VMWare version, and check that the vmware command (the console) runs successfully.

Code: /vmware-install.pl
Creating a new installer database using the tar3 format.

You must read and accept the End User License Agreement to continue.
Press enter to display it.

Do you accept? (yes/no) yes

Thank you.

Installing the content of the package.

Setup is unable to find the "vmware" program on your machine. Please make sure
it is installed. Do you want to specify the location of this program by hand?
[yes]

What is the location of the "vmware" program on your
machine? /opt/vmware/server/bin/vmware

In which directory do you want to install the binary files?
[/usr/bin]

The file /usr/bin/vmware-uninstall-mui.pl that this program was about to
install already exists. Overwrite? [yes]

The file /usr/bin/vmware-config-mui.pl that this program was about to install
already exists. Overwrite? [yes]

What is the directory that contains the init directories (rc0.d/ to
rc6.d/)? /usr/scripts/vmware

What is the directory that contains the init scripts?
[/usr/scripts/vmware]

In which directory do you want to install the VMware Management Interface
files? [/usr/lib/vmware-mui] /opt/vmware/mui

The path "/opt/vmware/mui" does not exist currently. This program is going to
create it, including needed parent directories. Is this what you want?
[yes]

In which directory would you like to install the documentation files?
[/opt/vmware/mui/doc]

The path "/opt/vmware/mui/doc" does not exist currently. This program is going
to create it, including needed parent directories. Is this what you want?
[yes]

The installation of VMware Management Interface 1.0.3 build-44356 for Linux
completed successfully. You can decide to remove this software from your system
at any time by invoking the following command:
"/usr/bin/vmware-uninstall-mui.pl".

Before running VMware Management Interface for the first time, you need to
configure it by invoking the following command:
"/usr/bin/vmware-config-mui.pl". Do you want this program to invoke the command
for you now? [yes]

Setup is unable to find the "vmware" program on your machine.  Please make sure
it is installed.  Do you want to specify the location of this program by hand?
[yes]

What is the location of the "vmware" program on your
machine? /opt/vmware/server/bin/vmware

Configuring httpd.conf to run Apache as:
User: nobody and Group: nogroup

Set the number of minutes before a http session times out. (This is the length
of time before someone connecting to VMware Management Interface will be logged
out) [60]

Generating SSL Server Certificate

   Starting httpd.vmware:                                              done
The configuration of VMware Management Interface completed successfully.

Now we fix some library issues.

  cd /opt/vmware/mui/lib
  mkdir old
  mv libcrypto.so.4 old
  mv libssl.so.4 old
  mv libdb.so.3 old
  ln -s /usr/lib/libcrypto.so libcrypto.so.4
  ln -s /usr/lib/libssl.so libssl.so.4
  ln -s /usr/lib/libdb.so libdb.so.3
Note: If you are missing /usr/lib/libcrypto.so and/or /usr/lib/libssl.so:
  emerge dev-libs/openssl
Note: If you are missing /usr/lib/libdb.so
  emerge sys-libs/db


If you use version 1.0.5 or higher, then you could fix some other library issues as well.

  cd /opt/vmware/mui/lib
  mkdir old
  emerge -u app-text/sablotron
  mv -iv libsablot.so* old
  ln -s /usr/lib/libsablot.so libsablot.so
  ln -s libsablot.so libsablot.so.0
  ln -s libsablot.so libsablot.so.0.4
  cd /opt/vmware/mui/lib
  mkdir old
  emerge -u net-libs/libwww
  mv -iv libxmlparse.* libxmltok.* old
  ln -s /usr/lib/libxmlparse.so libxmlparse.so
  ln -s libxmlparse.so libxmlparse.so.1
  ln -s libxmlparse.so libxmlparse.so.1.0
  ln -s /usr/lib/libxmltok.so libxmltok.so
  ln -s libxmltok.so libxmltok.so.1
  ln -s libxmltok.so libxmltok.so.1.0


Now some symlinks and make vmware-authd executable

  ln -s /opt/vmware/server/sbin/vmware-authd /usr/sbin
  chmod +xs /opt/vmware/server/sbin/vmware-authd
Note: If you get a permission denied it might be worth making sure that /etc/xinetd.d/vmware-authd is configured for with 127.0.0.1 (along with any other IP addresses you need) under only_from

Finally

Go to https://hostname:8333/

You might like to either create a custom init script to call /usr/scripts/vmware/httpd.vmware start at boot (and maybe one at shutdown!?)

Old And Broken Stuff

As stated before, this is probably rubbish as it should be broken and incomplete. Otherwise: happy reading!

Web Interface (Broken)

Warning: You are now wandering off the gold brick road aka portage. The next section is probably broken which means that we won't have the MUI in Gentoo until someone creates a package.

Now you can install the MUI (Not verified with the build 24927)

MUI fails to install as of v1.0.1 build 29996 --Ihatemyisp 15:14, 17 November 2006 (UTC)

You have to install libXi to get this to work for build 29996

# emerge libXi

User Gondolin found a simple solution to the vmware-authd issue that was preventing use of the MUI http://forums.gentoo.org/viewtopic-t-530616-highlight-.html

# chmod -rws--x--x /opt/vmware/server/sbin/vmware-authd

According to the site :http://www.trustix.org/wiki/index.php/Installing_VMware_on_TSL

You need to add links to lib modules:

I found the correct modules comparing the site's info and "ls libdb*", "ls libssl*" and "ls libcrypto*"

# cd /usr/lib
# ln -s libdb-4.2.so libdb.so.3
# ln -s libssl.so.0.9.7 libssl.so.4
# ln -s libcrypto.so.0.9.7 libcrypto.so.4


Download the Vmware-mui-e.x.p-23869 package and unzip

# tar -xf Vmware-mui-e.x.p-23869.tar -C /tmp
# cd /tmp/vmware-mui-distrib
# ./vmware-install.pl

>Setup is unable to find the "vmware" program on your machine. Please make sure >it is installed. Do you want to specify the location of this program by hand? [yes]

>What is the location of the "vmware" program on your >machine?

/usr/local/bin/vmware

>What is the directory that contains the init directories (rc0.d/ to >rc6.d/)?

/etc/vmware/init.d/

>What is the directory that contains the init scripts? >[/etc/vmware/init.d]

Answer default

continue with /usr/bin/vmware-config-mui.pl

>What is the location of the "vmware" program on your >machine? /

usr/local/bin/vmware


>Can't exec "/etc/init.d/httpd.vmware": No such file or directory at /usr/bin/vmware-config-mui.pl line 7872. Ignore this, the startup script is not yet in place

Copy the init script:

# cp /etc/vmware/init.d/httpd.vmware /etc/init.d/

Start it:

# /etc/init.d/httpd.vmware start

Add it to boot:

# rc-update add httpd.vmware default

Browse to https://yourserver:8333/vmware/en/

From here you can download the console installation (didn't try it on linux)

Sharing sound

To share sound with other applications:

emerge -av alsa-oss

Launch vmware as follows:

aoss vmware

Tips

One handy hint add the users you want to run the console into the vmware group.

To get started, just connect with the vmware-server-console program. Then create a new virtual machine. If you don't have physical access to the machine to load up a CD-ROM, just download a boot iso somewhere on the host OS filesystem and configure the virtual machine CD-ROM to point to that file instead of the physical CD-ROM.

Not able to connect to Vmware-server

Not even locally ?

To connect locally just do as root:

ln -s /opt/vmware/server/sbin/vmware-authd /usr/sbin/vmware-authd


Then you might want to edit your xinetd config file for the vmware auth daemon. Mine looks like this:


service vmware-authd
{
disable = no
port = 902
socket_type = stream
protocol = tcp
wait = no
user = root
only_from = 192.168.0.0/24
server = /opt/vmware/server/sbin/vmware-authd
type = unlisted
}

After a fresh install, the "only_from" line was missing. only_from = 0.0.0.0 will allow access from anywhere. Without this line you will only be able to connect from localhost to localhost.

This error was thrown when attempting to connect via BOTH the web interface AND the server console application from a separate machine:

"Unable to connect to the remote host: 511 Error connecting to /usr/sbin/vmware-serverd process."

I solved this by:
Look at /etc/resolv.conf to ensure that all nameservers exist and can be reached.
In other words, DO NOT set resolv.conf on the host to use a DNS server VM served by the same host. In practice, it seems to work if the DNS server VM IP is listed AFTER another remote DNS server IP is listed.
I did not need to add the "only_from" lines as mentioned above.

Not able to connect to Vmware-server (another situation)

You should add user root (not yourself) to vmware group (by default it's not here). You could also try playing with vmware-authtrusted

Keyboard for Vmware-Server in guest os not working as expected

You can edit the /etc/vmware/config or ~/.vmware/config file if you have keys that do not work in your installed guest operating systems. Add entries that look like the one below to the end of the config file.

xkeymap.keycode.108 = 0x138 # Alt_R
xkeymap.keycode.106 = 0x135 # KP_Divide
xkeymap.keycode.104 = 0x11c # KP_Enter
xkeymap.keycode.111 = 0x148 # Up
xkeymap.keycode.116 = 0x150 # Down
xkeymap.keycode.113 = 0x14b # Left
xkeymap.keycode.114 = 0x14d # Right
xkeymap.keycode.105 = 0x11d # Control_R
xkeymap.keycode.118 = 0x152 # Insert
xkeymap.keycode.119 = 0x153 # Delete
xkeymap.keycode.110 = 0x147 # Home
xkeymap.keycode.115 = 0x14f # End
xkeymap.keycode.112 = 0x149 # Prior
xkeymap.keycode.117 = 0x151 # Next
xkeymap.keycode.78 = 0x46 # Scroll_Lock
xkeymap.keycode.127 = 0x100 # Pause
xkeymap.keycode.133 = 0x15b # Meta_L
xkeymap.keycode.134 = 0x15c # Meta_R
xkeymap.keycode.135 = 0x15d # Menu
 


Replace the xkeymap.keycode.* number with the key number (use xev to find this) of your keyboard and 0x* with the correct hex code.

Note: For further details refer to http://communities.vmware.com/message/508070#508070 .


The example above sets all needed keys. Probably you do not have to change anything and can copy the entries from above.

Retrieved from "http://www.gentoo-wiki.info/VMware_Server"

Last modified: Mon, 11 Aug 2008 06:39:00 +0000 Hits: 63,884