Search:  
Gentoo Wiki

TIP_AMD64-x86-distcc


Please improve it in any way that you see fit, and remove this notice {{Cleanup}} from the article. For tips on cleaning and formatting see Cleanup process



Please format this article according to the guidelines and Wikification suggestions, then remove this notice {{Wikify}} from the article


This article is part of the Tips & Tricks series.
Terminals / Shells Network X Window System Portage System Filesystems Kernel Other

Contents

Introduction

At first I tried the whole crossdev bit et al.. and now I feel a bit foolish, so I thought I'd save someone else some time...

On x86_64 you can build 32 bit x86 binaries..natively..

  1. Make sure you're on the multilib amd64 profile. If you don't know what I'm talking about, then you already are so don't worry.
  2. Setup distcc on host and client(s) where host is amd64.
  3. Add '-m32' to CFLAGS='...' on your client systems.
  4. Since gentoo now uses i686-pc-linux-gnu-gcc etc., you're are going have to run emerge as follows:
    CC=gcc CXX=g++ emerge <package>
  5. Off you go.. (assuming you followed the distcc howto you're all set)

Setup (without crossdev)

If you want to check whether its working with the distcc monitors you have to set the DISTCC_DIR to /var/tmp/portage/.distcc for it to work thus we run distccmon-text for example as follows:

DISTCC_DIR="/var/tmp/portage/.distcc" distccmon-text N  (where N, an integer, is the update interval)
Note: Due to assembly being in certain packages amongst other things, this whole method doesn't always work. In which case remove the amd64 servers from distcc and reemerge that package

Warning: This only works if gcc versions on client and server are the same (i.e. 4.3.x & 4.3.y will work, 4.2 & 4.3 will not work)

Warning: This might mess your system if the ebuild strips all flags!!

Don't use this method for emerge -e system.

Edit--
There isn't much point in using distcc for emerge -e system anyway as most of the packages disable distcc by default. However if you really want to, emerge -e system can be made to work by:
1. running emerge -e --pretend system > sys-emrg.sh
2. editing sys-emrg.sh to replace '[ebuild N ]' with 'emerge -1O =' {ensure that the '=' sign ends up immediately to the left of the package name}
3. edit sys-emrg.sh to have #!/bin/bash as the first line and remove the line containing emerge =sys-devel/gcc-w.x.y-rz {where w,x,y,z are the version and release numbers for your installed version of gcc eg. gcc-3.4.4-r1}
4. sh sys-emrg.sh and then find something to keep you entertained for quite some time!

This method is probably more appropriate for 'emerge -e world', as far more of the packages will allow distcc to act as the compiler.


Also the emerge seems to work fine without needing to use the prefix 'CC=gcc CXX=g++'


Setup (with crossdev)

Like the original author of this wiki page, I had built crossdev support on AMD64's to assist in the build of my Celeron 300.

Note: Changing the path in /etc/init.d/distccd didn't work for me -- compiling on my x86_64 nodes still failed. However, the Gentoo Cross-Compiling DISTCC guide did work. See here for the short version

On the AMD64's run

emerge crossdev (if you haven't already)

crossdev -t i686-pc-linux-gnu


This may simplify cross compiling later on. For example I don't need to prefix emerge with 'CC=gcc CXX=g++' each time which is nice.

Distcc Setup

To enable cross compiling while still preserving your existing distcc configuration, copy and edit the following files on your x86_64 machines.

cp /etc/init.d/distccd /etc/init.d/distccd-x86 edit /etc/init.d/distccd-x86 and change the path variable as shown in bold below (assumes i686 arch in this example)

#!/sbin/runscript
# $Header: /var/cvsroot/gentoo-x86/sys-devel/distcc/files/2.18/init,v 1.1 2004/10/12 17:21:43 lisa Exp $

depend() {
        need net
        use ypbind
}

start() {
        [ -e "${DISTCCD_PIDFILE}" ] && rm -f ${DISTCCD_PIDFILE} &>/dev/null

        ebegin "Starting distccd"
        chown distcc `dirname ${DISTCCD_PIDFILE}` &>/dev/null
        TMPDIR="${TMPDIR}" \
        PATH="$/usr/i686-pc-linux-gnu/bin:${PATH}" \
        /sbin/start-stop-daemon --start --quiet --startas ${DISTCCD_EXEC} \
        --pidfile ${DISTCCD_PIDFILE} -- \
        --pid-file ${DISTCCD_PIDFILE} -N ${DISTCCD_NICE} --user distcc \
        ${DISTCCD_OPTS}

        eend $?
}

stop() {
        ebegin "Stopping distccd"
        start-stop-daemon --stop --quiet --pidfile "${DISTCCD_PIDFILE}"
        rm -f "${DISTCCD_PIDFILE}"
        eend $?
}
Note: To get this working properly, I had to make a few changes. Instead of $/usr/i686-pc-linux-gnu/bin I had to put /usr/x86_64-pc-linux-gnu/i686-pc-linux-gnu/gcc-bin/4.1.2 and then create the symlinks in that directory for c++, cpp, g++, gcc, gfortran on the x86_64 machine, to their i686-pc-linux-gnu-* equivalent. Once I did this, everything worked as noted here. Not sure if this is GCC 4.1.2 related or not.

cp /etc/conf.d/distccd /etc/conf.d/distccd-x86 edit /etc/conf.d/distccd-x86 and make the changes as shown in bold below

# Copyright 1999-2004 Gentoo Foundation
# $Header: /var/cvsroot/gentoo-x86/sys-devel/distcc/files/2.18/conf,v 1.2 2004/10/21 16:20:10 vapier Exp $
# distccd configuration file

DISTCCD_OPTS=""

# this is the distccd executable
DISTCCD_EXEC="/usr/bin/distccd"

# this is where distccd will store its pid file
DISTCCD_PIDFILE="/var/run/distccd/distccd-x86.pid"

# set this option to run distccd with extra parameters
# Default port is 3632.  For most people the default is okay.
DISTCCD_OPTS="${DISTCCD_OPTS} --port 3633"
#This is a change from the defaul of 3622
DISTCC_DIR="/dev/shm/.distcc-x86"
#I run the distcc_dir in memory for increased performance you will need to alter the DISTCC_DIR flag for distccmon accordingly
# Logging
# You can change some logging options here:
# --log-file FILE
# --log-level LEVEL  [critical,error,warning, notice, info, debug]
#
# Leaving --log-file blank will log to syslog
# example: --log-file /dev/null --log-level warning
# example: --log-level critical

DISTCCD_OPTS="${DISTCCD_OPTS} --log-level critical"

# SECURITY NOTICE:
# It is HIGHLY recomended that you use the --listen option
# for increased security. You can specify an IP to permit connections
# from or a CIDR mask
# --listen accepts only a single IP
# --allow is now mandatory as of distcc-2.18.
# example:  --allow 192.168.0.0/24
# example:  --allow 192.168.0.5 --allow 192.168.0.150
# example:  --listen 192.168.0.2
DISTCCD_OPTS="${DISTCCD_OPTS} --allow 192.168.0.1"
#This assumes your x86 machine has address 192.168.0.1
DISTCCD_OPTS="${DISTCCD_OPTS} --listen 192.168.0.2"
#This assumes your x86_64 host has address 192.168.0.2"
# set this for niceness
# Default is 15
DISTCCD_NICE="0"


On your x86 machine you need to make the following changes:

edit /etc/distcc/hosts

192.168.0.2:3633 192.168.0.3:3633 localhost
#:3633 ensures we are sending on the port number specified on the host machines

Run the x86 distcc version on each AMD64 host:

/etc/init.d/distcc-x86 start

Compiling x86_64 packages on 32bit x86

On the x86's run crossdev -t x86_64-pc-linux-gnu

To enable cross compiling while still preserving your existing distcc configuration, copy and edit the following files on your x86 machines.

cp /etc/init.d/distcc /etc/init.d/distcc-x86_64 edit /etc/init.d/distcc-x86_64 and change the path variable as shown in bold below (assumes x86_64 arch in this example)

#!/sbin/runscript
# $Header: /var/cvsroot/gentoo-x86_64/sys-devel/distcc/files/2.18/init,v 1.1 2004/10/12 17:21:43 lisa Exp $

depend() {
        need net
        use ypbind
}

start() {
        [ -e "${DISTCCD_PIDFILE}" ] && rm -f ${DISTCCD_PIDFILE} &>/dev/null

        ebegin "Starting distccd"
        chown distcc `dirname ${DISTCCD_PIDFILE}` &>/dev/null
        TMPDIR="${TMPDIR}" \
        PATH="$/usr/x86_64-pc-linux-gnu/bin:${PATH}" \
        /sbin/start-stop-daemon --start --quiet --startas ${DISTCCD_EXEC} \
        --pidfile ${DISTCCD_PIDFILE} -- \
        --pid-file ${DISTCCD_PIDFILE} -N ${DISTCCD_NICE} --user distcc \
        ${DISTCCD_OPTS}

        eend $?
}

stop() {
        ebegin "Stopping distccd"
        start-stop-daemon --stop --quiet --pidfile "${DISTCCD_PIDFILE}"
        rm -f "${DISTCCD_PIDFILE}"
        eend $?
}
Note: As noted for x86_64 host, I had to make a some changes.

Instead of $/usr/x86_64-pc-linux-gnu/bin I put gcc-config --get-bin-path x86_64-pc-linux-gnu-4.2.2 and then created symlinks by executing
cd /usr/i686-pc-linux-gnu/x86_64-pc-linux-gnu/gcc-bin/4.2.2
ln -s x86_64-pc-linux-gnu-c++ c++
ln -s x86_64-pc-linux-gnu-cpp cpp
ln -s x86_64-pc-linux-gnu-g++ g++
ln -s x86_64-pc-linux-gnu-gcc gcc
ln -s x86_64-pc-linux-gnu-gfortran gfortran

Then everything worked OK.

cp /etc/conf.d/distcc /etc/conf.d/distcc-x86_64 edit /etc/conf.d/distcc-x86_64 and make the changes as shown in bold below

# Copyright 1999-2004 Gentoo Foundation
# $Header: /var/cvsroot/gentoo-x86_64/sys-devel/distcc/files/2.18/conf,v 1.2 2004/10/21 16:20:10 vapier Exp $
# distccd configuration file

DISTCCD_OPTS=""

# this is the distccd executable
DISTCCD_EXEC="/usr/bin/distccd"

# this is where distccd will store its pid file
DISTCCD_PIDFILE="/var/run/distccd/distccd-x86_64.pid"
 
# set this option to run distccd with extra parameters
# Default port is 3632.  For most people the default is okay.
DISTCCD_OPTS="${DISTCCD_OPTS} --port 3633"
#This is a change from the defaul of 3622
#EDIT: i couldnt find this line when i tried this step... so a left it out!!  
#DISTCC_DIR="/dev/shm/.distcc-x86_64" 
#I run the distcc_dir in memory for increased performance you will need to alter the DISTCC_DIR flag for distccmon accordingly
# Logging
# You can change some logging options here:
# --log-file FILE
# --log-level LEVEL  [critical,error,warning, notice, info, debug]
#
# Leaving --log-file blank will log to syslog
# example: --log-file /dev/null --log-level warning
# example: --log-level critical

DISTCCD_OPTS="${DISTCCD_OPTS} --log-level critical"

# SECURITY NOTICE:
# It is HIGHLY recomended that you use the --listen option
# for increased security. You can specify an IP to permit connections
# from or a CIDR mask
# --listen accepts only a single IP
# --allow is now mandatory as of distcc-2.18.
# example:  --allow 192.168.0.0/24
# example:  --allow 192.168.0.5 --allow 192.168.0.150
# example:  --listen 192.168.0.2
DISTCCD_OPTS="${DISTCCD_OPTS} --allow 192.168.0.1"
#This assumes your x86 machine has address 192.168.0.1
DISTCCD_OPTS="${DISTCCD_OPTS} --listen 192.168.0.2"
#This assumes your x86_64 host has address 192.168.0.2"
# set this for niceness
# Default is 15
DISTCCD_NICE="0"


On your amd64 machine you need to make the following changes:

edit /etc/distcc/hosts

192.168.0.2:3633 192.168.0.3:3633 localhost
#:3633 ensures we are sending on the port number specified on the host machines

Run the AMD64 distcc version on each x86 host:

/etc/init.d/distcc-x86_64 start

Happy Cross-Compiling!!

Retrieved from "http://www.gentoo-wiki.info/TIP_AMD64-x86-distcc"

Last modified: Fri, 05 Sep 2008 03:53:00 +0000 Hits: 27,909