Search:  
Gentoo Wiki

HOWTO_Setup_local_Portage_and_Package_Mirror

Contents

Introduction

This document describes how to set up a local mirror of the Gentoo portage and package repositories. Both server and client configuration options are included. These can be used for configuring private mirrors or unofficial public mirrors. Once configured, this platform is highly versatile. The original author of this document is using this platform for providing local mirrors of Gentoo, CentOS, and Ubuntu, all of which are synchronized and distributed according to this document.

Server Requirements

Hosting the base packages for a complete Gentoo mirror will require the following:

At this stage the distfiles directory alone is about 60G.

Server Configuration

Portage Syncronization

The private mirror server needs to be set up with a directory to house the complete portage tree and package repository. A separate volume is suggested, but not required. /mirror will be the convention followed throughout the remainder of this document.

/mirror/gentoo-portage will serve as the root of the portage tree. 
/mirror/gentoo will serve as file repository.

Create a script file to synchronize the portage tree and package repository.

File: /etc/cron.daily/sync_portage
#! /bin/bash
if [ ! –f /tmp/portage_sync ]
then
touch /tmp/portage_sync

rsync rsync.gentoo.org::gentoo-portage \
      /mirror/gentoo-portage \
      -t --recursive -P --verbose \
      --delete --delete-after --links -q
      # sync the public portage tree against
      # the main rsync rotation

rsync distro.ibiblio.org::gentoo \
      /mirror/gentoo \
      -t --recursive -P --verbose \
      --delete --delete-after --links -q
      # sync the main gentoo mirror
      # against the ibiblio.org servers
      # this mirrors the whole  tree, including
      # distfiles, releases, and snapshots

rm /tmp/portage_sync
fi

Note: you have to delete /tmp/portage_sync in order for that script to run effectively. Otherwise the if statement will never be executed.

rm /tmp/portage_sync

This script file should be configured to run under cron once per day. Under vixie cron, simply place it under the /etc/cron.daily directory. If you want to use a less frequent synchronization schedule, simply place the script file into a different /etc/cron. directory. Please select an appropriate server from the Gentoo Mirror List for retrieving the distfile directory - it is about 50 gigabytes and will take a minimum of several days to download. On a related note, the /usr/portage directory can be used as a starting point for the new hosted ebuild directory:

rsync /usr/portage/ /mirror/gentoo-portage --exclude="distfiles" -t --recursive -P

Note: The /usr/portage directory could can theoretically be shared directly - there are many documents that state how to do just that - but there are security issues associated with hosting files that are actively used by the operating system.

Portage Sharing

To allow access to your portage directories, an rsync daemon will need to be started.

File: /etc/rsyncd.conf
[gentoo]
        path = /mirror/gentoo
        comment = Complete Main Gentoo Tree
[gentoo-portage]
        path = /mirror/gentoo-portage
        comment = Complete Portage Tree

Start the rsync daemon with /etc/init.d/rsyncd start, and add it to the default run level with rc-update add rsyncd default.

Package Sharing

To allow access to the package repository, a ftp daemon will need to be started.

ProFTP is the suggested package, and is capable of handling large files without problem. Oftp is also an option, but files over 2GB will not transfer correctly. This only poses an issue if hosting DVD images.


Emerge proftpd

File: /etc/proftpd/proftpd.conf
ServerName                      "Your Name Here"
ServerType                      standalone
DefaultServer                   on
RequireValidShell               off
AuthPAM                         off
AuthPAMConfig                   ftp
Port                            21
Umask                           022
MaxInstances                    30

User                            ftp
Group                           ftp
# These need to be changed to use the standard "ftp" user and group.

<Anonymous /mirror>
        User                            ftp
        Group                           ftp
        UserAlias                       anonymous ftp
        <Limit WRITE>
                DenyAll
        </Limit>
</Anonymous>

Start the proftp daemon with /etc/init.d/proftpd start, and add it to the default run level with rc-update add proftpd default.

Client Configuration

To configure clients to make use of the new portage server, the only changes that need to be made are a few lines in /etc/make.conf:

File: /etc/make.conf
…
SYNC="rsync://YourServerHere/gentoo-portage"
GENTOO_MIRRORS="ftp://YourServerHere/gentoo/"
…

Where RsyncServerHere is the DNS name or IP address of your gentoo server.

Additional Information

Strictly speaking, it isn't necessary to have both the [gentoo-packages] module active and the FTP server installed; either route alone is enough to allow the packages to be installed. Rsync isn't supported at the current time under the GENTOO_MIRRORS option. By making use of the GENTOO_MIRRORS option, it is possible to have fallback settings in the event of server or package unavailability. The FETCHCOMMAND options within make.conf do not provide that functionality.

The [gentoo-packages] module is included here so other servers can sync to the new local portage tree, or if you wish to make your server publicly available.

Retrieved from "http://www.gentoo-wiki.info/HOWTO_Setup_local_Portage_and_Package_Mirror"

Last modified: Sun, 01 Jun 2008 18:21:00 +0000 Hits: 9,034