Search:  
Gentoo Wiki

HOWTO_Encrypted_Backup_System_Using_Bacula_and_GnuPG

Contents

General Overview

Why do it this way?

On January 4, 2007 Bacula project released version 2.0.0 which includes data encryption using public/private RSA certificates and AES128 encryption. Encrypting data with GnuPG will still work without any changes to the process.

Intended Audience

Why use this setup

Why NOT use this setup

General Overview of the data flow

The Director will do FULL backup every day for the ease of restoration. Each day it will create a new backup volume.

During the backup process Bacula server (called the Director) requests the files from the client (called File-Daemon or FD for short). During the request Director passes arguments to the FD - Director authentication password, files & folder list, request for compression and MD5/SHA1 sum. The received files being written to the hard drive volume (one large file) and cataloged to the database.

Once the Director is done with all clients, the director calls a script, which will encrypt the backup volume with GnuPG and delete the unencrypted volume. After the encryption is completed, the file will be transferred via FTP or CIFS/SMB for offsite storage. (Optionally it can be written to a tape with a 3rd party utility)

Hardware Configuration

Any sufficiently fast server will do. The requirements vary greatly between different environments. To give you a starting point, here is one example Dell PowerEdge 2800

This setup backs up and encrypts data for about 50 clients, in my organization it equates to about 40 Gig compressed & encrypted data.

Software Configuration

GnuPG

GnuPG is an OpenPGP compliant utility that can use symmetric or asymmetric methods of encrypting data.

GnuPG Installation

emerge gnupg

Public / Private Key creation

For a good overview on what GnuPG is all about, please visit our own Gentoo GnuPG Guide You may or may not follow the suggested values for the key in the guide, but this is a safe setup for a backup system

Please note - you do not need and should NOT store the private key on the backup server. It is a security risk. The best way is to generate the priv/pub key pairs on a secured workstation. Export the public key only as a text file and import public key only into the backup server. (see Gentoo GnuPG Guide guide on how to do export/import)

Store the following items in a safe place

If you loose your private key or passphrase - bye-bye data.

Bacula

Server Installation

The latest version of Bacula in the portage is outdated (bacula-1.36.3-r3) where the latest version as of writing is 1.38.11. Below is example on how to install Bacula from source.

1. Download latest bacula source from SourceForge.net
2 Extract the bacula source
tar -xvzf bacula-x.xx.xx.tar.gz
3 Enter the source folder
cd bacula-x.xx.xx
4 Run the configure command
./configure \
--prefix=/usr \
--sbindir=/usr/sbin \
--sysconfdir=/etc/bacula \
--with-scriptdir=/etc/bacula \
--enable-smartalloc \
--with-mysql \
--with-working-dir=/var/bacula \
--with-pid-dir=/var/run \
--with-subsys-dir=/var/lock/subsys \
--enable-conio \
--with-openssl \
--enable-largefile \
--enable-wx-console \
--with-python
5 Compile and Install
make
make install
6 Setup Database (assuming you have MySQL installed and working)
/etc/bacula/create_mysql_database
/etc/bacula/make_mysql_tables
/etc/bacula/grant_mysql_privileges
7 Add bacula to the /etc/conf.d/local.start
echo /etc/bacula/bacula start >> /etc/conf.d/local.start
8 add bacula to the /etc/conf.d/local.stop
echo /etc/bacula/bacula stop >> /etc/conf.d/local.stop

Server (Director) Configuration (bacula-dir.conf)

This is the most confusing part of the whole process. In the following example, just substitute $VARIABLE with your own value.

File: /etc/bacula/bacula-dir.conf
###################   DEFINE DIRECTOR   #############################
Director                     
{                                     
 Name = backup-dir                               #or whatever name you want
 Description = "Bacula Director - $MY_LOCATION"  # "Bacula Director - Main Office"
 DIRport = 9101                       
 QueryFile = "/etc/bacula/query.sql"
 WorkingDirectory = "/var/bacula"
 PidDirectory = "/var/run"
 Maximum Concurrent Jobs = 2
 FDConnectTimeout = 1min
 SDConnectTimeout = 1min
 Password = "$CONSOLE_PASSWORD"                  # "gsdfgfdsg44"
 Messages = Daemon
}

###################   DEFINE CLIENTS   ##############################

Client {
  Name = $SOME_IMPORTANT_CLIENT-fd              # accountant-fd
  Address = $FQDN_OR_IP                         # accountant.mydomain.local
  FDPort = 9102
  Catalog = GiantCatalog
  Password = "$CLIENT_ACCESS_PASSWORD"          # "strong_password"
  File Retention = 6 days
  Job Retention = 6 days
  AutoPrune = yes
  Maximum Concurrent Jobs = 2
}

Client {
  Name = $LAST_CLIENT-fd                       # mainserver-fd
  Address = $FQDN_OR_IP                        # mainserver.mydomain.local
  FDPort = 9102
  Catalog = GiantCatalog
  Password = "$LAST_CLIENT_ACCESS_PASSWORD"    # "strong_password2"
  File Retention = 6 days
  Job Retention = 6 days
  AutoPrune = yes
  Maximum Concurrent Jobs = 2
}

Client {
  Name = $BACKUP_SERVER-fd                       # the bacula server itself backup-fd
  Address = $FQDN_OR_IP                              # backup.mydomain.lan
  FDPort = 9102
  Catalog = GiantCatalog
  Password = "$BACKUP_SERVER_ACCESS_PASSWORD"    # "strong_password2"
  File Retention = 6 days
  Job Retention = 6 days
  AutoPrune = yes
  Maximum Concurrent Jobs = 2
}

############### DEFINE DEFAULT JOB PARAMETERS #####################

JobDefs                     
 {                               
 Name = "DefaultBackupJob"
 Type = Backup
 Level = Full
 Client = backup-fd
 Schedule = "WeeklyCycle"
 Storage = File
 Pool = Default
 Messages = Standard
 Priority = 10
 Max Start Delay = 18h
}

#######################  DEFINE JOBS  ############################

Job
{
 Name = "$SOME_IMPORTANT_CLIENT_JOB"             # "Accountant Backup Job"
 Client = $SOME_IMPORTANT_CLIENT-fd              # accountant-fd
 JobDefs = "DefaultBackupJob"
 FileSet = "Generic User Fileset"                #it can be reused for many clients
 Write Bootstrap = "/backups/bootstraps/$SOME_IMPORTANT_CLIENT.bsr"
 Maximum Concurrent Jobs = 2
}

#Backup the catalog database (after the nightly save)
Job {
  Name = "BackupCatalog"
  JobDefs = "DefaultBackupJob"
  Client = BACKUP_SERVER-fd
  FileSet="Catalog"
  # This creates an ASCII copy of the catalog
  RunBeforeJob = "/etc/bacula/make_catalog_backup bacula bacula"
  # This deletes the copy of the catalog
  RunAfterJob = "/etc/bacula/delete_catalog_backup"
  Write Bootstrap = "/backups/bootstraps/BackupCatalog.bsr"
  Priority = 10
}

Job
{
 Name = "$LAST_CLIENT_JOB"                      # "Mainserver Backup Job"
 Client = $LAST_CLIENT-fd                       # mainserver-fd
 JobDefs = "DefaultBackupJob"
 FileSet = "Server Fileset"                     #we can define a separate FileSet
 Write Bootstrap = "/backups/bootstraps/$LAST_CLIENT.bsr"
 Maximum Concurrent Jobs = 2
 RunAfterJob = "/backups/encrypted/scripts/EncryptVolume.sh"
## NOTE - we will run the script EncryptVolume.sh after this job completes.   ##
## It will encrypt volume, delete unencrypted volume and transfer it via FTP ##
}

################## Define which files to backup #####################

FileSet
{
  Name = "Generic User Fileset"
  Enable VSS = yes
  Include
  {
        Options
    {
        signature = MD5         #Create MD5 Signature
        compression=GZIP5       #Compress Incoming Data on the client
        Exclude = yes
        IgnoreCase = yes
        # Exclude Mozilla-based programs' file caches
        WildDir = "[A-Z]:/Documents and Settings/*/Application Data/*/Profiles/*/*/Cache"
        WildDir = "[A-Z]:/Documents and Settings/*/Application Data/*/Profiles/*/*/Cache.Trash"

        # Exclude directories full of lots and lots of useless little files
        WildDir = "[A-Z]:/Documents and Settings/*/Cookies"
        WildDir = "[A-Z]:/Documents and Settings/*/Recent"
        WildDir = "[A-Z]:/Documents and Settings/*/Local Settings/History"
        WildDir = "[A-Z]:/Documents and Settings/*/Local Settings/Temp"
        WildDir = "[A-Z]:/Documents and Settings/*/Local Settings/Temporary Internet Files"

        # These are always open and unable to be backed up
        WildFile = "[A-Z]:/Documents and Settings/All Users/Application Data/Microsoft/Network/Downloader/qmgr[01].dat"

        # Temporary directories & files
        WildDir = "[A-Z]:/WINNT/Temp"
        WildDir = "[A-Z]:/temp"
        WildFile = "*.tmp"

        # Recycle bins
        WildDir = "[A-Z]:/RECYCLER"

        # Swap files
        WildFile = "[A-Z]:/pagefile.sys"
   }

    File = "c:/Documents and Settings"          #Define Folders to back up
    File = "d:/SomePath/To Folder/Or/File.txt" 
  }

  Exclude
  {
    File = "c:/test"                            #Define Folders to exclude
  }
}

FileSet
{
  Name = "Server Fileset"
  Enable VSS = yes
  Include
  {
        Options
    {
        signature = MD5         #Create MD5 Signature
        compression=GZIP5       #Compress Incoming Data on the client
        Exclude = yes
        IgnoreCase = yes
        # Swap files
        WildFile = "[A-Z]:/pagefile.sys"
   }
    File = "d:/"          #Back Everything for this SPECIAL_USER
  }

  Exclude
  {
    File = "c:/test"                         #Define Folders to exclude
  }
}

###################   DEFINE SCHEDULE   ################################

Schedule {
  Name = "WeeklyCycle"
  Run = Level=Full Pool=MondayPool Monday at 12:05       #PM That is
  Run = Level=Full Pool=TuesdayPool Tuesday at 02:12     #AM that is
  Run = Level=Full Pool=WednesdayPool Wednesday at 12:05
  Run = Level=Full Pool=ThursdayPool Thursday at 12:05
  Run = Level=Full Pool=FridayPool Friday at 12:05
  Run = Level=Full Pool=SaturdayPool Saturday at 16:00
  Run = Level=Full Pool=SundayPool Sunday at 16:00
}

################# DEFINE STORAGE (file, tape, CDR..) ###################

Storage {
  Name = File                   
  Address = FQDN_OR_IP_OF_DIRECTOR         
  SDPort = 9103
  Password = "$STORAGEPASS"
  Device = FileStorage
  Media Type = File
}

############################ DEFINE CATALOG #############################

Catalog {
  Name = GiantCatalog
  dbname = bacula; user = bacula; password = ""
}

########################## DEFINE MESSAGES #############################

Messages {
  Name = Standard
  console = all, !skipped, !saved
  append = "/var/bacula/log" = all, !skipped
}

Messages {
  Name = Daemon
  console = all, !skipped, !saved
  append = "/var/bacula/log" = all, !skipped
}

#######################   DEFINE POOLS   ################################
Pool {
  Name = MondayPool
  Pool Type = Backup
  AutoPrune = yes               # Prune expired volumes
  Volume Retention = 6 days
  Accept Any Volume = yes       # write on any volume in the pool
  Volume Use Duration = 6 days
  Recycle = yes
  RecycleOldestVolume = yes
  LabelFormat = "BackupMondayVolume"
}

Pool {
  Recycle = yes                 
  Name = TuesdayPool
  Pool Type = Backup
  AutoPrune = yes                   
  Volume Retention = 6 days
  Accept Any Volume = yes             
  Volume Use Duration = 6 days
  Recycle = yes
  RecycleOldestVolume = yes
  LabelFormat= "BackupTuesdayVolume"
}

Pool {
  Recycle = yes               
  Name = WednesdayPool
  Pool Type = Backup
  AutoPrune = yes                     
  Volume Retention = 6 days
  Accept Any Volume = yes             
  Volume Use Duration = 6 days
  Recycle = yes
  RecycleOldestVolume = yes
  LabelFormat="BackupWednesdayVolume"
}

Pool {
  Recycle = yes                 
  Name = ThursdayPool
  Pool Type = Backup
  AutoPrune = yes                     
  Volume Retention = 6 days
  Accept Any Volume = yes             
  Volume Use Duration = 6 days
  Recycle = yes
  RecycleOldestVolume = yes
  LabelFormat="BackupThursdayVolume"
}

Pool {
  Recycle = yes                 
  Name = FridayPool
  Pool Type = Backup
  AutoPrune = yes                   
  Volume Retention = 6 days
  Accept Any Volume = yes           
  Volume Use Duration = 6 days
  Recycle = yes
  RecycleOldestVolume = yes
  LabelFormat="BackupFridayVolume"
}

Pool {
  Recycle = yes                 
  Name = SaturdayPool
  Pool Type = Backup
  AutoPrune = yes                   
  Volume Retention = 6 days
  Accept Any Volume = yes           
  Volume Use Duration = 6 days
  Recycle = yes
  RecycleOldestVolume = yes
  LabelFormat="BackupSaturdayVolume"
}

Pool {
  Recycle = yes               
  Name = SundayPool
  Pool Type = Backup
  AutoPrune = yes                 
  Volume Retention = 6 days
  Accept Any Volume = yes           
  Volume Use Duration = 6 days
  Recycle = yes
  RecycleOldestVolume = yes
  LabelFormat="BackupSundayVolume"
}

# Default pool definition
Pool {
  Name = Default
  Pool Type = Backup
}

# Restricted console used by tray-monitor to get the status of the director
Console {
  Name = backup-mon
  Password = "$RESTRICTED_CONSOLE_PASSWORD"
  CommandACL = status, .status
} 

Define Storage Daemon (SD)

File: /etc/bacula/bacula-sd.conf
Storage {                             # definition of myself
  Name = backup-sd
  SDPort = 9103                 
  WorkingDirectory = "/var/bacula"
  Pid Directory = "/var/run"
  Maximum Concurrent Jobs = 20
}

# List Directors who are permitted to contact Storage daemon
Director {
  Name = backup-dir
  Password = "$STORAGEPASS"
}

#
# Restricted Director, used by tray-monitor to get the
#   status of the storage daemon
#
Director {
  Name = backup-mon
  Password = "$RESTRICTED_DIR_PASS"
  Monitor = yes
}

Device {
  Name = FileStorage
  Media Type = File
  Archive Device = /backups
  LabelMedia = yes;                   # lets Bacula label unlabeled media
  Random Access = Yes;
  AutomaticMount = yes;               # when device opened, read it
  RemovableMedia = no;
  AlwaysOpen = no;
}

Messages {
  Name = Standard
  director = backup-dir = all
}

Define Console

File: /etc/bacula/bconsole.conf
Director {
  Name = backup-dir            #mainofficebackup-dir
  DIRport = 9101
  address = $FQDN_OR_IP
  Password = "$DIRECTOR_CONSOLE_PASSWORD"
}

Create the backup script

This is the script that will encrypt the backup volume and upload it to a different server This script is to be executed via Bacula's RunAfterJob or Cron. It will encrypt a backup volume with GnuPG using Twofish algorithm. If encryption is to be changed use only encryption with 128-bit BLOCK size. Currently only Twofish, AES, MARS and Serpent have 128bit block.

Ciphers with 64-bit block (3DES, CAST5, BLOWFISH...) are not suitable for encrypting large files. Couple Gig volume is Ok, but as you will approach 64 Gigs, you will increase the risk of leaking decryption information (see http://en.wikipedia.org/wiki/Birthday_attack).

File: /backups/encrypted/scripts/EncryptVolume.sh
# Remove Yesterday’s encrypted backup from HDD
cd /backups/encrypted/
rm -f *.gpg

# encrypt the file with GPG using Public Key.
# use only --cipher-algo aes|aes192|aes256|twofish
gpg --encrypt --batch -r backup@giantmarkets.com  --cipher-algo twofish \
--bzip2-compress-level 6 --output /backups/encrypted/BakupVolume.gpg /backups/Backup*

# Remove unencrypted backup volume, we encrypted and saved
# encrypted copy in the above step.
cd /backups/
rm Backup*

# transfer encrypted volume via ftp to the server with tape
cd /backups/encrypted/

ftp -ni $FQDN_OR_IP <<SCRIPT
user Anonymous backup_job@backup
cd $SOME_FOLDER
binary
mdelete *.gpg
mput *.gpg
quit

Please note. If the script works manually, but fails to encrypt automatically (trust errors), copy the .gnupg folder in the home directory for the working user (root for example) into the root directory

cp -r /root/.gnupg/ /

Win32 Client Installation and Configuration

Download and install the winbacula.x.x.x.exe Edit the bacula-fd.conf to let your director to connect to the client

File: c:\bacula\bin\bacula-fd.conf
Director {
  Name = backup-dir			#Director's name as defined on the server
  Password = "$CLIENT_ACCESS_PASSWORD"	#same as in client definition on server
}

FileDaemon {                          
  Name = $SOME_IMPORTANT_CLIENT-fd	
  FDport = 9102                  	
  WorkingDirectory = "C:\\Documents and Settings\\All Users\\Application Data\\Bacula\\Work"
  Pid Directory = "C:\\Documents and Settings\\All Users\\Application Data\\Bacula\\Work"
  Maximum Concurrent Jobs = 2
}

# Send all messages except skipped files back to Director
Messages {
  Name = Standard
  director = backup-dir = all, !skipped
}

(re)Start the Bacula service and you are ready.

*NIX Client Installation and Configuration

1. Download latest bacula source from SourceForge.net
2 Extract the bacula source
tar -xvzf bacula-x.xx.xx.tar.gz
3 Enter the source folder
cd bacula-x.xx.xx
4 Run the configure command
./configure \
--prefix=/usr \
--sbindir=/usr/sbin \
--sysconfdir=/etc/bacula \
--with-scriptdir=/etc/bacula \
--enable-smartalloc \
--with-mysql \
--with-working-dir=/var/bacula \
--with-pid-dir=/var/run \
--with-subsys-dir=/var/lock/subsys \
--enable-conio \
--with-openssl \
--enable-largefile \
--enable-wx-console \
--with-python \
–-enable-client-only 
5 Compile and Install
make
make install
6 Add bacula to the /etc/conf.d/local.start
echo /etc/bacula/bacula start >> /etc/conf.d/local.start
7 add bacula to the /etc/conf.d/local.stop
echo /etc/bacula/bacula stop >> /etc/conf.d/local.stop

8: Edit the bacula-fd.conf

File: c:\bacula\bin\bacula-fd.conf
Director {
  Name = backup-dir			        #Director's name as defined on the server
  Password = "$LAST_CLIENT_ACCESS_PASSWORD"	#same as in client definition on server
}

FileDaemon {                          
  Name = $LAST_CLIENT-fd	
  FDport = 9102                  	
  WorkingDirectory = /bacula/working
  Pid Directory = /bacula/working
}

# Send all messages except skipped files back to Director
Messages {
  Name = Standard
  director = backup-dir = all, !skipped
}

(re)Start the Bacula service and you are ready.

Restoration Process

1. Decrypt the file

gpg BackupVolume.gpg

(Assuming that gpg has your private key it will ask for the password)

2. Move the decrypted file to the bacula server (/backups directory in this example)

3a. If Bacula database is intact Follow the restoration procedures in the Bacula handbook http://bacula.org/dev-manual/Restore_Command.html

OR

3b. If Bacula database is not intact (new server for example) Use the bscan utility to recreate the database from your volume.

cd /etc/bacula/
bscan -V RestoreVolume -v -s -m -c bacula-sd.conf FileStorage

and then follow the restoration procedures in the Bacula handbook http://bacula.org/dev-manual/Restore_Command.html

Retrieved from "http://www.gentoo-wiki.info/HOWTO_Encrypted_Backup_System_Using_Bacula_and_GnuPG"

Last modified: Sat, 06 Sep 2008 07:21:00 +0000 Hits: 14,296