Author: Cody Dunne
Last Edited: 07/27/2011
Most of this guide is not original work, and is based substantially on the numerous sources available online for these tools. It does, however, combine many of the disparate sources for this information, add some of my own experience and insights, and provides an excellent backup scheme for Windows 7/Vista/XP clients. Hopefully it will help other users in this process and prevent them from spending as much time getting it working as I did.
Useful reading for BackupPC are the FAQ, wiki, mailing list, HowtoForge article, setup guide, and UCSD manual. Info on SSH tunnels can be found at SSH Tunnel HOWTO for BackupPC, Auto-closing SSH tunnels and this thread on the BackupPC mailing list. Cygwin info is at the project site http://cygwin.com.
These instructions are written for Windows 7, Vista, or XP clients. We will be installing Cygwin to provide access to Linux tools and using rsync over a SSH tunnel created with OpenSSH. If you are using an untrusted network or backing up clients over the Internet you definitely want to do it over an encrypted SSH tunnel.
First create a backuppc user account and add it to the Administrators
and Backup Operators
groups. Set it so the password never expires. Make sure every Windows user has a password set.
Login to the new backuppc account and install Cygwin using their setup.exe. It is easier to create the root directory before installation so the BackupPC user has the necessary permissions already. Good configuration choices are Root Directory -> c:\cygwin
and LocalPackageDirectory -> c:\cygwin\packages
. Packages you need are below, where '*' signifies optional packages:
Dependencies should be selected automatically.
When completed, edit the path system variable. On Windows 7/Vista right click Computer -> Properties -> Advanced System Settings. On XP, right click My Computer -> Properties -> Advanced. Then for either, click Environment Variables -> System Variables and edit the PATH
variable adding ;c:\cygwin\bin
to the end. You can reboot to make sure it was successful.
Open the Cygwin Bash Shell by running
C:\cygwin\Cygwin.bator clicking the
Cygwin Bash Shell
shortcut in the start menu. It is important to set them to run as administrator, though. Otherwise you may have problems with permissions. Right-click on the shortcut and go to 'properties'. Click 'advanced' and check the 'Run as administrator' box and hit 'ok'.
This should be done from the backuppc account. See How to install OpenSSH sshd... and Install Cygwin and SSH on a Windows Client for more details.
Create ssh-keys with
ssh-host-configAnswer yes for privilege separation and to have sshd as a service. Leave the value of cygwin for the daemon the default, and the default name is fine too. Answer yes to create a privileged account, and enter a password for
cyg_server
that will be printed in clear text.
chmod u+w,+r /etc/passwd /etc/group chmod +x /varJust make sure to only create the permissions necessary and not more. For example, don't use
chmod 777
.
When questioned about environment variable CYGWIN
press enter to use the default value. Then start the SSH server with
net start sshd
The CYGWIN sshd service is starting. The CYGWIN sshd service could not be started. The service did not report an error. More help is available by typing NET HELPMSG 3534.try this solution.
Finally, run
ssh-user-configto configure keys for the backuppc user to generate a SSH2 RSA key. Without this step, passwordless SSH will not work.
ssh backuppc@localhostand use the password you chose to verify SSH is working. Then run from another machine
ssh backuppc@client
C:\cygwin\usr\sbin\sshd.exe
to the Windows firewall or any other firewall (e.g. - Norton). You may also need to add firewall exceptions for echo (ping) and file sharing so nmblookup works as BackupPC needs to be able to find the client with these.
/var/log/sshd.log
for lines like 5 [main] sshd 1728 child_copy: linked dll data write copy failed, 0x265000..0x2659F8, done 0, windows pid 1828, Win32 error 87Sometimes antivirus or firewall software can cause these, though I was able to solve this by running
rebaseall
. First, close all Cygwin processes (bash, sshd, rsyncd, etc.) and start a CMD prompt with administrative privileges. Then runcd c:\cygwin\bin ash /usr/bin/rebaseallAfter restarting sshd, test to see if it works.
Modify /etc/sshd_config
.
chmod a+w /etc/sshd_config nano /etc/sshd_configChange or add these lines, where
highport
is a high port number for SSH connections and bpcserverip
is the IP address of the BackupPC server. If you are configuring a remote client (over the Internet), omit the @backuppcserverip
portion unless you have a static IP address for the server.
Port highport LoginGraceTime 30 PermitRootLogin no MaxStartups 3:50:10 AllowUsers backuppc@bpcserveripBy using an unusual port and allowing only particular user:ip pairs you limit intrusion attempts on your client.
MaxStartups
and LoginGraceTime
limit the number and length of new connections to reduce the speed of brute force password crackers.
Afterwards, restore the old permissions with
chmod go-w /etc/sshd_configYou can now test SSH with other accounts on the client, IP addresses, etc. to make sure the connection rules block them.
For mobile clients, you can disable SSH over the wireless interface or when roaming. Add this to /etc/sshd_config
like before to only listen on wiredipaddress
, which is the static IP address of the wired adapter.
ListenAddress wiredipaddress
sshd
will fail to start. Similarly, when the laptop is roaming on other networks the CYGWIN sshd
service will stop with these messages in the Event Viewer (eventvwr.msc
): Information: sshd: PID 2972: `sshd' service stopped, exit status: 255 Error: sshd: PID 3056: fatal: Cannot bind any address. Error: sshd: PID 3056: error: Bind to port XXXXX on XXX.XXX.XXX.XXX failed: Cannot assign requested address.One way to work around this is to start
sshd
when connecting to the home network. Another is to have CYGWIN sshd
try to restart itself after failures. Run services.msc
and find CYGWIN sshd
. Right-click, open its properties, and go to the Recovery tab. There you can specify Restart the Service
as the 1st, 2nd, and subsequent failure options and specify a time to wait before restarting, say 120 minutes.
Set up port forwarding in their router so we can SSH to them remotely. Pick high external ports so it is more difficult to find them. Each client needs a different external port, though they can all have the same internal SSH port. You can set up DynDNS or the like if your router supports it or on one of your clients if you don't have a static IP for your router. This allows you to find their IP remotely. You can then test SSH using your own hostname and external port:
ssh dyndnshostname -p highexternalport
You can ensure each client's fingerprint is correct before you cache them on the server. The fingerprint should match the client's output of
ssh-keygen -l -f /etc/ssh_host_rsa_key
The easiest transfer option is rsync running over SSH. If you have the latest Cygwin and followed the SSH setup above, you don't need to do anything else on the client.
You may want to configure the clients to automatically wake up when BackupPC connects to them so they don't need to always be on. First, configure Wake on LAN in the BIOS and configure the network interface. Under the Power Management tab set AC Recover -> Last
, Suspend Mode -> S3
, and Remote Wake Up -> On
. You may want to configure BackupPC to put a client in standby after a backup. If so, run the command below from KB555569 as the DumpPostUserCmd
.
/cygdrive/c/Windows/System32/rundll32.exe powrprof.dll,SetSuspendState
Ubuntu is an excellent OS for the BackupPC server. BackupPC is avaiable via the Synaptic Package Manager in Ubuntu for extremely easy installation.
Install BackupPC and sshd so you can connect remotely.
sudo apt-get install backuppc sudo apt-get install openssh-server
Set a password for the backuppc web user.
sudo htpasswd /etc/backuppc/htpasswd backuppc
We need to setup SSH for the backuppc user. Follow the default suggestions for ssh-keygen and don't use a passphrase. You could try to use ssh-agent or Seahorse to store the passphrases for encrypted keys, but it is difficult to ensure they are unlocked whenever the backuppc user needs them.
sudo -i su backuppc bash ssh-keygen -t rsa
Setup passwordless SSH for each of the clients so the backuppc user can connect without prompts. First copy the backuppc user's public key to the authorized_users
file on each client like below. Here highport
is the SSH port you set for the client if you secured SSH, otherwise you can omit -p highport
and the apostrophes.
ssh-copy-id '-p highport username@client'Or, if you don't have
ssh-copy-id
, use the following.
scp -P highport /var/lib/backuppc/.ssh/id_rsa.pub username@client:~/.ssh/backuppc_id_rsa.pub ssh -p highport username@client cat ~/.ssh/backuppc_id_rsa.pub >> ~/.ssh/authorized_keys rm ~/.ssh/backuppc_id_rsa.pub
Most of the global settings are easily configured via the web interface, at http://localhost/backuppc but you can also edit /etc/backuppc/config.pl
directly.
Define each host you want to back up. It is easiest to automate email notifications if you use the user's email address as their username.
ClientCharset -> cp1252
Only choose if your primary clients run Windows - use the default UTF8 for other clients. If you don't set this as default, you'll have to configure it for each Windows client.
XferMethod -> rsync
RsyncShareName -> /cygdrive/c
BackupFilesExclude
BackupFilesExclude is easier to modify in /etc/backuppc/config.pl
directly. Pick the exclude list for the primary operating system of your clients. Unfortunately, you will have to hard code the exclude list for your other clients. For example, if Windows 7 or Vista is the primary OS and XP is the only other one, use the 7/Vista excludes below in config.pl
and the XP excludes in each clients clientname.pl
file. You can check out the wiki page I wrote part of to get ideas for excludes for Windows 2000 or Linux.
You may also want to exclude some of the Cygwin directories like /bin
, /tmp
and /packages
(where I told Cygwin to put the package directory during setup). Just make sure to backup /etc
so you don't lose your configuration!
I ran into a lot of problems on my Windows 7 and Vista machines where rsync would follow the junction points Microsoft added for backward compatibility (see this site for more info). This caused extra-long filenames rsync couldn't handle.
dir /aL /s > c:\users\USERNAME\JunctionPoints.txt
The rules below will exclude these junction points. Rules with asterisks in them will match the junction points that are in every user profile by default without having to code excludes for each user manually. We can remove any temp data and program or system installation files as well. On 7/Vista, only original installation data is stored in Program Files
. Any data programs write to their installation folder goes to ProgramData
automatically instead. The Windows
folder shouldn't hold anything interesting, either.
Exclude list for Windows 7/Vista clients with the rsync share being /cygdrive/c
:
$Conf{BackupFilesExclude} = { #Windows 7/Vista specific! '*' => [ #7/Vista junction points '/Documents and Settings', '/ProgramData/Application Data', '/ProgramData/Desktop', '/ProgramData/Documents', '/ProgramData/Favorites', '/ProgramData/Start Menu', '/ProgramData/Templates', '/Users/All Users', '/Users/Users/Default User', '/Users/Users/All Users/Application Data', '/Users/Users/All Users/Desktop', '/Users/All Users/Documents', '/Users/All Users/Favorites', '/Users/All Users/Start Menu', '/Users/All Users/Templates', #Junction points common to every user profile '/Users/*/Application Data', '/Users/*/Cookies', '/Users/*/Local Settings', '/Users/*/My Documents', '/Users/*/NetHood', '/Users/*/PrintHood', '/Users/*/Recent', '/Users/*/SendTo', '/Users/*/Start Menu', '/Users/*/Templates', '/Users/*/AppData/Local/Application Data', '/Users/*/AppData/Local/History', '/Users/*/AppData/Local/Temporary Internet Files', '/Users/*/Documents/My Music', '/Users/*/Documents/My Pictures', '/Users/*/Documents/My Videos', #Temporary and in-use user data '/Users/*/AppData/Local/Microsoft/Windows/Temporary Internet Files', '/Users/*/AppData/Local/Temp', '/Users/*/NTUSER.DAT*', '/Users/*/ntuser.dat*', '/Users/*/AppData/Local/Microsoft/Windows/UsrClass.dat*', '/Users/*/AppData/Local/Microsoft/Windows Defender/FileTracker', '/Users/*/AppData/Local/Microsoft/Windows/Explorer/thumbcache_*.db', '/Users/*/AppData/Local/Microsoft/Windows/WER', '/Users/*/AppData/Local/Mozilla/Firefox/Profiles/*/Cache', '/Users/*/AppData/Local/Mozilla/Firefox/Profiles/*/OfflineCache', '/Users/*/AppData/Roaming/Microsoft/Windows/Cookies', '/Users/*/AppData/Roaming/Microsoft/Windows/Recent', 'ProgramData/Microsoft/Search', 'ProgramData/Microsoft/Windows Defender', '*.lock', 'Thumbs.db', 'IconCache.db', 'Cache*', 'cache*', #Installation folders and system data '/Program Files', '/Windows', '/$Recycle.Bin', '/MSOCache', '/System Volume Information', '/Boot', '/autoexec.bat', '/bootmgr', '/BOOTSECT.BAK', '/config.sys', '/hiberfil.sys', '/pagefile.sys' ] };
Exclude list for Windows XP clients with the rsync share being /cygdrive/c
:
$Conf{BackupFilesExclude} = { #XP specific! '*' => [ #Temporary and in-use user data '/Documents and Settings/*/Cookies', '/Documents and Settings/*/Local Settings/Temporary Internet Files', '/Documents and Settings/*/Local Settings/Temp', '/Documents and Settings/*/NTUSER.DAT*', '/Documents and Settings/*/ntuser.dat*', '/Documents and Settings/*/Local Settings/Application Data/Microsoft/Windows/UsrClass.dat*', '/Documents and Settings/*/Local Settings/Application Data/Mozilla/Firefox/Profiles/*/Cache', '/Documents and Settings/*/Local Settings/Application Data/Mozilla/Firefox/Profiles/*/OfflineCache', '/Documents and Settings/*/Recent', '*.lock', 'Thumbs.db', 'IconCache.db', 'Cache*', 'cache*', #Installation folders and system data '/WINDOWS', '/RECYCLER', '/MSOCache', '/System Volume Information', '/AUTOEXEC.BAT', '/BOOTSECT.BAK', '/CONFIG.SYS', '/hiberfil.sys', '/pagefile.sys' ] };
EMailAdminUserName -> youremail@host.com
CgiUserConfigEditEnable -> no
You probably don't want users editing their configuration.
FullKeepCnt -> 4, 2, 3, 0, 0, 4
Spreads out backups exponentially over almost three years. The schedule is approximately 0wk (current), 1wk, 2wk, 3wk, 5wk, 7wk, 11wk, 15wk, 19wk, 51wk/1yr, 83wk/1.6yr, 115wk/2.2yr, 147wk/2.8yr.
FullAgeMax -> 1095
3 years.
IncrLevels -> 3, 2, 5, 4, 7, 6
This is based on the Tower of Hanoi puzzle and is a way of balancing your backups so you don't backup the same data a bunch of times yet retain duplicate backups of the data. See Tower of Hanoi pattern for backup for more info.
BlackoutPeriods
BlackoutPeriods:hourEnd -> 1
Better for night owls.
BlackoutPeriods:weekDays -> 0, 1, 2, 3, 4, 5, 6
Avoid the same times every day for a home environment.
RsyncClientCmd -> $sshPath -q -x -l backuppc -p highport $host $rsyncPath $argList+
Add the -p highport
if you set a different client SSH port (highport
) when securing SSH. Otherwise, there is no change
RsyncClientRestoreCmd -> $sshPath -q -x -l backuppc -p highport $host $rsyncPath $argList+
Add the -p highport
as in RsyncClientCmd
PingCmd -> /etc/backuppc/mktnl clienthostname clientsshport localport
In order to setup the SSH tunnel beforehand, we can create script that BackupPC can run before the backup. I'm placing it in the PingCmd
field so it is run when BackupPC pings the host before it starts. It also allows me to return a valid ping even though we set the alias above to localhost.
Here is the contents of /etc/backuppc/mktnl
:
#!/bin/sh host=$1 sshPort=$2 locPort=$3 remPort=873 username=backuppc /bin/ping -c 1 -w 3 $host /usr/bin/ssh -f -L $locPort:localhost:$remPort $username@$host -p $sshPort sleep 10 1>/dev/null 2>/dev/null && echo "SSH tunnel started successfully."
The ping command returns its results to BackupPC and the SSH tunnel is started and prints into the log for you. The sleep 10
makes the tunnel wait for 10 seconds after all traffic finishes before closing.
PingMaxMsec -> 200
Only set for external (over the Internet) hosts.
BlackoutPeriods
hourEnd
and hourBegin
adjusted for user.
You can test backups for each client by running the following where hostname
is the name of the host in the BackupPC global settings.
/usr/share/backuppc/bin/BackupPC_dump -f -v hostname
You need to set a password for each email address/username you defined previously so users can access their backup pages. Run the following for each host where clientemail
is the email address/username you defined for that host.
htpasswd /etc/backuppc/htpasswd clientemail
The easiest way for a home user to do this is via Gmail. See Using Exim4 to send Messages through Gmail and this post for more info. First, set up POP in a new Gmail account. Set smtp.gmail.com as the smarthost by running
dpkg-reconfigure exim4-configand picking to use a smarthost on the second page (
mail sent by smarthost; no local mail
). You can then edit /etc/exim4/update-exim4.conf.conf
to make sure dc_smarthost='smtp.gmail.com'
Next, add these lines to /etc/exim4/passwd.client
:
smtp.gmail.com : ACCOUNT_NAME@gmail.com : PASSWORD gmail-smtp.l.google.com : ACCOUNT_NAME@gmail.com: PASSWORD
To test email delivery, try
/usr/share/BackupPC/bin/BackupPC_sendEmail -u emailaddress
tail /var/log/exim4/mainlogand see the current queue of messages with
mailqOne common problem is finding an
Authentication Required
message in your log. If the host is something other than gmail-smtp.l.google.com
, replace it in your /etc/exim4/passwd.client
with whatever it says. Once you've fixed your problem, you can send all frozen messages with exim -qff
You can add user: email
combos to /etc/aliases
for the backuppc and/or root users so you get admin mail for your box, too. Afterwards, recompute the Sendmail aliases with
newaliases
To schedule checks for BackupPC status, use cron (see this site for more info). Run
crontab -eand add the following line to get a message every half hour if the BackupPC server goes down:
*/30 * * * * /usr/share/backuppc/bin/BackupPC_sendEmail -c
If you want to send your email using the message submit port (587) instead of smtp (25), see this post. This will allow you to get around the port 25 blocks some ISPs use to reduce spam on their networks.
You probably want to use a separate hard drive or group of drives for your BackupPC pool for easier maintenance, expansion, and reliability. We are going to use Logical Volume Manager (LVM) to create a resizable pool we can easily add more drives to.
LVM allows us to create expandable logical volumes across many drives. See A Beginner's Guide To LVM for more detailed setup instructions. Also check out Managing Disk Space with LVM and A simple introduction to working with LVM.
Set up your drive using fdisk. The command below is for a drive at /dev/sdb
fdisk -l fdisk /dev/sdbThe choices you want to make are:
n, p, 1, ENTER, ENTER, t, 1, L, 8e, w
. Then run
pvcreate /dev/sdb1 pvdisplay vgcreate bpcfs /dev/sdb1 vgdisplay vgscan lvcreate --name bpclv --extents 100%FREE bpcvg lvdisplay lvscan mkfs.ext3 /deb/bpcvg/bpclv
modprobe dm-mod
See here for more info. If your new partition is /dev/bpcvg/bpclv
, execute the following commands as root
mount /dev/bpcvg/bpclv /mnt cp -dpR /var/lib/backuppc/. /mnt umount /dev/md0 mv /var/lib/backuppc /var/lib/backuppc.orig mkdir /var/lib/backuppc mount /dev/bpcvg/bpclv /var/lib/backuppc df -hThis makes a backup of your data then copies it to the new drive. Once you're sure that worked, you need to make it mount on startup. Edit
/etc/fstab
and add the following line:
/dev/bpcvg/bpclv /var/lib/backuppc ext3 rw,noatime 0 0Then restart and run
df -hto make sure it worked.
You can use the web interface to restore ZIP or TAR archives, but the current version of BackupPC doesn't always produce readable ZIP files. There is a bug in Archive::Zip 1.30 that prevents some BackupPC ZIP restores. TAR restores are unaffected, however.