Author: Cody Dunne
Last Edited: 12/26/2009
Most of this guide is not original work, and is based substantially on the numerous sources available online for these tools. It does, however, combine many of the disparate sources for this information, add some of my own experience and insights, and provides an excellent backup scheme for Windows 7/Vista/XP clients. Hopefully it will help other users in this process and prevent them from spending as much time getting it working as I did.
Useful reading for BackupPC are the FAQ, wiki, mailing list, HowtoForge article, setup guide, and UCSD manual. Info on SSH tunnels can be found at SSH Tunnel HOWTO for BackupPC, Auto-closing SSH tunnels and this thread on the BackupPC mailing list. Cygwin info is at the project site http://cygwin.com.
These instructions are written for Windows 7, Vista, or XP clients. We will be installing Cygwin to provide access to Linux tools and using rsyncd over a SSH tunnel created with OpenSSH. You could just use rsyncd, and you may want to for testing, but if you are using an untrusted network or backing up clients over the Internet you want to do it over an encrypted SSH tunnel. Regular rsync over SSH doesn't work between Linux and Windows/Cygwin boxes because of a bug fixed in rsync 3.0. However, BackkupPC doesn't use the features of rsync 3.0 and the problem remains. See this post on the BackupPC mailing list for details. Using rsyncd over a pre-established SSH tunnel or VPN is the best way around this.
First create a backuppc user account and add it to the Administrators and Backup Operators groups. Set it so the password never expires. Make sure every Windows user has a password set.
Login to the new backuppc account and install Cygwin using their setup.exe. It is easier to create the root directory before installation so the BackupPC user has the necessary permissions already. Good configuration choices are Root Directory -> c:\cygwin and LocalPackageDirectory -> c:\cygwin\packages. Packages you need are nano* (text editor), openssh (encryption), rsync (transfer method), shutdown* (for remote shutdown and restart), and wget* (for downloading). '*' signifies optional packages, and dependencies that should be selected automatically are diffutils (needed utilities) and zlib (needed utilities).
When completed, create a system environment variable by right clicking My Computer -> Properties -> Advanced -> Environment Variables -> System Variables and creating a new system variable CYGWIN = ntsec tty and edit the PATH variable adding ;c:\cygwin\bin to the end. You can reboot to make sure it was successful.
Open the Cygwin Bash Shell by running
C:\cygwin\Cygwin.bator clicking the
Cygwin Bash Shell shortcut in the start menu. It is important to set them to run as administrator, though. Otherwise you may have problems with permissions. Right-click on the shortcut and go to 'properties'. Click 'advanced' and check the 'Run as administrator' box and hit 'ok'.
This should be done from the backuppc account. See How to install OpenSSH sshd... and Install Cygwin and SSH on a Windows Client for more details.
Create ssh-keys with
ssh-host-configAnswer yes to all prompts. If you run into errors with permissions, you can fix them like for these examples:
chmod u+w,+r /etc/passwd /etc/group chmod +x /varJust make sure to only create the permissions necessary and not more. For example, don't use
chmod 777.
When questioned about environment variable CYGWIN type
ntsecThen start the SSH server with
net start sshd
Finally, run
ssh-user-configto configure keys for the backuppc user. Without this step, passwordless SSH will not work.
ssh backuppc@localhostand use the password you chose to verify SSH is working. Then run from another machine
ssh backuppc@clientIf you have problems you may need to add exceptions for
/usr/sbin/sshd.exe to the Windows firewall or any other firewall (e.g. - Norton). You may also need to add firewall exceptions for echo (ping) and file sharing so nmblookup works as BackupPC needs to be able to find the client with these.
Additionally, if you run into frequent errors connecting check in /var/log/sshd.log for lines like
5 [main] sshd 1728 child_copy: linked dll data write copy failed, 0x265000..0x2659F8, done 0, windows pid 1828, Win32 error 87Sometimes antivirus or firewall software can cause these, though I was able to solve this by running
rebaseall. First, close all Cygwin processes (bash, sshd, rsyncd, etc.) and start a CMD prompt with administrative privileges. Then runcd c:\cygwin\bin ash /usr/bin/rebaseallAfter restarting sshd, test to see if it works.
Modify /etc/sshd_config.
chmod a+w /etc/sshd_config nano /etc/sshd_configChange or add these lines, where
highport is a high port number for SSH connections and bpcserverip is the IP address of the BackupPC server. If you are configuring a remote client (over the Internet), omit the @backuppcserverip portion unless you have a static IP address for the server.
LoginGraceTime 30 PermitRootLogin no Port highport MaxStartups 3:50:10 AllowUsers backuppc@bpcserveripBy using an unusual port and allowing only particular user:ip pairs you limit intrusion attempts on your client.
MaxStartups and LoginGraceTime limit the number and length of new connections to reduce the speed of brute force password crackers.
For laptops, add this value as well if you want to disable SSH over the wireless interface where wiredipaddress is the IP address of your wired adapter.
ListenAddress wiredipaddressAfterwards, restore the old permissions with
chmod go-w /etc/sshd_config
You can now test SSH with other accounts on the client, IP addresses, connecting over the wireless, etc to make sure the connection rules block them.
Set up port forwarding in their router so we can SSH to them remotely. Pick high external ports so it is more difficult to find them. Each client needs a different external port, though they can all have the same internal SSH port. You can set up DynDNS or the like if your router supports it or on one of your clients if you don't have a static IP for your router. This allows you to find their IP remotely. You can then test SSH using your own hostname and external port:
ssh dyndnshostname -p highexternalportAgain, if you have problems you may need to add firewall exceptions.
You can ensure each client's fingerprint is correct before you cache them on the server. The fingerprint should match the client's output of
ssh-keygen -l -f /etc/ssh_host_rsa_key
To configure rsyncd see RsyncdWindowsClients on the BackupPC wiki. First, Create /etc/rsyncd.conf with the following:
gid = users read only = true use chroot = false transfer logging = false log file = /var/log/rsyncd.log log format = %h %o %f %l %b hosts allow = 127.0.0.1 ::1 hosts deny = * auth users = backuppc secrets file = /etc/rsyncd.secrets strict modes = false list = false [user] path = /cygdrive/c/Users #exclude from = /etc/exclude-user.txt [cyg] path = /cygdrive/c #exclude from = /etc/exclude-cyg.txtTwo shares,
user and cyg are defined here, and the commented exclude files could be used for client-side exclude lists instead of sending them from the server each time. That would prevent the excludes from showing in your logs, but makes them more difficult to change. See my post rsyncd Exclusions for using BackupPC on Windows 7 and Vista for what you could fill /etc/exclude-cyg.txt with. We will only be using cyg in the examples.
Next, you need to create a username and password in /etc/rsyncd.secrets like so:
backuppc:PASSWORDThen create and start rsyncd service with
cygrunsrv.exe -I "rsyncd" -p /cygdrive/c/cygwin/bin/rsync.exe -a "--config=/cygdrive/c/cygwin/etc/rsyncd.conf --daemon --no-detach" -f "Rsync daemon service" net start rsyncdIf you have problems you may need to add exceptions for
/usr/bin/rsync.exe to the Windows firewall or any other firewall (e.g. - Norton).
You may want to configure the clients to automatically wake up when BackupPC connects to them so they don't need to always be on. First, configure Wake on LAN in the BIOS and configure the network interface. Under the Power Management tab set AC Recover -> Last, Suspend Mode -> S3, and Remote Wake Up -> On. You may want to configure BackupPC to put a client in standby after a backup. If so, run the command below from KB555569 as the DumpPostUserCmd.
/cygdrive/c/Windows/System32/rundll32.exe powrprof.dll,SetSuspendState
Ubuntu is an excellent OS for the BackupPC server. BackupPC is avaiable via the Synaptic Package Manager in Ubuntu for extremely easy installation. The latest version of Ubuntu, 8.10/Intrepid Ibex, includes rsync 3.03 which is more efficient and will fix the Cygwin/rsync/SSH bug when BackupPC's File:RsyncP is updated to include at least protocol version 30.
Install BackupPC and sshd so you can connect remotely.
apt-get install backuppc apt-get install openssh-server
Set a password for the backuppc web user.
htpasswd /etc/backuppc/htpasswd backuppc
We need to setup SSH for the backuppc user. Follow the default suggestions for ssh-keygen and don't use a passphrase. You could try to use ssh-agent or Seahorse to store the passphrases for encrypted keys, but it is difficult to ensure they are unlocked whenever the backuppc user needs them.
sudo -i su backuppc bash ssh-keygen -t rsa
Setup passwordless SSH for each of the clients so the backuppc user can connect without prompts. First copy the backuppc user's public key to the authorized_users file on each client.
ssh-copy-id username@clientor, if you don't have
ssh-copy-id, use the following where highport is the SSH port you set for the client if you secured SSH. Otherwise you can omit -P highport.
scp -P highport /var/lib/backuppc/.ssh/id_rsa.pub username@client:~/.ssh/backuppc_id_rsa.pub ssh username@client cat ~/.ssh/backuppc_id_rsa.pub >> ~/.ssh/authorized_keys rm ~/.ssh/backuppc_id_rsa.pub
Most of the global settings are easily configured via the web interface, at http://localhost/backuppc but you can also edit /etc/backuppc/config.pl directly.
Define each host you want to back up. It is easiest to automate email notifications if you use the user's email address as their username.
ClientCharset -> cp1252XferMethod -> rsyncdRsyncdUserName -> backuppcRsyncShareName -> cygRsyncdUserName -> backuppcRsyncdPasswd -> picksomethingrsyncd.conf on the clients.BackupFilesExcludeBackupFilesExclude is easier to modify in /etc/backuppc/config.pl directly. Pick the exclude list for the primary operating system of your clients. Unfortunately, you will have to hard code the exclude list for your other clients. For example, if Windows 7 or Vista is the primary OS and XP is the only other one, use the 7/Vista excludes below in config.pl and the XP excludes in each clients clientname.pl file. You can check out the wiki page I wrote most of to get ideas for excludes for Windows 2000 or Linux.
You may also want to exclude some of the Cygwin directories like /bin, /tmp and /packages (where I told Cygwin to put the package directory during setup). Just make sure to backup /etc so you don't lose your configuration!
I ran into a lot of problems on my Windows 7 and Vista machines where rsync would follow the junction points Microsoft added for backward compatibility (see this site for more info). This caused extra-long filenames rsync couldn't handle. To find all junction points on your 7/Vista machine open the Command Prompt and run from the root of the c drive:
dir /aL /s > c:\users\USERNAME\JunctionPoints.txt
The rules below will exclude these junction points. Rules with asterisks in them will match the junction points that are in every user profile by default without having to code excludes for each user manually. We can remove any temp data and program or system installation files as well. On 7/Vista, only original installation data is stored in Program Files. Any data programs write to their installation folder goes to ProgramData automatically instead. The Windows folder shouldn't hold anything interesting, either.
Exclude list for Windows 7/Vista clients with the rsyncd share being /cygdrive/c:
$Conf{BackupFilesExclude} = {
#Windows 7/Vista specific!
'*' => [
#7/Vista junction points
'/Documents and Settings',
'/ProgramData/Application Data',
'/ProgramData/Desktop',
'/ProgramData/Documents',
'/ProgramData/Favorites',
'/ProgramData/Start Menu',
'/ProgramData/Templates',
'/Users/All Users',
'/Users/Users/Default User',
'/Users/Users/All Users/Application Data',
'/Users/Users/All Users/Desktop',
'/Users/All Users/Documents',
'/Users/All Users/Favorites',
'/Users/All Users/Start Menu',
'/Users/All Users/Templates',
#Junction points common to every user profile
'/Users/*/Application Data',
'/Users/*/Cookies',
'/Users/*/Local Settings',
'/Users/*/My Documents',
'/Users/*/NetHood',
'/Users/*/PrintHood',
'/Users/*/Recent',
'/Users/*/SendTo',
'/Users/*/Start Menu',
'/Users/*/Templates',
'/Users/*/AppData/Local/Application Data',
'/Users/*/AppData/Local/History',
'/Users/*/AppData/Local/Temporary Internet Files',
'/Users/*/Documents/My Music',
'/Users/*/Documents/My Pictures',
'/Users/*/Documents/My Videos',
#Temporary and in-use user data
'/Users/*/AppData/Local/Microsoft/Windows/Temporary Internet Files',
'/Users/*/AppData/Local/Temp',
'/Users/*/NTUSER.DAT*',
'/Users/*/ntuser.dat*',
'/Users/*/AppData/Local/Microsoft/Windows/UsrClass.dat*',
'/Users/*/AppData/Local/Microsoft/Windows Defender/FileTracker',
'/Users/*/AppData/Local/Microsoft/Windows/Explorer/thumbcache_*.db',
'/Users/*/AppData/Local/Microsoft/Windows/WER',
'/Users/*/AppData/Local/Mozilla/Firefox/Profiles/*/Cache',
'/Users/*/AppData/Local/Mozilla/Firefox/Profiles/*/OfflineCache',
'/Users/*/AppData/Roaming/Microsoft/Windows/Cookies',
'/Users/*/AppData/Roaming/Microsoft/Windows/Recent',
'ProgramData/Microsoft/Search',
'ProgramData/Microsoft/Windows Defender',
'*.lock',
'Thumbs.db',
'IconCache.db',
'Cache*',
'cache*',
#Installation folders and system data
'/Program Files',
'/Windows',
'/$Recycle.Bin',
'/MSOCache',
'/System Volume Information',
'/Boot',
'/autoexec.bat',
'/bootmgr',
'/BOOTSECT.BAK',
'/config.sys',
'/hiberfil.sys',
'/pagefile.sys'
]
};
Exclude list for Windows XP clients with the rsyncd share being /cygdrive/c:
$Conf{BackupFilesExclude} = {
#XP specific!
'*' => [
#Temporary and in-use user data
'/Documents and Settings/*/Cookies',
'/Documents and Settings/*/Local Settings/Temporary Internet Files',
'/Documents and Settings/*/Local Settings/Temp',
'/Documents and Settings/*/NTUSER.DAT*',
'/Documents and Settings/*/ntuser.dat*',
'/Documents and Settings/*/Local Settings/Application Data/Microsoft/Windows/UsrClass.dat*',
'/Documents and Settings/*/Local Settings/Application Data/Mozilla/Firefox/Profiles/*/Cache',
'/Documents and Settings/*/Local Settings/Application Data/Mozilla/Firefox/Profiles/*/OfflineCache',
'/Documents and Settings/*/Recent',
'*.lock',
'Thumbs.db',
'IconCache.db',
'Cache*',
'cache*',
#Installation folders and system data
'/WINDOWS',
'/RECYCLER',
'/MSOCache',
'/System Volume Information',
'/AUTOEXEC.BAT',
'/BOOTSECT.BAK',
'/CONFIG.SYS',
'/hiberfil.sys',
'/pagefile.sys'
]
};
EMailAdminUserName -> youremail@host.comCgiUserConfigEditEnable -> noFullKeepCnt -> 4, 2, 3, 0, 0, 4FullAgeMax -> 1095IncrLevels -> 3, 2, 5, 4, 7, 6BlackoutPeriodsBlackoutPeriods:hourEnd -> 1BlackoutPeriods:weekDays -> 0, 1, 2, 3, 4, 5, 6RsyncdClientPort -> XXXXXClientNameAlias -> localhostPingCmd -> /etc/backuppc/mktnl clienthostname clientsshport localportIn order to setup the SSH tunnel beforehand, we can create script that BackupPC can run before the backup. I'm placing it in the PingCmd field so it is run when BackupPC pings the host before it starts. It also allows me to return a valid ping even though we set the alias above to localhost.
Here is the contents of /etc/backuppc/mktnl:
#!/bin/sh host=$1 sshPort=$2 locPort=$3 remPort=873 username=backuppc /bin/ping -c 1 -w 3 $host /usr/bin/ssh -f -L $locPort:localhost:$remPort $username@$host -p $sshPort sleep 10 1>/dev/null 2>/dev/null && echo "SSH tunnel started successfully."
The ping command returns its results to BackupPC and the SSH tunnel is started and prints into the log for you. The sleep 10 makes the tunnel wait for 10 seconds after all traffic finishes before closing.
PingMaxMsec -> 200BlackoutPeriodshourEnd and hourBegin adjusted for userYou can test backups for each client by running the following where hostname is the name of the host in the BackupPC global settings.
/usr/share/backuppc/bin/BackupPC_dump -f -v hostname
If you are backing up clients over the Internet, you may want to tell SSH to use compression and a faster cipher for the encryption. To do so, change the permissions on /etc/ssh/ssh_config with
chmod go+w /etc/ssh_configthen add the following lines to the file where
remotehostname* are the remote names or aliases you set up in your SSH config files:
Host remotehostname1 remotehostname2 remotehostname3 Compression yes Cipher blowfishblowfish is less secure than 3des, but might give you performance improvements. Remember to restore the previous permissions with
chmod go-w /etc/ssh_config
You need to set a password for each email address/username you defined previously so users can access their backup pages. Run the following for each host where clientemail is the email address/username you defined for that host.
htpasswd /etc/backuppc/htpasswd clientemail
The easiest way for a home user to do this is via Gmail. See Using Exim4 to send Messages through Gmail and this post for more info. First, set up POP in a new Gmail account. Set smtp.gmail.com as the smarthost by running
dpkg-reconfigure exim4-configand picking to use a smarthost on the second page (
mail sent by smarthost; no local mail). You can then edit /etc/exim4/update-exim4.conf.conf to make sure dc_smarthost='smtp.gmail.com'
Next, add these lines to /etc/exim4/passwd.client:
smtp.gmail.com : ACCOUNT_NAME@gmail.com : PASSWORD gmail-smtp.l.google.com : ACCOUNT_NAME@gmail.com: PASSWORD
To test email delivery, try
/usr/share/BackupPC/bin/BackupPC_sendEmail -u emailaddress
If it isn't working, you can watch the log file with
tail /var/log/exim4/mainlogand see the current queue of messages with
mailqOne common problem is finding an
Authentication Required message in your log. If the host is something other than gmail-smtp.l.google.com, replace it in your /etc/exim4/passwd.client with whatever it says. Once you've fixed your problem, you can send all frozen messages with exim -qff
You can add user: email combos to /etc/aliases for the backuppc and/or root users so you get admin mail for your box, too. Afterwards, recompute the Sendmail aliases with
newaliases
To schedule checks for BackupPC status, use cron (see this site for more info). Run
crontab -eand add the following line to get a message every half hour if the BackupPC server goes down:
*/30 * * * * /usr/share/backuppc/bin/BackupPC_sendEmail -c
If you want to send your email using the message submit port (587) instead of smtp (25), see this post. This will allow you to get around the port 25 blocks some ISPs use to reduce spam on their networks.
You probably want to use a separate hard drive or group of drives for your BackupPC pool for easier maintenance, expansion, and reliability. We are going to use Logical Volume Manager (LVM) to create a resizable pool we can easily add more drives to.
LVM allows us to create expandable logical volumes across many drives. See A Beginner's Guide To LVM for more detailed setup instructions. Also check out Managing Disk Space with LVM and A simple introduction to working with LVM.
Set up your drive using fdisk. The command below is for a drive at /dev/sdb
fdisk -l fdisk /dev/sdbThe choices you want to make are:
n, p, 1, ENTER, ENTER, t, 1, L, 8e, w. Then run
pvcreate /dev/sdb1 pvdisplay vgcreate bpcfs /dev/sdb1 vgdisplay vgscan lvcreate --name bpclv --extents 100%FREE bpcvg lvdisplay lvscan mkfs.ext3 /deb/bpcvg/bpclvIf you get a 'no entry for device-mapper found' error, run
modprobe dm-mod
See here for more info. If your new partition is /dev/bpcvg/bpclv, execute the following commands as root
mount /dev/bpcvg/bpclv /mnt cp -dpR /var/lib/backuppc/. /mnt umount /dev/md0 mv /var/lib/backuppc /var/lib/backuppc.orig mkdir /var/lib/backuppc mount /dev/bpcvg/bpclv /var/lib/backuppc df -hThis makes a backup of your data then copies it to the new drive. Once you're sure that worked, you need to make it mount on startup. Edit
/etc/fstab and add the following line:
/dev/bpcvg/bpclv /var/lib/backuppc ext3 rw,noatime 0 0Then restart and run
df -hto make sure it worked.