| BackupPC |
This documentation describes BackupPC version 3.3.0, released on 14 Apr 2013.
BackupPC is a high-performance, enterprise-grade system for backing up Unix, Linux, WinXX, and MacOSX PCs, desktops and laptops to a server's disk. BackupPC is highly configurable and easy to install and maintain.
Given the ever decreasing cost of disks and raid systems, it is now practical and cost effective to backup a large number of machines onto a server's local disk or network storage. For some sites this might be the complete backup solution. For other sites additional permanent archives could be created by periodically backing up the server to tape.
Features include:
A clever pooling scheme minimizes disk storage and disk I/O. Identical files across multiple backups of the same or different PC are stored only once (using hard links), resulting in substantial savings in disk storage and disk writes.
Optional compression provides additional reductions in storage (around 40%). The CPU impact of compression is low since only new files (those not already in the pool) need to be compressed.
A powerful http/cgi user interface allows administrators to view the current status, edit configuration, add/delete hosts, view log files, and allows users to initiate and cancel backups and browse and restore files from backups.
The http/cgi user interface has internationalization (i18n) support, currently providing English, French, German, Spanish, Italian, Dutch, Polish, Portuguese-Brazilian and Chinese
No client-side software is needed. On WinXX the standard smb protocol is used to extract backup data. On linux, unix or MacOSX clients, rsync, tar (over ssh/rsh/nfs) or ftp is used to extract backup data. Alternatively, rsync can also be used on WinXX (using cygwin), and Samba could be installed on the linux or unix client to provide smb shares).
Flexible restore options. Single files can be downloaded from any backup directly from the CGI interface. Zip or Tar archives for selected files or directories from any backup can also be downloaded from the CGI interface. Finally, direct restore to the client machine (using smb or tar) for selected files or directories is also supported from the CGI interface.
BackupPC supports mobile environments where laptops are only intermittently connected to the network and have dynamic IP addresses (DHCP). Configuration settings allow machines connected via slower WAN connections (eg: dial up, DSL, cable) to not be backed up, even if they use the same fixed or dynamic IP address as when they are connected directly to the LAN.
Flexible configuration parameters allow multiple backups to be performed in parallel, specification of which shares to backup, which directories to backup or not backup, various schedules for full and incremental backups, schedules for email reminders to users and so on. Configuration parameters can be set system-wide or also on a per-PC basis.
Users are sent periodic email reminders if their PC has not recently been backed up. Email content, timing and policies are configurable.
BackupPC is Open Source software hosted by SourceForge.
A full backup is a complete backup of a share. BackupPC can be configured to do a full backup at a regular interval (typically weekly). BackupPC can be configured to keep a certain number of full backups. Exponential expiry is also supported, allowing full backups with various vintages to be kept (for example, a settable number of most recent weekly fulls, plus a settable number of older fulls that are 2, 4, 8, or 16 weeks apart).
An incremental backup is a backup of files that have changed since the last successful full or incremental backup. Starting in BackupPC 3.0 multi-level incrementals are supported. A full backup has level 0. A new incremental of level N will backup all files that have changed since the most recent backup of a lower level. $Conf{IncrLevels} is used to specify the level of each successive incremental. The default value is all level 1, which makes the behavior the same as earlier versions of BackupPC: each incremental will back up all the files that changed since the last full (level 0).
For SMB and tar, BackupPC uses the modification time (mtime) to determine which files have changed since the last lower-level backup. That means SMB and tar incrementals are not able to detect deleted files, renamed files or new files whose modification time is prior to the last lower-level backup.
Rsync is more clever: any files whose attributes have changed (ie: uid, gid, mtime, modes, size) since the last full are backed up. Deleted, new files and renamed files are detected by Rsync incrementals.
BackupPC can also be configured to keep a certain number of incremental backups, and to keep a smaller number of very old incremental backups. If multi-level incrementals are specified then it is likely that more incrementals will need to be kept since lower-level incrementals (and the full backup) are needed to reconstruct a higher-level incremental.
BackupPC "fills-in" incremental backups when browsing or restoring, based on the levels of each backup, giving every backup a "full" appearance. This makes browsing and restoring backups much easier: you can restore from any one backup independent of whether it was an incremental or full.
When a full backup fails or is canceled, and some files have already been backed up, BackupPC keeps a partial backup containing just the files that were backed up successfully. The partial backup is removed when the next successful backup completes, or if another full backup fails resulting in a newer partial backup. A failed full backup that has not backed up any files, or any failed incremental backup, is removed; no partial backup is saved in these cases.
The partial backup may be browsed or used to restore files just like a successful full or incremental backup.
With the rsync transfer method the partial backup is used to resume the next full backup, avoiding the need to retransfer the file data already in the partial backup.
BackupPC pools identical files using hardlinks. By "identical files" we mean files with identical contents, not necessary the same permissions, ownership or modification time. Two files might have different permissions, ownership, or modification time but will still be pooled whenever the contents are identical. This is possible since BackupPC stores the file meta-data (permissions, ownership, and modification time) separately from the file contents.
Based on your site's requirements you need to decide what your backup policy is. BackupPC is not designed to provide exact re-imaging of failed disks. See Limitations for more information. However, the addition of tar transport for linux/unix clients, plus full support for special file types and unix attributes in v1.4.0 likely means an exact image of a linux/unix file system can be made.
BackupPC saves backups onto disk. Because of pooling you can relatively economically keep several weeks of old backups.
At some sites the disk-based backup will be adequate, without a secondary tape backup. This system is robust to any single failure: if a client disk fails or loses files, the BackupPC server can be used to restore files. If the server disk fails, BackupPC can be restarted on a fresh file system, and create new backups from the clients. The chance of the server disk failing can be made very small by spending more money on increasingly better RAID systems. However, there is still the risk of catastrophic events like fires or earthquakes that can destroy both the BackupPC server and the clients it is backing up if they are physically nearby.
Some sites might choose to do periodic backups to tape or cd/dvd. This backup can be done perhaps weekly using the archive function of BackupPC.
Other users have reported success with removable disks to rotate the BackupPC data drives, or using rsync to mirror the BackupPC data pool offsite.
The BackupPC Open Source project is hosted on SourceForge. The home page can be found at:
http://backuppc.sourceforge.netThis page has links to the current documentation, the SourceForge project page and general information.
The SourceForge project page is at:
http://sourceforge.net/projects/backuppcThis page has links to the current releases of BackupPC.
BackupPC has a Wiki at http://backuppc.wiki.sourceforge.net. Everyone is encouraged to contribute to the Wiki. Anyone with a SourceForge account can edit the Wiki.
The old FAQ is at http://backuppc.sourceforge.net/faq, but is deprecated in favor of the Wiki.
Three BackupPC mailing lists exist for announcements (backuppc-announce), developers (backuppc-devel), and a general user list for support, asking questions or any other topic relevant to BackupPC (backuppc-users).
The lists are archived on SourceForge and Gmane. The SourceForge lists are not always up to date and the searching is limited, so Gmane is a good alternative. See:
http://news.gmane.org/index.php?prefix=gmane.comp.sysutils.backup.backuppc
http://sourceforge.net/mailarchive/forum.php?forum=backuppc-usersYou can subscribe to these lists by visiting:
http://lists.sourceforge.net/lists/listinfo/backuppc-announce
http://lists.sourceforge.net/lists/listinfo/backuppc-users
http://lists.sourceforge.net/lists/listinfo/backuppc-develThe backuppc-announce list is moderated and is used only for important announcements (eg: new versions). It is low traffic. You only need to subscribe to one of backuppc-announce and backuppc-users: backuppc-users also receives any messages on backuppc-announce.
The backuppc-devel list is only for developers who are working on BackupPC. Do not post questions or support requests there. But detailed technical discussions should happen on this list.
To post a message to the backuppc-users list, send an email to
backuppc-users@lists.sourceforge.netDo not send subscription requests to this address!
If you want to mirror linux or unix files or directories to a remote server you should use rsync, http://rsync.samba.org. BackupPC uses rsync as a transport mechanism; if you are already an rsync user you can think of BackupPC as adding efficient storage (compression and pooling) and a convenient user interface to rsync.
Two popular open source packages that do tape backup are Amanda (http://www.amanda.org) and Bacula (http://www.bacula.org). These packages can be used as complete solutions, or also as back ends to BackupPC to backup the BackupPC server data to tape.
Various programs and scripts use rsync to provide hardlinked backups. See, for example, Mike Rubel's site (http://www.mikerubel.org/computers/rsync_snapshots), JW Schultz's dirvish (http://www.dirvish.org/), Ben Escoto's rdiff-backup (http://www.nongnu.org/rdiff-backup), and John Bowman's rlbackup (http://www.math.ualberta.ca/imaging/rlbackup).
Unison is a utility that can do two-way, interactive, synchronization. See http://freshmeat.net/projects/unison. An external wrapper around rsync that maintains transfer data to enable two-way synchronization is drsync; see http://freshmeat.net/projects/drsync.
BackupPC provides many additional features, such as compressed storage, hardlinking any matching files (rather than just files with the same name), and storing special files without root privileges. But these other programs provide simple, effective and fast solutions and are definitely worthy of consideration.
The new features planned for future releases of BackupPC are on the Wiki at http://backuppc.wiki.sourceforge.net.
Comments and suggestions are welcome.
BackupPC is free. I work on BackupPC because I enjoy doing it and I like to contribute to the open source community.
BackupPC already has more than enough features for my own needs. The main compensation for continuing to work on BackupPC is knowing that more and more people find it useful. So feedback is certainly appreciated, both positive and negative.
Beyond being a satisfied user and telling other people about it, everyone is encouraged to add links to http://backuppc.sourceforge.net (I'll see them via Google) or otherwise publicize BackupPC. Unlike the commercial products in this space, I have a zero budget (in both time and money) for marketing, PR and advertising, so it's up to all of you! Feel free to vote for BackupPC at http://freshmeat.net/projects/backuppc.
Also, everyone is encouraged to contribute patches, bug reports, feature and design suggestions, new code, Wiki additions (you can do those directly) and documentation corrections or improvements. Answering questions on the mailing list is a big help too.
BackupPC requires:
A linux, solaris, or unix based server with a substantial amount of free disk space (see the next section for what that means). The CPU and disk performance on this server will determine how many simultaneous backups you can run. You should be able to run 4-8 simultaneous backups on a moderately configured server.
Several users have reported significantly better performance using reiserfs compared to ext3 for the BackupPC data file system. It is also recommended you consider either an LVM or RAID setup (either in HW or SW; eg: 3Ware RAID10 or RAID5) so that you can expand the file system as necessary.
When BackupPC starts with an empty pool, all the backup data will be written to the pool on disk. After more backups are done, a higher percentage of incoming files will already be in the pool. BackupPC is able to avoid writing to disk new files that are already in the pool. So over time disk writes will reduce significantly (by perhaps a factor of 20 or more), since eventually 95% or more of incoming backup files are typically in the pool. Disk reads from the pool are still needed to do file compares to verify files are an exact match. So, with a mature pool, if a relatively fast client generates data at say 1MB/sec, and you run 4 simultaneous backups, there will be an average server disk load of about 4MB/sec reads and 0.2MB/sec writes (assuming 95% of the incoming files are in the pool). These rates will be perhaps 40% lower if compression is on.
Perl version 5.8.0 or later. If you don't have perl, please see