How is my account backed up?
There are three independent backup systems running at CSE.Nightly incrementals
Incremental backups are run each night. Files modified in the last day are copied to a large, specially indexed, filesystem and are kept for approximately 4 weeks. After this time the most recent updates are collected together to make a weekly incremental, compressed, and stored for another 6 months (and possibly longer depending on available space).These backups are normally used for restoring individual files which may have been lost or damaged. The incremental backups cannot normally restore a complete directory.
Because of the space limitations on the backup filesystem, a filter program called buff is used to select the files to be backed up. A full description is in the manual page, however the main effects of buff are to:
- omit files which seem to be easily reproduceable (eg, .o, .dvi),
- omit singularly large files,
- limit any set of backups from a home directory to a designated maximum. This is called the treemax.
The treemax limit
The treemax is currently set to 60MB per account per night.The treemax limit often confuses people. What happens with the incremental backups is they take a list of every file you modified that day, then start backing them up one by one, counting the bytes, until they reach the treemax. The total file size, not the size of the changes, is counted. This means that if you have one very big file, and you make a tiny change to it, it will get included in your nightly backup and take up most of the limit, squeezing out other files that you may have extensively changed. This tends to happen to people who never clean out their old mails. New mail gets added every day so the file gets included in the incremental backups, but most of the file is old and has already been backed up many times - and this old stuff is preventing newer files from being backed up.
If you are having problems with incremental backups you can find solutions at this page: Account not backed up.
Full dumps
are a complete dump of the contents of each home directory. As long as the home directory can fit on the backup media, the home directory will be backed up on Full Dumps. Note that the current media used for full dumps (LTO tapes) can hold up to 3TB (compressed).There are three classes of Full Dumps taken throughout the year:
- Regular full dumps of all home directories, taken every 28 days, and kept for seven years;
- Session full dumps of all home directories, taken a few weeks after sessions one and two every year, and kept indefinitely;
- Archive full dumps of expired home directories, taken a month or so after they have expired, and kept indefinitely.
Full backups are normally used when:
- There are no copies of the missing file on incremental
dumps. This will happen when the file to be restored:
- Had not been modified within the last 6 months or so;
- Had not been backed up onto incrementals because of buff limitations.
Snapshots
are taken once every 7 days of the file systems on most of CSE's major servers - particularly of the major file servers hosting user directories. Snapshots contain a complete dump of the entire filesystem, and are stored on 4 duplicated sets of 5 LTO3 tapes rotated over 28 days, with one of the sets of 5 being stored off-site (also on a rotational basis).These backups are normally used when there has been a major disk catastrophe on a server.