How is my account backed up?

Different backups are provided at CSE depending on whether the backups are accessed via the "New" or "Old" CSE Linux systems (also referred to as the "New World" (NW) or the "Old World" (OW) systems). Most lab machines used by undergraduate students have been operating under the NW system since around 2020, while many older legacy machines and some servers are still run under the OW system. The OW system is expected to be phased out and unavailable by the mid 2020s.

New World Backup System

There is only one system of backups taken in the New World:

• Every night, each CSE user's home directory is copied into their archive directory stored at:

  /import/nw-syd-backup1/1/backups/users/${LASTTWO}/${USER}/

  where:

  • ${USER}: The user's CSE login name.
  • ${LASTTWO}: The last two characters of ${USER}.
• Each archive directory contains up to thirty daily archives of the user's home directory, one taken each day, and are organised such that:
  • The most recent daily archive is stored under daily.0/${USER}/${USERHOME}/.
  • Successively older daily archives are stored under daily.N/${USER}/${USERHOME}/
  • The oldest daily archive is stored under daily.29/${USER}/${USERHOME}/.
  where:
  ${USERHOME}: The full pathname of the user's home directory (eg: /export/adams/1/${USER})

• The ownership and permissions of the files and directories in the user's archive directory duplicate those of the user's original home directory. If the user has set up ownership or permissions within their home directory such that other users are permitted to access their files and/or directories, then these other users will be able to access the same files and/or directories within the user's daily archives.
• None of the archived files/directories can be modified or deleted, regardless of their ownerships or permissions.

Restoring backups from the New World System

• User: sam
• CSE home directory: /import/adams/1/sam (as displayed by acc sam)
• Wanted to restore: /home/sam/COMPX/assign1/first.c
• From a daily archive taken 5 days ago

% cp /import/nw-syd-backup1/1/backups/users/am/sam/daily.5/sam/export/adams/1/sam/COMPX/assign1/first.c first.c.copy

Note:

• ${USER} (in this case sam) will always appear three times in the full archive pathname.
• The user should not have to explicitly type many of the subdirectories in this pathname in full, as the shell will be able to perform tab filename completion on many (if not most) of them. In particular, the shell should automatically complete everything in the archive pathname from daily.n to the start of the contents of the user's actual home directory (in this case sam/export/adams/1/sam/).
• Users can (and should) cd into their desired archive directory, and inspect the files they want to restore, making sure that they are the ones they want, and that they have the desired modification dates etc, before copying them over into their home directories.
• All files and directories that existed anywhere in your home directory in the last 30 days (including those that were themselves more than 30 days old) will be present somewhere in the your backup archives. However, you will not be able to find or restore any file (or directory) from the new world backup system if it was deleted from your home directory more than 30 days ago.
• NW archives can only be accessed from NW machines.

Old World Backup System

There are six levels of backups provided under the Old World system:

Incremental

Taken nightly

Files modified in the last 24 hours (see Incremental limits)

Kept online for at least two months

Incremental summary

Taken weekly

Latest versions of files from nightly backups

Kept online for at least two years

Full backup

Taken monthly

All files in your home directory

Kept on tape for many years

Server snapshot

Taken every 7-10 day cycle

All user files on each file system on each file server

Kept on tape for four cycles

Session dump

Taken at the end of each semester

All files in your home directory and other directories

Kept on tape indefinitely

Archive dump

Taken after your account expires

All files in your home directory

Kept on tape indefinitely

You can restore files from incremental, incremental summary and full backups by yourself - see Restoring files from backup.

For files that may only be in the snapshots or dumps, contact System Support.

Note that server snapshots are chiefly for disaster recovery, and aren't used for normal file recovery except in emergencies.

Incremental limits

Unfortunately, we have limited space available for incremental backups.

Because of this, there are limits on which files in your account will be backed up every night:

• Only files modified in the last 24 hours will be backed up.
• Temporary and easily-replaced files (such as core, *.o and files in the trash) are excluded.
• Large files (8MB by default) are excluded.
• No more than 100MB per user will be backed up per night.

Because of the hard 100MB limit, it's quite possible that less-important recently modified files that are encountered and copied before the limit is reached, will displace from the backup more-important ones that would have been encountered after the limit is reached.

However, you can control which files are excluded and which files have priority, to ensure your most important files are covered.