Home and Links
 Your PC and Security
 Server NAS
 DVD making
 Raspberry Pi
 PIC projects
 Other projects
 Next >>

Backup and Synchronisation

Backup and Synchronisation

What's the difference between Backup and Synchronisation ?

A1. Backup is a process that typically packs copies of all your files into a single large compressed 'archive' (in order to minimise the space used).

The main advantage is that a new backup does not overwrite the old (unless you want it to). This means you can keep all the historical versions of your files.

The main disadvantage is that all your individual files are packed up together within the archive, which makes it very difficult (if not impossible) to 'recover' one or two files you accidentally deleted, rather than 'restoring' the entire PC.

This is what MS Backup does - which makes it virtually useless for the 'average' user who accidentally deletes a file or just wants to 'go back' to some previous version of a document.

A2. Synchronisation is a process that maintains copies all your files in a duplicate folder (eg. on your Server). If you choose 'contribute' (rather than 'full' synchronisation) files are only added - so anything you delete from your own computer won't also be deleted from the Server.

The main advantage is that the copies on the Server can be browsed as 'normal' (and the "copy & pasted" back to your PC).

The main disadvantage is that it does not maintain a 'compatible set' of files that need to 'work together' - so is of no use in 'saving' your system settings

Many synchronisation systems can be set to work 'in real time' (typically by defining a 'watched folder') so when you save a modified file, a copy is immediately taken. Some even have the option of maintaining 'versions' (so that the old copy is not overwritten every time a new copy is made).Note that some external "Back-Up" (joke) drives come with a cut-down version of some commercial software that will 'almost do the job' (the idea being to tempt you into paying $$$ for the 'Full' or 'Pro' (i.e. usable) version). Don't be tempted - there are many Open Source solutions (eg FileSync) that will maintain synchronisation for free (and without 'phoning home', power-on overhead or never-ending 'nags' trying to get you to pay)If you have some very specific requirements (such as copying photos to your Server / NAS 'archive' and then making them 'read only'), it may be quicker to write your own 'synchronisation' script (see my previous Using scripts page) than to spend hours wading through hundreds of pages of Google 'advertising' trying to find an application the will do exactly what you want

With many Corporations issuing their employees with laptops (allowing them to work 'away from the office', or 'at home'), Microsoft introduced their own 'synchronisation' software (aka 'Offline Files') with Windows XP Pro.

Intended to allow modified files to be 'automatically' copied to the Company Server on return to the office, the UI is so hard to understand that, more often than not, the user would end up overwriting their modified files by copying older versions from** the Company Server instead !**the idea being to 'update' your Laptop with the latest versions of 'company files' (such as price lists, customer 'lead' details) before 'un-docking' and leaving the office. However, unless you are very careful, you end up overwriting all that days work as well ..

What about 'cloning' / 'drive imaging' (aka 'Ghosting') ?

Making a 'clone' or 'drive image' means taking an exact copy of the entire disk drive (or entire partition(s) ).

The disadvantage is that a 'clone' or 'image' can usually only be 'restored' to a drive of the same (or larger) size as the 'source' (even if only a few Mb's of a multi-Gb drive are actually used - but see EASUS ToDo below).

IT Departments of large Corporations often use a pre-saved 'clone' image to setup their employees computers to some pre-defined 'standard', however this is not a practical approach for the home user who has simply 'lost' a file :-)

It is, however, an excellent way to set up two identical hard drives prior to creating a RAID (mirror), or doing a complete 'roll back' after making the mistake of installing some impossible-to-get-rid-of commercial 'free trial' garbage (see my disk imaging software page) Note that some so-called 'imaging' software actually only copies the visible-to-Windows 'active partition' (C:), meaning that any hidden 'System Restore' / 'Factory Restore' partition (common on most systems built during the last 15 years) gets 'left behind' ..

EASUS ToDo backup v6.5 (Free ed)

This is about the best free 'clone' software available. It has the overwhelming advantage of ONLY cloning 'used' parts of the 'source' drive and allows you to adjust the destination partition sizes during the cloning process (so you can clone a large drive to a smaller one). It also 'spots' hidden ('system restore') partitions and allows you to choose which partitions to clone.

Make sure to get v6.5 (search for it using Google). The vendor has removed the ability to change partition sizes from the current version (8.5) Free Ed.

Since it only copies 'live' data it can cope with failing drives (so long as none of the 'live' sectors are 'bad' == so run 'chkdsk /f' from a CMD prompt before trying to clone a failing drive) V6.5 runs just fine on Windows 7 64bit and even offers to 'optimise for SSD' (if it spots one as the destination)

The disadvantage is that it's exceedingly slow when cloning a 'live' system disk (cloning a 500Gb system disk that's 'offline' eg as D: to E:, takes 10 hours or so - copying the same drive 'live' (as C: to E:) jumps this to 50 hrs+).

If you MUST copy a 'live' system, ONLY clone the actual system partition (C:) and not the whole drive. Further, move as much data off the system partition before doing the 'clone' (e.g. move data off to a network 'share' (use Free File Sync (see below) = it's about twice as fast as Windows drag and drop, although even Windows 'drag and drop' to a USB drive / stick will be 5x faster than letting EASUS 'clone' the C: disk). NB. just a reminder == DO NOT attempt to clone a C: partition with a 'live' 'swap file' :-)

Finally, whilst EASUS 6.5 doesn't try to take over you browser or install 'ad-ware', the publisher does try to 'harvest' your email address - you have to enter a valid email address to get the 'download link' from their site - however since v6.5 is no longer available from EASUS themselves, this need not concern you.

Acronis TrueImage "solution"

AVOID this like the plague it is. Like all 'solutions', this software tries to take over your entire PC, installing multiple unwanted and unasked for 'services' and 'tasks' that are hard to get rid of.

This garbage is often included 'for free' with SSD drives and is a really good example of the sort of useless crap that commercial software publishers pay margin stressed hardware manufacturers to foist onto their unsuspecting customers. If you find a 'complementary licence code' in the box, just throw it away

What's an 'incremental back-up' ?

In the old days file copying took a long time, hard drives were expensive and capacities limited, so backups were recorded to removable magnetic tapes. Further, network speeds were so slow (10 mbs) that backups had to be done 'overnight'.

To support multiple computers each night, there was only enough time to copy 'changes' (it was faster for the employees computer to send just the changes than to send the whole file). These where called 'Incremental' backups. A 'Full' backup of all the employees files would typically be done at the weekend, once a month.

This meant that to 'restore' an entire file you had to first find the tape with the last complete copy (i.e the last 'Full backup') and then 'add in' all the changes, one at a time, in sequence, from multiple 'Incremental' backup tapes.

Needless to say, file recovery was a process that no-one in IT ever wanted to do, especially as the chances of one or more tapes failing to 'read' before you had recovered all the changes was quite high.

Thankfully hard drive storage capacities advanced much faster than tape capacity, resulting in such low '$ per Gb' costs that tape soon became totally uneconomic as a back-up solution. Instead hard disk based RAID systems containing massive ultra-fast ultra reliable 'always on-line' archives were built that allowed 'full backups' of all modified files every night. As network speeds improved, some companies moved to 'real time' backups during the day (whilst others moved their users 'Desktop' folder to the Company Server, however that's another story) These days only some very specialist systems use 'incremental' back-up (and even that is likely done to hard disk, not tape)

What about 'versioning' software ?

This is software that automatically makes copies of modified files ('versions') each time you 'close' your edit application.

Commercial versions (such as 'Beyond Compare') are aimed at Software Development Teams, and intended to allow multiple modifications to the same file to be compared and the changes 'merged' into a new 'master' version

Synchronisation software (such as the commercial Memeo AutoSync) often includes basic 'versioning', however you may have to 'dig down' the list of feature to discover how it works :-)

What do you recommend as the 'best' backup approach ?

You should use MSBackup, but ONLY for your 'System'. All your data files (docs, spreadsheets, music, photo's video's etc. etc.), which, after your existing computer fails, you may wish to 'migrate' onto a brand new computer with a new Operating System, should be COPIED using file 'synchronisation' software.

You should avoid any software that claims to 'compress' your data = you want all your files copied 'as they are' into standard Microsoft NTFS folders where they can be 'browsed' and 'drag&dropped / 'copy&pasted' by the Windows Operating System (rather than having to be 'extracted' or 'restored' by some proprietary software which you won't have on your new computer).

Synchronisation software will look for 'differences' between your 'source' folder contents and an existing 'destination' (backup) folder. This means it only needs to copy files that are new or have been modified.

More clever software will monitor ('watch') your source folder and automatically make copies every time you make changes. Alternatively, you can set-up a 'schedule' that lets it look for changes eg. overnight

Are there any disadvantages to scheduling backups ?

Yes - if your computer goes to sleep overnight and fails to 'wake up' and perform a scheduled backup, it will start running the backup when it's next 'woken up' by you.

How do you separate System backup from 'data file copy' ?

Use MS Backup for the system, and choose some suitable (Open Source) 'synchronisation' software to maintain up to date copies of your documents.

What file synchronisation software do you recommend ?

A1. Microsoft 'SyncToy' (free)

This is a basic synchronisation application, a free download from Microsoft's web site. It's main advantage is that it can handle multiple 'watched folders' and will copy files to a networked Server. It should be set to run as a Task (via Scheduled Tasks) to ensure synchronisation is performed at regular intervals during the day (or once a day, eg. at midnight).

Note = when setting up a 'folder pair', make sure to select CONTRIBUTE** - otherwise when you (accidentally) delete files from your local (source) folders, SyncToy will delete them from the 'backup' destination folder.**MS now includes a little 'explanation' of what each setting will do. This is a big improvement over the previous 'Off-line Folders' UI which usually ended up with you overwriting newer files with older versions ...

A2. DirSync Pro (Open Source)

This is very similar to MS SyncToy, however it can be set to automatically include sub-folders, which means you can 'back-up' your entire Desktop and 'My Documents' etc. simply by choosing C:\Documents and Settings\{your user name} as the "top level" folder.

It also generates detailed log files and incorporates it's own Task Scheduler (so you can disable Microsoft's own 'hacker hole' Task Scheduler) which can be set to run every 15 mins during the day (so, in effect, you can 'watch' the folders).

DirSync Pro should be added to the 'all users' Startup folder, where it will run automatically when any user logs-in (it does not have to launch at power-on, because, until some-one logs-in and changes a file, there will be nothing for it to do :-) )

A3. Free File Sync 6.7 (Open Source - Recommended)

This application supports both 'updating' and 'mirror' of the contents of any folder (or drive) to any other chosen folder (or drive). It is at least 2x faster than using Windows Explorer 'drag and drop' and is especially fast at copying to and from network 'shares'. It uses VSS to copy 'locked' (system) files.

The 'Update' (rather than Mirror) mode allows new files on the source added to the destination and older dated files to be replaced with newer ones (files and folders of different name on the destination are ignored, which can be a problem, see below). You can, of course, view what it intends to do (create / replace / delete) and 'deselect' on a file by file basis before letting it go ahead with the rest

FreeFileSync is how I keep my Laptop and Desktop work 'in step' whilst automatically creating a backup (via my Server) every time I do an Update (or Mirror). The only drawback is when you rename a file/folder on your 'source' = the rename is not 'spotted' as such (it's seen as a new file/folder) and an 'Update' (rather than a Mirror) will result in both old and new files/folders existing on the destination

The current version (7.2, July 2015) also runs on Android tablets. It also offers some sort of auto-running script option that supports 'watched folder' (auto-sync) functionality

A4. Dsynchronize (Freeware)

Similar to the others, this can be set to 'ask before acting' and also supports 'Real time' synchronisation.

Click 'Next >>' in the Navigation Bar (left) for How to recover from a Hardware failure.

The pages in this topic are :-

Next subject :- Hardware failure - (recovery after)