Custom Search
Web
 
 
 

RsyncBackup for Os/2



server

Warning: not compatible with RsyncBack-004

Abstract

Rsync is a fast and extraordinarily versatile file copying tool. It can copy locally, to/from another host over any remote shell, or to/from a remote rsync daemon. It offers a large number of options thatcontrol every aspect of its behavior and permit very flexible specification of the set of files to be copied. It is famous for its delta-transfer algorithm, which reduces the amount of data sent over the network by sending only the differences between the source files and the existing files in the destination. Rsync is widely used for backups and mirroring and as an improved copy command for everyday use.

Rsync finds files that need to be transferred using a "quick check" algorithm (by default) that looks for files that have changed in size or in last-modified time. Any changes in the other preserved attributes(as requested by options) are made on the destination file directly when the quick check indicates that the file's data does not need to be updated.


Purpose

I put together this package (RsyncBackup)  that will install a working setup for Rsync. You can install it on only one machine or install the same package on two machine for a (client / server) setup.  However, the main focus will be on setting up a working Rsync server with clients pushing data to that server. The RsyncBackup quick guide will allow you to complete this task in about 10 minutes (Windows included) !

I broke things down to three parts:
  1.  Local copy,
  2. Network copy, and
  3. Rsync server
The 'Backup Boot' will backup your boot drive.


rsyncfolder

Each WPS object or script is a template, an example on how each operation works.


I cover three methods of backing up data, the Local & Network folders have three options each:
  • Copy - a backup that does not examine the archive bit and does not alter the state of the archive bit after the backup, as the name implies it's just a plain copy of your data.
  • Full (sync) - same as 'copy', but adds the --delete option. The --delete option makes a complete backup and syncs the source and destination directories. If you delete something in the source directory, it gets deleted in the destination directory (so be careful) on the next run.
  • Incremental - a type of backup where only the files that have been changed, are backed up. Once the file has been successfully backed up, the archive bit is reset to indicate that the file has been backed up. Usage - my source directory for my web site changes daily, and I can easily  find changes by date and day of the week.

RsyncBackup will use this directory structure contained within RsyncBackup, because there are just to many variable with the scripts and WPS objects. Rsync's specialty is efficiently synchronizing file trees across a network, but it works fine on a single machine too.

Directory Structure

\RsyncBackup\Backups
                                     \BootDrive
                                     \Copy
                                     \Full
                                     \Incremental
                                     \Partitions

Trailing Slashes Do Matter...Sometimes

This isn't really an article about rsync, but I would like to take a momentary detour to clarify one potentially confusing detail about its use. You may be accustomed to commands that don't care about trailing slashes. For example, if a and b are two directories, then cp -a a b is equivalent to cp -a a/b/. However, rsync does care about the trailing slash, but only on the source argument. For example, let a and b be two directories, with the file foo initially inside directory a. Then this command:

rsync -a a b

produces b/a/foo, whereas this command:

rsync -a a/ b

produces b/foo. The presence or absence of a trailing slash on the destination argument (b, in this case) has no effect.

Local Basics

Suppose you have a directory called source, and you want to back it up into the directory destination. To accomplish that,you'd use:

rsync --archive --verbose \source \destination
This is command is the same as above:
rsync -av \source \destination

This would recursively transfer all files from the directory \source on the local machine into the \destination directory on the local machine. The files are transferred in "archive" mode, which ensures that symbolic links, devices, attributes, permissions, ownerships, etc. are preserved in the transfer.

Network Basics

Access via rsync daemon:
Push: rsync [OPTION...] SRC... [USER@]HOST::DEST
rsync [OPTION...] SRC... rsync://[USER@]HOST[:PORT]/DEST


Suppose you have a directory called source, and you want to back it up into the directory destination on your Rsync Server. To accomplish that,you'd use:

rsync [OPTIONS...] SRC... HOST::MODULE\DEST
rsync -azX C:\Home 192.168.1.200::RsyncBackup\Copy

This would recursively transfer all files from the directory \source on the local machine into the \destination directory on the remote Rsync server (192.168.1.200).


Using 'rsyncbackup' to make a local backup


localfolder



If you are going to use 'RsyncBackup' for local operations only, then you do not need these folders
  • 'Network copy' folder, or
  • 'Rsync server' folder

Setting up RsyncBackup locally

IMPORTANT - all scripts and program objects have the (--dry-run ) option
inserted, it performs a trial run with no changes made to your system. So,
this package will not change anything on your system until (--dry-run ) is
removed.

1.) Download the 'RsyncBackup' package here: RsyncBackup.zip

2.) For my setup and this example, I'll be using the root directory of my E: drive for installation.

3.) Unzip and move the directory x:\RsyncBackup to the root directory of a drive with plenty of space, the directory's name MUST NOT BE CHANGED. The restrictions of the directory's name will be addressed in the next version.

4.) Run the 'install.cmd' to install 'RsyncBackup' and create the WPS objects on the desktop.


RsyncBackup - Local Copy

4a) Open up the 'RsyncBackup => Local copy => Copy ' :

WPS object, 'Local Copy - C:\Home' is a template, it makes an exact copy of 'c:\Home' directory to e:\RsyncBackup\Backups\Copy


Usage: make new 'copy' object

  • Right click on the 'Local Copy - C:\Home' object and select on menu 'copy'
  • In the copy dialog, rename 'Local Copy - C:\Home' to your new copy operation, for example 'Local copy - Torrents'.
  • Right click on the new object and select properties, under parameters:
    •  change:  --dry-run C:\home G:\RsyncBackup\Backups\Copy
    •          to:                  C:\Torrents G:\RsyncBackup\Backups\Copy
  • All done, modify as needed and the \source \destination can be any directory or location.


4b) Open up the 'RsyncBackup => Local copy => Local_Scripts ' :

Cmd scripts, 'HomeDir(copy local).cmd' - does the same thing as the WPS object, but gives you an idea on how to use rsync in scripts - useful for automated schedule backups.


RsyncBackup - Full Copy

5a) Open up the 'RsyncBackup => Local copy => Full ' :

WPS object, 'Local Full - C:\Home' is a template, it makes an exact full copy of 'c:\Home' directory to e:\RsyncBackup\Backups\Full


Usage: make new 'copy' object

  • Right click on the 'Local Full - C:\Home' object and select on menu 'copy'
  • In the copy dialog, rename 'Local Full - C:\Home' to your new copy operation, for example 'Local Full - c:\Programs'.
  • Right click on the new object and select properties, under parameters:
    •  change:  --dry-run C:\home G:\RsyncBackup\Backups\Full
    •          to:                  C:\Programs G:\RsyncBackup\Backups\Full
  • All done, modify as needed and the \source \destination can be any directory or location.

5b) Open up the 'RsyncBackup => Local copy => Local_Scripts ' :

Cmd scripts, 'HomeDir(full_local).cmd' - does the same thing as the WPS object, but gives you an idea on how to use rsync in scripts - useful for automated schedule backups.

Full copy - makes a complete backup and sync the source and destination directories. If you delete something in the source directory, it gets deleted in the destination directory (so be careful).


RsyncBackup - Local Incremental Copy

Not implemented yet, because it pissed me off.



Using 'rsyncbackup' to make network backups


networkfolder

*** Note: Full directions for setting up networking and the Rsync Server is further below.



Network - Copy

6a) Open up the 'RsyncBackup folder => Network copy => Copy ' :

WPS object, 'Network Copy - C:\Home' is a template, it makes an exact copy of 'c:\Home' directory to e:\RsyncBackup\Backups\Copy on the Rsync Server

6b) Open up the 'RsyncBackup => Network copy => Network_Scripts ' :

Cmd script, 'HomeDir(Full_network).cmd' - does the same thing as the WPS object, but gives you an idea on how to use rsync in scripts - useful for automated schedule backups.


Network - Full copy (Synchronize)

Examples (synchronize copy):
 
8a) Open up the 'RsyncBackup => Network copy => Full ' :

WPS object, 'Network Full - C:\Home' is a template, it makes an exact full copy of 'c:\Home' directory to e:\RsyncBackup\Backups\Full, it synchronize the directories.

8b) Open up the 'RsyncBackup => Network copy => Network_Scripts ' :

Cmd scripts, 'HomeDir(full_local).cmd' - does the same thing as the WPS object, but gives you an idea on how to use rsync in scripts - useful for automated schedule backups.

The --delete flag: any file that has been deleted in the \source directory, will so been deleted in the \destination directory (synchronize copy).


Network - Incremental copy

This creates a full backup and a 7 day incremental backup on the Rsync server.  For full details on Rsync and Incremental backups see:  http://www.os2notes.com/os2backup.html

Usage - my source directory for my web site changes daily, and I can easily  find changes by date and day of the week.

This works, but still needs some work - the scripts need to be re-written, because my knowledge of Rexx and programming is looking and copying what I see. So, things will look weird and dumb - but hey if they work it's good enough for me - unless you want to re-write it for me ;) 


Setting up & Connecting to a Rsync Server

It is also possible to use rsync without a remote shell as the transport. In this case you will directly connect to a remote rsync daemon, typically using TCP port 873. This obviously requires the daemon to be running on the remote system, so lets learn how to setup a Rsync Server.

1.) Download the RsyncBackup package to your server:

2.) For my setup and this example, I'll be using the root directory of my E:\ drive for installation.

3.) Unzip and move the directory to e:\RsyncBackup, the directory's name MUST NOT BE CHANGED.

4.) Run the 'install.cmd' to install the Rsync Server and create the WPS objects on the desktop.


folder


5.) And that's it - Well, we're almost done unless you're installing to the E: drive then you're done.

Rsync uses a configuration file 'rsyncd.conf' to define areas where backups will be located on the Rsync server, so we need to modify it to your system setup. These areas are called -- [Modules], when clients connect to the server they use the pre-defined [Modules] to store data in the right place on your Rsync server.

6.) Open up the 'rsyncd.conf' file and change the drive letter to match your setup:

path = e:/RsyncBackup/Backups

8.) Now we are all set, start the Rsync Server now and open a command-line window !!!

9.) Type:      rsync locahost::


Server


10) Try out a test run, no changes will be made - on your client machine type:

rsync -avzX --dry-run c:\var 192.168.1.200::RsyncBackup\Copy

-a = archive, -v = verbose, -z = compress, -X = xattribs, --dry-run = test only no changes

c:\var - is the source directory

192.168.1.200 is the IP address of the Rsync Server

RsyncBackup/Copy is the [Module] to use and use the directory 'Copy' 


Just remember that [RsyncBackup] module = e:/RsyncBackup/Backups, so:

RsyncBackup/Copy = e:/RsyncBackup/Backups/Copy