to scdbackup homepage


scdbackup
- Examples -




to top of page


A sample configuration with both a CD and a DVD writer


I do recommend that a non privileged user performs installation and configuration. Only very few actions of the installation procedure actually need superuser authority. They are collected in the script SUPERUSER_CONFIGURE which is executed in Step 3 below.

The user's actions are shown in bold kbd typeface.

Step 1 of the installation procedure (see README) is to unpack the archive in an arbitrary directory (here: /home/test) and to go to the newly created directory scdbackup-0.8 :
$ cp scdbackup-0.8.tar.gz /home/test
$ cd /home/test
$ tar xvzf scdbackup-0.8.tar.gz
... all the filenames of the archive ...
$ cd scdbackup-0.8
Now (after having read README "Planning" and deciding what to put in the system backup) i start the configuration script for CD (step 2a). If you are solely interested in DVD you may jump to Step 2b , DVD configuration.
$ ./CONFIGURE_CD
 
scdbackup for CD installation script 0.8
If there is already an installed version of cdrecord in the command path, then an overview of the SCSI devices is shown.
If not, the superuser can run ./cdrecord_prog -scanbus to get this overview (provided that this binary program is suitable for your system).

Cdrecord 1.9 (i686-suse-linux) Copyright (C) 1995-2000 Jörg Schilling
...
scsibus0:
0,0,0 0) *
0,1,0 1) *
0,2,0 2) *
0,3,0 3) 'TEAC ' 'CD-ROM CD-532S ' '1.0A' Removable CD-ROM
0,4,0 4) *
0,5,0 5) *
0,6,0 6) *
0,7,0 7) *
scsibus1:
1,0,0 100) 'LITE-ON ' 'LTR-48125S ' '1S05' Removable CD-ROM
...
Since i know that my CD recorder is a LITE-ON, i learn that its SCSI address is 1,0,0 .

Note: Empty input leaves the current setting unchanged
 
SCSI-Address of CD recorder ? (e.g. 3,0)
current setting: ""
1,0,0

Speed ? (e.g. 2)
current setting: ""
12

Note: erasing CDs is usable for rewritable CD-RW media only

Automatically erase CD before writing ? (yj1/n0)
current setting: ""
n

Note: recording CDs without buffering at higher speed may fail
if the system is slow or its load by other processes is high.
afio (system) backups without buffering cannot be compressed.
On the other hand, the buffer is as large as the CD (~ 650MB)
and must be outside the backup area or explicitely excluded.
Take care not to use the same file as with DVD !

Buffer file name ? ( "-" = without buffering )
current setting: ""
/cdpuffer/isofs_cd

Note: Options for use of the fundamental CD writing software :
y = use the binaries cdrecord_prog and mkisofs_prog
which are included in scdbackup.
n = use the locally installed version of cdrecord and mkisofs
b = like n + use option driveropts=burnproof with local cdrecord
The following lines contain hints deduced from the output of
both versions of cdrecord and mkisofs :
* Your mkisofs knows option -path-list which allows large sets of
fileadresses and therefore is very helpful.
* Your cdrecord offers -driveropts=burnproof with your recorder.
Advised input is : b

Use mkisofs_prog and cdrecord_prog ? (yj1/n0/b)
current setting: ""
b

Note: if you enter 0 then the directory size is determined
by a very slow method which previews the compressed
afio sizes.

Estimated capacity for a compressed afio CD ? (eg. 1024m)
current setting: ""
1200m

Exclusion list of directories for system backup ?
current setting: "/cdrom /cdrw /floppy /home /mnt /opt /proc /tmp"
/cdrom /cdrw /floppy /home /mnt /opt /proc /tmp /cdpuffer /n
(Please note that a single line is expected here, despite your browser right now may show more than one)
Note: The locking protocol is advised for real multi user environments
It reserves the result scripts and the CD recorder and prevents
access to them by other parts of this program system.

Apply locking protocol ? (yj1/n0)
current setting: ""
n

Note: The install directory will be used by SUPERUSER_CONFIGURE.
It has to contain the subdirectories bin and sbin.
bin should be mentioned in the PATH variable of normal users.
sbin should be in the PATH of the superuser.

Install directory ?
current setting: "/usr/local"
/usr

The interogation ends now and the parameters get written to their files.
In future runs of CONFIGURE_CD you may confirm your settings by empty input lines and only have to enter those values which you want to change. You also may write scripts which manipulate the files directly.
See README Appendix CONFIGURATION for a definition of the files' content (if it isn't obvious by name and sequence :)

writing file scdbackup_scsi_adr_value
done
writing file scdbackup_speed_value
done
...
writing file scdbackup_bin_sbin_root_value
done
checking for ./tmp directory
done (found, access enabled)
Note: compiling with Large File Support
compiling program temp_lock.c
done
compiling program raedchen.c
done
compiling program askme.c
done
compiling program cd_backup_planer.c
done
checking for ./tmp directory
done (found, access enabled)

publishing directory address : /home/test/scdbackup-0.8 in /home/thomas/.scdbackup_dir
done
writing file scdbackup_is_configured
done

Note: Enabled MD5 checksum filter for command scdbackup_verify.
If you do not want that, write "-" to file scdbackup_make_checksum_value

scdbackup for CD 0.8 : First stage of installation done.
--------------------------------------------------------
Do not forget to run ./SUPERUSER_CONFIGURE as superuser
--------------------------------------------------------
If you got no DVD writer or if you want to test CD in advance, then you may skip now to Step 3

Now for DVD configuration (step 2b) :
$ ./CONFIGURE_DVD
 
scdbackup for DVD installation script 0.8

Note: Empty input leaves the current setting unchanged

Device file address of DVD writer ? (e.g. /dev/dvd)
current setting: ""
/dev/sr2

Speed ? (e.g. any , 1)
current setting: ""
any

Note: afio (system) backups without buffering are not compressed.
On the other hand, the buffer is as large as the DVD (~ 4.7 GB)
and must be outside the backup area or explicitely excluded.
Take care not to use the same file as with CD !

Buffer file name ? ( "-" = without buffering )
current setting: ""
-

Note: Choose what version of mkisofs to use :
y = use the binaries cdrecord_prog and mkisofs_prog
which are included in scdbackup.
n = use the locally installed version of cdrecord and mkisofs
The following lines contain hints deduced from the output of
both versions of mkisofs :
* Your mkisofs knows option -path-list which allows large sets of
fileadresses and therefore is very helpful.

Advised input is : n

Use mkisofs_prog and cdrecord_prog ? (yj1/n0/b)
current setting: ""
n

Note: if you enter 0 then the directory size is determined
by a very slow method which previews the compressed
afio sizes.

Estimated capacity for a compressed afio DVD ? (eg. 8000m)
current setting: ""
8000m

Exclusion list of directories for system backup ?
current setting: "/cdrom /cdrw /floppy /home /mnt /opt /proc /tmp"
/cdrom /cdrw /floppy /home /mnt /opt /proc /tmp /cdpuffer /n
(Please note that a single line is expected here, despite your browser right now may show more than one)
Note: The locking protocol is advised for real multi user environments
It reserves the result scripts and the DVD writer and prevents
access to them by other parts of this program system.

Apply locking protocol ? (yj1/n0)
current setting: ""
n

Note: The install directory will be used by SUPERUSER_CONFIGURE.
It has to contain the subdirectories bin and sbin.
bin should be mentioned in the PATH variable of normal users.
sbin should be in the PATH of the superuser.

Install directory ?
current setting: "/usr/local"
/usr

Now the parameters get written to their files.
In future runs of CONFIGURE_DVD you may confirm your settings by empty input lines and only have to enter those values which you want to change. You also may write scripts which manipulate the files directly.
See README Appendix CONFIGURATION for a definition of the files' content (if it isn't obvious by name and sequence :)

writing file sdvdbackup_dev_adr_value
done
...
scdbackup for DVD 0.8 : First stage of installation done.
--------------------------------------------------------
Do not forget to run ./SUPERUSER_CONFIGURE as superuser
--------------------------------------------------------
So, i perform Step 3 of the installation procedure.
$ su
Password:
# ./SUPERUSER_CONFIGURE

This script gives ownership of cdrecord_prog to the superuser
and sets the setuid bit in order to allow the usage of the
CD recorder for all users via this program.
This script installs following commands owned by the superuser :
scdbackup in /usr/bin access: a+rx,ug+w,o-w
scdbackup_home in /usr/bin access: a+rx,ug+w,o-w
scdbackup_afio in /usr/bin access: a+rx,ug+w,o-w
scdbackup_verify in /usr/bin access: a+rx,ug+w,o-w
scdbackup_blank in /usr/bin access: a+rx,ug+w,o-w
scdbackup_askme in /usr/bin access: a+rx,ug+w,o-w
scdbackup_sys in /usr/sbin access: ug+rwx,o-rwx
For use with the DVD writer it installs accordant commands: sdvdbackup*
like: sdvdbackup , sdvdbackup_home , sdvdbackup_verify , ...
in directories /usr/bin resp. /usr/sbin
Note: the install directories can be set with ./CONFIGURE_CD , ./CONFIGURE_DVD
The first question appears only if ./CONFIGURE_CD has been run :
Do you want to enable access to the CD recorder via cdrecord_prog ? (y/n)
n
Permissions of cdrecord_prog left unchanged
The second question appears unconditionally :
Do you want to install the commands below /usr ? (y/n)
y
confirmed

To enable access to scdbackup for the superuser, i also execute ADD_USER.

# ./ADD_USER
publishing directory address : /home/test/scdbackup-0.8 in /root/.scdbackup_dir
done
# exit

to top of page | to scdbackup homepage


Backup run with scdbackup_home


The installation was performed in /home/test. Locking protocol is not enabled (necessary only if more than one person is working with the machine concurrently).
The user's actions (1 command, 5 Enter keys) are shown in bold typeface

$ scdbackup_home
The disk is rattling for about five minutes now.
My box was a K6/350 and below my HOME directory were about 3 GB.
From time to time one of these information lines appears:

... splitting /home/thomas
... splitting /home/thomas/p
... splitting /home/thomas/p/sorten
########### 1 (9 items 638.8m bytes)
########### 2 (16 items 634.3m bytes)
########### 3 (10 items 638.6m bytes)
########### 4 (335 items 579.3m bytes)
... splitting /home/thomas/.netscape
########### 5 (4 items 411.1m bytes)
########## part 1 of 5 (9 items 638.8m bytes)
insert CD and press Enter
The distribution plan for five CDs is made and the result script took over control now.
This is the time to insert a CD-R and press the Enter key.

The following messages come from mkisofs and cdrecord:
Total extents scheduled to be written = 326878
1.53% done, estimate finish Sat Nov 27 22:37:54 1999
...
99.43% done, estimate finish Sat Nov 27 22:28:10 1999
...
326878 extents written (638 Mb)
Cdrecord release 1.6 Copyright (C) 1995-1998 Jörg Schilling
...
/home/test/scdbackup-0.5/cdrecord_prog: fifo was 0 times empty and 20071 times full, min fill was 96%.
Fixating time: 62.121s
The first CD is done, gets ejected and scdbackup asks for the next one:
########## part 2 of 5 (16 items 634.3m bytes)
insert CD and press Enter
This is again the time to insert a CD-R and press the Enter key.
...
########## part 3 of 5 (10 items 638.6m bytes)
insert CD and press Enter
Press Enter
...
########## part 4 of 5 (335 items 579.3m bytes)
insert CD and press Enter
Press Enter
...
########## part 5 of 5 (4 items 411.1m bytes)
insert CD and press Enter
Press Enter
$
It is done. Five CDs. I watched two movies in TV meanwhile and changed CDs every half hour. (Speed factor 4)

I label my CD-RWs at first use with a unique id number. After each use i write a leaflet with this id number, the date , the format ("ISO" or "afio"), the content ("system", "music collection", whatever). Very important is the volume number and the total number of volumes ("1 of 5", "2 of 5" ...). With incremental backups i additionaly record level number and date of the previous level.
Leaflet and CD are stored together (most time). As soon as a CD-RW gets blanked i destroy the old leaflet to avoid any confusion.

to scdbackup homepage

to top of page


Resuming an interrupted backup run


After a backup run has been interrupted, it can be resumed without distributing the data over the CDs again. This guarantees that the new CDs match the old ones. Care should be taken that not too many new files have been created meanwhile. Otherwise particular CDs might overflow.
If you expect an interupted backup then you should decrease the size of the pieces in order to get more buffer space for subsequent size increases.
Use -max_size 600m to reserve 50 MB on each CD (with 650 MB capacity).

If you want to interrupt a lengthy backup session then kill the script when it asks for the next CD (e.g. by pressing CTRL+C rather than Enter). The interrupted backup can be resumed until you start a new backup.

Let us assume that the run was interrupted before or during the forth piece:
$ scdbackup_home -resume
...
########## part 4 of 5 (335 items 579.3m bytes)
insert CD and press Enter
Press Enter
...
########## part 5 of 5 (4 items 411.1m bytes)
insert CD and press Enter
Press Enter
...
$
It is done again. Two missing CDs have been created.

If a certain volume shall be done it can be addressed by an optional volume number like in :
$ scdbackup_home -resume 2

to scdbackup homepage

to top of page


Verifying a result CD


Some CD recorders and some CD media are not as reliable as they should be. It may be necessary to verify the readabilty and correctness of the backup CDs.
Be aware that this check does not ensure the completeness of your backup. Any faults of scdbackup or yourself will not be detected. Only the success of your CD recorder's efforts, the individual CD and your CD reading device will be tested.

So you should look at the total backup size wether it is plausible. After verifying the single CDs you should also use ASKME to locate some files and check wether they really are on their CD.

If you put the most recently burned CD into the tray of /dev/cdrom then you just have to execute
$ scdbackup_verify

Comparing with record :
1_1 A30110.222002 260765696 c30d13d53977cecb87017896d72616ae

OK. Record and data input match. Daten passen zum Record.

248.7m bytes

In case of failure there would be no "OK" but for example :
SIZE MISMATCH. FALSCHE GROESSE :
record=260796416 stdin=255420416
or
MD5 CHECKSUM MISMATCH. FALSCHE MD5 PRUEFSUMME :
record= c30d13d53977cecb87017896d72616ae
stdin = c7a17c2d8e7de65e82756527fe498c75
In such a case you should hurry to burn a new CD by scdbackup -resume # . Do this as long as the data from the disk still fit on the CD.

If you want to verify at /dev/cdrecorder the first volume of the most recent three volume backup then execute
$ scdbackup_verify /dev/cdrecorder 1_3
You may also search for the record of an arbitrary CD by record name "-auto"
$ scdbackup_verify /dev/cdrom -auto
Loaded 216 checksum records from /home/test/scdbackup-0.8/tmp/backup_log
/ 312.5m bytes

Found matching record :
1_1 A30113.235425 327899136 e11b1728684dad978c7699005350a569

Note: read 18432 surplus bytes

OK. Matching Record found . Passender Record gefunden .

312.7m bytes
The note about surplus bytes is typical for CD reading devices. An amount of more than 65536 is quite suspicious, though.

More options of scdbackup_verify are explained by

$ scdbackup_verify -help
(or in file scdbackup_verify_help )


to scdbackup homepage

to top of page


Searching and restoring a particular file or directory


From time to time, i spoil some of my files. Linux is quite safe, i am not.
For example, i lost the file $HOME/gifs/milleniumbug.gif
But i have got my backup !

First i mount one of the CDs (it doesn't matter which one)
$ mount /cdrom
Since i have got the "exec" option in the /cdrom line of /etc/fstab, i can execute the ASKME script directly from CD.
$ /cdrom/ASKME /home/thomas/gifs/milleniumbug.gif
5 /home/thomas/gifs /home/thomas/gifs/milleniumbug.gif
From this answer i learn that the whole directory "gifs" is on CD number 5.
Now i have to mount this CD and can restore my lost file.
$ umount /cdrom
I insert CD number 5 and execute
$ mount /cdrom
$ cp /cdrom/gifs/milleniumbug.gif $HOME/gifs
and the milleniumbug (75 K GIF movie) is back on my disk.

For some more info about ASKME see ASKME -help



to scdbackup homepage

to top of page


Incremental backups


If you got a substantial amount of data, it becomes annoying to backup the old data with each new run. To avoid this (within certain limitations) one may use an incremental backup scheme.

For this purpose a backup configuration needs to be created. We simply use the $HOME backup but one could also record an arbitrarily complicated set of data sources and options. After the configuration is created, only its name and the desired update level have to be given with the scdbackup command.
$ scdbackup_home \
-conf_dir $HOME/home_backup \
-level -create_configuration
This creates the directory $HOME/home_backup (and fails if it already exists), records the parameters of the backup (which are set within script scdbackup_home) and enables recording of the state of the backup area (i.e your $HOME with its sub directory trees).
Also it starts the initial base backup which is called -level 0. This backup is like any other run of scdbackup_home (see above) besides it may take quite some time to obtain CRC fingerprints from all the files in your $HOME tree. Also it eats some of your disk space by writing file $HOME/home_backup/level_0/content_file_list.gz
The difference comes with the update run a few days later:
$ scdbackup \
-conf_dir $HOME/home_backup \
-level 1
This command causes the backup configuration $HOME/home_backup to be read and updated on level 1. Level 1 backups all files that were changed or newly created since the last run of level 0. By comparing the current state with the one recorded from level 0, it also detects files which have vanished or got moved.

So from now on, you do base backups with -level 0. These base backups include all files without any time constraint. Updates of the base backups are done by -level 1 in regular intervals.
You may also do short term updates which refer to the last run of level 1 by running a -level 2 backup.
There is no upper limit of level numbers but you may only increase them by one with each run. That is, after a level 0 run was done, you cannot do a level 2 run but only level 1. The special level -next can be used to get the highest permittable level.

A backup scheme which from monday to saturday records only the daily changes and makes a full backup each sunday:
  Monday  :  -conf_dir $HOME/home_backup -level -next
  ...
  Saturday:  -conf_dir $HOME/home_backup -level -next
  Sunday  :  -conf_dir $HOME/home_backup -level 0
Now the media of monday to saturday may be recycled.

A scheme which makes quarterly (13 weeks) full backups, four-weekly and weekly updates (-conf_dir same as above). This scheme lets you retrieve old file versions for quite a long time.
  Week  1 :  -level 1
  Week  2 :  -level 2
  Week  3 :  -level 2
  Week  4 :  -level 2
  Week  5 :  -level 1
  Week  6 :  -level 2
  Week  7 :  -level 2
  Week  8 :  -level 2
  Week  9 :  -level 1
  Week 10 :  -level 2
  Week 11 :  -level 2
  Week 12 :  -level 2
  Week 13 :  -level 0
For week 14 hop to week 1.
It is advised to keep levels 0 and 1 for a longer time.

To combine both schemes shown, simply replace the sunday run of scheme 1 by the weekly run of scheme 2.

If you want to change the configuration parameters, you may either run the altered command with special level -update_configuration:
$ scdbackup_home \
-not trashdir newsdir \
-conf_dir $HOME/home_backup \
-update_configuration
or you may directly edit the parameter list file
$HOME/home_backup/specific_arguments
In this file, each single argument is recorded in a single line. The list begins with arguments automatically set by scdbackup and ends with arguments given by you (if any).
Be aware what the options mean before you delete or change them. Also perform a new -level 0 backup afterwards.

Incremental backups include the ASKME scripts of lower levels. If you use the ASKME script of level 2 it will try to execute the ASKMEs of level 1 and level 0 if it cannot find a file by itself. The option -levelall causes all levels to be searched in any case.
Since a large multilayer backup may produce very large and slow ASKME scripts there is a specialized binary program which can extract the information from the scripts and can perform searches much faster. (See scdbackup_askme_help )

These features are not restricted to scdbackup_home but may also be used with scdbackup and scdbackup_afio. One may have an abritrary number of configurations in distinct directories.

Now for some restrictions:

to scdbackup homepage

to top of page


Restoring a multilevel backup


There are moments when a backup area needs to be restored from scratch. Possibly on a completely new computer system. This example describes how to restore a big $HOME tree by using a bash shell with standard commands and programs.
In complicated disk situations you will have to prepare the mount tree below $HOME in advance. It is up to you to provide the disk space necessary to take the backup's content.


to top of page


CD mount options


The /cdrom line in my /etc/fstab looks like this
/dev/cdrom /cdrom iso9660 ro,user,noauto,unhide,exec



to top of page | to scdbackup homepage