to scdbackup homepage
scdbackup
- Examples -
to top of page
A sample configuration with both a CD and a DVD writer
I do recommend that a non privileged user performs installation
and configuration. Only very few actions of the installation procedure
actually need superuser authority. They are collected in the script
SUPERUSER_CONFIGURE which is executed in Step 3 below.
The user's actions are shown in bold kbd typeface.
Step 1 of the installation procedure (see README) is
to unpack the archive in an arbitrary directory (here: /home/test) and to go
to the newly created directory scdbackup-0.8 :
- $ cp scdbackup-0.8.tar.gz /home/test
- $ cd /home/test
- $ tar xvzf scdbackup-0.8.tar.gz
- ... all the filenames of the archive ...
- $ cd scdbackup-0.8
Now (after having read README "Planning" and deciding what to put in the
system backup) i start the configuration script for CD (step 2a).
If you are solely interested in DVD you may jump to
Step 2b , DVD configuration.
- $ ./CONFIGURE_CD
-
- scdbackup for CD installation script 0.8
If there is already an installed version of cdrecord in the command path,
then an overview of the SCSI devices is shown.
If not, the superuser can run
./cdrecord_prog -scanbus
to get this overview (provided that this binary program is suitable
for your system).
- Cdrecord 1.9 (i686-suse-linux) Copyright (C) 1995-2000 Jörg Schilling
- ...
- scsibus0:
- 0,0,0 0) *
- 0,1,0 1) *
- 0,2,0 2) *
- 0,3,0 3) 'TEAC ' 'CD-ROM CD-532S ' '1.0A' Removable CD-ROM
- 0,4,0 4) *
- 0,5,0 5) *
- 0,6,0 6) *
- 0,7,0 7) *
- scsibus1:
- 1,0,0 100) 'LITE-ON ' 'LTR-48125S ' '1S05' Removable CD-ROM
- ...
Since i know that my CD recorder is a LITE-ON, i learn that its SCSI address
is 1,0,0 .
- Note: Empty input leaves the current setting unchanged
-
- SCSI-Address of CD recorder ? (e.g. 3,0)
- current setting: ""
- 1,0,0
- Speed ? (e.g. 2)
- current setting: ""
- 12
- Note: erasing CDs is usable for rewritable CD-RW media only
- Automatically erase CD before writing ? (yj1/n0)
- current setting: ""
- n
- Note: recording CDs without buffering at higher speed may fail
- if the system is slow or its load by other processes is high.
- afio (system) backups without buffering cannot be compressed.
- On the other hand, the buffer is as large as the CD (~ 650MB)
- and must be outside the backup area or explicitely excluded.
- Take care not to use the same file as with DVD !
- Buffer file name ? ( "-" = without buffering )
- current setting: ""
- /cdpuffer/isofs_cd
- Note: Options for use of the fundamental CD writing software :
- y = use the binaries cdrecord_prog and mkisofs_prog
- which are included in scdbackup.
- n = use the locally installed version of cdrecord and mkisofs
- b = like n + use option driveropts=burnproof with local cdrecord
- The following lines contain hints deduced from the output of
- both versions of cdrecord and mkisofs :
- * Your mkisofs knows option -path-list which allows large sets of
- fileadresses and therefore is very helpful.
- * Your cdrecord offers -driveropts=burnproof with your recorder.
- Advised input is : b
- Use mkisofs_prog and cdrecord_prog ? (yj1/n0/b)
- current setting: ""
- b
- Note: if you enter 0 then the directory size is determined
- by a very slow method which previews the compressed
- afio sizes.
- Estimated capacity for a compressed afio CD ? (eg. 1024m)
- current setting: ""
- 1200m
- Exclusion list of directories for system backup ?
- current setting: "/cdrom /cdrw /floppy /home /mnt /opt /proc /tmp"
- /cdrom /cdrw /floppy /home /mnt /opt /proc /tmp /cdpuffer /n
(Please note that a single line is expected here, despite your browser right now may show more than one)
- Note: The locking protocol is advised for real multi user environments
- It reserves the result scripts and the CD recorder and prevents
- access to them by other parts of this program system.
- Apply locking protocol ? (yj1/n0)
- current setting: ""
- n
- Note: The install directory will be used by SUPERUSER_CONFIGURE.
- It has to contain the subdirectories bin and sbin.
- bin should be mentioned in the PATH variable of normal users.
- sbin should be in the PATH of the superuser.
- Install directory ?
- current setting: "/usr/local"
- /usr
The interogation ends now and the parameters get written to their files.
In future runs of CONFIGURE_CD you may confirm your settings by empty input
lines and only have to enter those values which you want to change.
You also may write scripts which manipulate the files directly.
See README Appendix CONFIGURATION for a definition of the files' content
(if it isn't obvious by name and sequence :)
- writing file scdbackup_scsi_adr_value
- done
- writing file scdbackup_speed_value
- done
...
- writing file scdbackup_bin_sbin_root_value
- done
- checking for ./tmp directory
- done (found, access enabled)
-
- Note: compiling with Large File Support
- compiling program temp_lock.c
- done
- compiling program raedchen.c
- done
- compiling program askme.c
- done
- compiling program cd_backup_planer.c
- done
- checking for ./tmp directory
- done (found, access enabled)
- publishing directory address : /home/test/scdbackup-0.8 in /home/thomas/.scdbackup_dir
- done
- writing file scdbackup_is_configured
- done
- Note: Enabled MD5 checksum filter for command scdbackup_verify.
- If you do not want that, write "-" to file scdbackup_make_checksum_value
- scdbackup for CD 0.8 : First stage of installation done.
- --------------------------------------------------------
- Do not forget to run ./SUPERUSER_CONFIGURE as superuser
- --------------------------------------------------------
If you got no DVD writer or if you want to test CD in advance,
then you may skip now to Step 3
Now for DVD configuration (step 2b) :
- $ ./CONFIGURE_DVD
-
- scdbackup for DVD installation script 0.8
- Note: Empty input leaves the current setting unchanged
- Device file address of DVD writer ? (e.g. /dev/dvd)
- current setting: ""
- /dev/sr2
- Speed ? (e.g. any , 1)
- current setting: ""
- any
- Note: afio (system) backups without buffering are not compressed.
- On the other hand, the buffer is as large as the DVD (~ 4.7 GB)
- and must be outside the backup area or explicitely excluded.
- Take care not to use the same file as with CD !
- Buffer file name ? ( "-" = without buffering )
- current setting: ""
- -
- Note: Choose what version of mkisofs to use :
- y = use the binaries cdrecord_prog and mkisofs_prog
- which are included in scdbackup.
- n = use the locally installed version of cdrecord and mkisofs
- The following lines contain hints deduced from the output of
- both versions of mkisofs :
- * Your mkisofs knows option -path-list which allows large sets of
- fileadresses and therefore is very helpful.
- Advised input is : n
- Use mkisofs_prog and cdrecord_prog ? (yj1/n0/b)
- current setting: ""
- n
- Note: if you enter 0 then the directory size is determined
- by a very slow method which previews the compressed
- afio sizes.
- Estimated capacity for a compressed afio DVD ? (eg. 8000m)
- current setting: ""
- 8000m
- Exclusion list of directories for system backup ?
- current setting: "/cdrom /cdrw /floppy /home /mnt /opt /proc /tmp"
- /cdrom /cdrw /floppy /home /mnt /opt /proc /tmp /cdpuffer /n
(Please note that a single line is expected here, despite your browser right now may show more than one)
- Note: The locking protocol is advised for real multi user environments
- It reserves the result scripts and the DVD writer and prevents
- access to them by other parts of this program system.
- Apply locking protocol ? (yj1/n0)
- current setting: ""
- n
- Note: The install directory will be used by SUPERUSER_CONFIGURE.
- It has to contain the subdirectories bin and sbin.
- bin should be mentioned in the PATH variable of normal users.
- sbin should be in the PATH of the superuser.
- Install directory ?
- current setting: "/usr/local"
- /usr
Now the parameters get written to their files.
In future runs of CONFIGURE_DVD you may confirm your settings by empty input
lines and only have to enter those values which you want to change.
You also may write scripts which manipulate the files directly.
See README Appendix CONFIGURATION for a definition of the files' content (if it
isn't obvious by name and sequence :)
- writing file sdvdbackup_dev_adr_value
- done
...
- scdbackup for DVD 0.8 : First stage of installation done.
- --------------------------------------------------------
- Do not forget to run ./SUPERUSER_CONFIGURE as superuser
- --------------------------------------------------------
So, i perform Step 3 of the installation procedure.
- $ su
- Password:
- # ./SUPERUSER_CONFIGURE
- This script gives ownership of cdrecord_prog to the superuser
- and sets the setuid bit in order to allow the usage of the
- CD recorder for all users via this program.
- This script installs following commands owned by the superuser :
- scdbackup in /usr/bin access: a+rx,ug+w,o-w
- scdbackup_home in /usr/bin access: a+rx,ug+w,o-w
- scdbackup_afio in /usr/bin access: a+rx,ug+w,o-w
- scdbackup_verify in /usr/bin access: a+rx,ug+w,o-w
- scdbackup_blank in /usr/bin access: a+rx,ug+w,o-w
- scdbackup_askme in /usr/bin access: a+rx,ug+w,o-w
- scdbackup_sys in /usr/sbin access: ug+rwx,o-rwx
- For use with the DVD writer it installs accordant commands: sdvdbackup*
- like: sdvdbackup , sdvdbackup_home , sdvdbackup_verify , ...
- in directories /usr/bin resp. /usr/sbin
- Note: the install directories can be set with ./CONFIGURE_CD , ./CONFIGURE_DVD
The first question appears only if ./CONFIGURE_CD has been run :
- Do you want to enable access to the CD recorder via cdrecord_prog ? (y/n)
- n
- Permissions of cdrecord_prog left unchanged
The second question appears unconditionally :
- Do you want to install the commands below /usr ? (y/n)
- y
- confirmed
To enable access to scdbackup for the superuser, i also execute ADD_USER.
- # ./ADD_USER
- publishing directory address : /home/test/scdbackup-0.8 in /root/.scdbackup_dir
- done
- # exit
to top of page |
to scdbackup homepage
Backup run with scdbackup_home
The installation was performed in /home/test.
Locking protocol is not enabled (necessary only if more than one person is working with the machine concurrently).
The user's actions (1 command, 5 Enter keys) are shown in bold typeface
- $ scdbackup_home
The disk is rattling for about five minutes now.
My box was a K6/350 and below my HOME directory were about 3 GB.
From time to time one of these information lines appears:
- ... splitting /home/thomas
- ... splitting /home/thomas/p
- ... splitting /home/thomas/p/sorten
- ########### 1 (9 items 638.8m bytes)
- ########### 2 (16 items 634.3m bytes)
- ########### 3 (10 items 638.6m bytes)
- ########### 4 (335 items 579.3m bytes)
- ... splitting /home/thomas/.netscape
- ########### 5 (4 items 411.1m bytes)
- ########## part 1 of 5 (9 items 638.8m bytes)
- insert CD and press Enter
The distribution plan for five CDs is made and the result script took over control now.
This is the time to insert a CD-R and press the Enter key.
The following messages come from mkisofs and cdrecord:
- Total extents scheduled to be written = 326878
- 1.53% done, estimate finish Sat Nov 27 22:37:54 1999
- ...
- 99.43% done, estimate finish Sat Nov 27 22:28:10 1999
- ...
- 326878 extents written (638 Mb)
- Cdrecord release 1.6 Copyright (C) 1995-1998 Jörg Schilling
- ...
- /home/test/scdbackup-0.5/cdrecord_prog: fifo was 0 times empty and 20071 times full, min fill was 96%.
- Fixating time: 62.121s
The first CD is done, gets ejected and scdbackup asks for the next one:
- ########## part 2 of 5 (16 items 634.3m bytes)
- insert CD and press Enter
This is again the time to insert a CD-R and press the Enter key.
- ...
- ########## part 3 of 5 (10 items 638.6m bytes)
- insert CD and press Enter
Press Enter
- ...
- ########## part 4 of 5 (335 items 579.3m bytes)
- insert CD and press Enter
Press Enter
- ...
- ########## part 5 of 5 (4 items 411.1m bytes)
- insert CD and press Enter
Press Enter
- $
It is done. Five CDs. I watched two movies in TV meanwhile and changed CDs every half hour. (Speed factor 4)
I label my CD-RWs at first use with a unique id number. After each use
i write a leaflet with this id number, the date , the format ("ISO" or "afio"),
the content ("system", "music collection", whatever). Very important is the
volume number and the total number of volumes ("1 of 5", "2 of 5" ...).
With incremental backups i additionaly record level number and date of
the previous level.
Leaflet and CD are stored together (most time).
As soon as a CD-RW gets blanked i destroy the old leaflet to avoid any
confusion.
to scdbackup homepage
to top of page
Resuming an interrupted backup run
After a backup run has been interrupted, it can be resumed without distributing
the data over the CDs again. This guarantees that the new CDs match the old
ones. Care should be taken that not too many new files have been created
meanwhile. Otherwise particular CDs might overflow.
If you expect an interupted backup then you should decrease the size of
the pieces in order to get more buffer space for subsequent size increases.
Use -max_size 600m to reserve 50 MB on each CD (with 650 MB
capacity).
If you want to interrupt a lengthy backup session then kill the script
when it asks for the next CD (e.g. by pressing CTRL+C rather than Enter).
The interrupted backup can be resumed until you start a new backup.
Let us assume that the run was interrupted before or during the forth piece:
- $ scdbackup_home -resume
- ...
- ########## part 4 of 5 (335 items 579.3m bytes)
- insert CD and press Enter
Press Enter
- ...
- ########## part 5 of 5 (4 items 411.1m bytes)
- insert CD and press Enter
Press Enter
- ...
- $
It is done again. Two missing CDs have been created.
If a certain volume shall be done it can be addressed by an optional
volume number like in :
- $ scdbackup_home -resume 2
to scdbackup homepage
to top of page
Verifying a result CD
Some CD recorders and some CD media are not as reliable as they should be.
It may be necessary to verify the readabilty and correctness of the backup
CDs.
Be aware that this check does not ensure the completeness of your
backup. Any faults of scdbackup or yourself will not be detected. Only the
success of your CD recorder's efforts, the individual CD and your CD reading
device will be tested.
So you should look at the total backup size wether it is plausible. After
verifying the single CDs you should also use ASKME to locate some files and
check wether they really are on their CD.
If you put the most recently burned CD into the tray of /dev/cdrom then you
just have to execute
- $ scdbackup_verify
- Comparing with record :
- 1_1 A30110.222002 260765696 c30d13d53977cecb87017896d72616ae
- OK. Record and data input match. Daten passen zum Record.
- 248.7m bytes
In case of failure there would be no "OK" but for example :
- SIZE MISMATCH. FALSCHE GROESSE :
- record=260796416 stdin=255420416
or
- MD5 CHECKSUM MISMATCH. FALSCHE MD5 PRUEFSUMME :
- record= c30d13d53977cecb87017896d72616ae
- stdin = c7a17c2d8e7de65e82756527fe498c75
In such a case you should hurry to burn a new CD by
scdbackup -resume # .
Do this as long as the data from the disk still fit on the CD.
If you want to verify at /dev/cdrecorder the first volume of the most recent
three volume backup then execute
- $ scdbackup_verify /dev/cdrecorder 1_3
You may also search for the record of an arbitrary CD by record name "-auto"
- $ scdbackup_verify /dev/cdrom -auto
- Loaded 216 checksum records from /home/test/scdbackup-0.8/tmp/backup_log
- / 312.5m bytes
- Found matching record :
- 1_1 A30113.235425 327899136 e11b1728684dad978c7699005350a569
- Note: read 18432 surplus bytes
- OK. Matching Record found . Passender Record gefunden .
- 312.7m bytes
The note about surplus bytes is typical for CD reading devices.
An amount of more than 65536 is quite suspicious, though.
More options of scdbackup_verify are explained by
- $ scdbackup_verify -help
(or in file scdbackup_verify_help )
to scdbackup homepage
to top of page
Searching and restoring a particular file or directory
From time to time, i spoil some of my files. Linux is quite safe, i am not.
For example, i lost the file $HOME/gifs/milleniumbug.gif
But i have got my backup !
First i mount one of the CDs (it doesn't matter which one)
- $ mount /cdrom
Since i have got the "exec" option in the /cdrom line of /etc/fstab, i can execute the ASKME script directly from CD.
- $ /cdrom/ASKME /home/thomas/gifs/milleniumbug.gif
- 5 /home/thomas/gifs /home/thomas/gifs/milleniumbug.gif
From this answer i learn that the whole directory "gifs" is on CD number 5.
Now i have to mount this CD and can restore my lost file.
- $ umount /cdrom
I insert CD number 5 and execute
- $ mount /cdrom
- $ cp /cdrom/gifs/milleniumbug.gif $HOME/gifs
and the milleniumbug (75 K GIF movie) is back on my disk.
For some more info about ASKME see ASKME -help
to scdbackup homepage
to top of page
Incremental backups
If you got a substantial amount of data, it becomes annoying to backup the old
data with each new run. To avoid this (within certain limitations) one may
use an incremental backup scheme.
For this purpose a backup configuration needs to be created. We simply
use the $HOME backup but one could also record an arbitrarily complicated
set of data sources and options. After the configuration is created, only
its name and the desired update level have to be given with the scdbackup
command.
- $ scdbackup_home \
- -conf_dir $HOME/home_backup \
- -level -create_configuration
This creates the directory $HOME/home_backup
(and fails if it already exists),
records the parameters of the backup (which are set within script
scdbackup_home) and enables recording of the state
of the backup area (i.e your $HOME with its sub directory trees).
Also it starts the initial base backup which is called -level 0.
This backup is like any other run of scdbackup_home (see above)
besides it may take quite some time to obtain CRC fingerprints from all
the files in your $HOME tree. Also it eats some of your disk space by
writing file
$HOME/home_backup/level_0/content_file_list.gz
The difference comes with the update run a few days later:
- $ scdbackup \
- -conf_dir $HOME/home_backup \
- -level 1
This command causes the backup configuration
$HOME/home_backup to be read
and updated on level 1. Level 1 backups all files that were changed or newly
created since the last run of level 0. By comparing the current state with
the one recorded from level 0, it also detects files which have vanished or
got moved.
So from now on, you do base backups with -level 0. These base
backups include all files without any time constraint. Updates of the base
backups are done by -level 1 in regular intervals.
You may also do short term updates which refer to the last run of level 1
by running a -level 2 backup.
There is no upper limit of level numbers but you may only increase them by
one with each run. That is, after a level 0 run was done, you cannot do
a level 2 run but only level 1. The special level -next can be
used to get the highest permittable level.
A backup scheme which from monday to saturday records only the
daily changes and makes a full backup each sunday:
Monday : -conf_dir $HOME/home_backup -level -next
...
Saturday: -conf_dir $HOME/home_backup -level -next
Sunday : -conf_dir $HOME/home_backup -level 0
Now the media of monday to saturday may be recycled.
A scheme which makes quarterly (13 weeks) full backups, four-weekly
and weekly updates (-conf_dir same as above). This scheme lets you
retrieve old file versions for quite a long time.
Week 1 : -level 1
Week 2 : -level 2
Week 3 : -level 2
Week 4 : -level 2
Week 5 : -level 1
Week 6 : -level 2
Week 7 : -level 2
Week 8 : -level 2
Week 9 : -level 1
Week 10 : -level 2
Week 11 : -level 2
Week 12 : -level 2
Week 13 : -level 0
For week 14 hop to week 1.
It is advised to keep levels 0 and 1 for a longer time.
To combine both schemes shown, simply replace the sunday run of
scheme 1 by the weekly run of scheme 2.
If you want to change the configuration parameters, you may either run
the altered command with special level -update_configuration:
- $ scdbackup_home \
- -not trashdir newsdir \
- -conf_dir $HOME/home_backup \
- -update_configuration
or you may directly edit the parameter list file
$HOME/home_backup/specific_arguments
In this file, each single argument is recorded in a single line. The list
begins with arguments automatically set by scdbackup and ends with arguments
given by you (if any).
Be aware what the options mean before you delete or change them. Also
perform a new -level 0 backup afterwards.
Incremental backups include the ASKME scripts of lower levels. If you use
the ASKME script of level 2 it will try to execute the ASKMEs of level 1
and level 0 if it cannot find a file by itself. The option -levelall
causes all levels to be searched in any case.
Since a large multilayer backup may produce very large and slow ASKME scripts
there is a specialized binary program which can extract the information from
the scripts and can perform searches much faster. (See
scdbackup_askme_help
)
These features are not restricted to scdbackup_home but may also
be used with scdbackup and scdbackup_afio. One may
have an abritrary number of configurations in distinct directories.
Now for some restrictions:
-
If you cannot afford the disk space and additional computation time for
the recording of the backup area state, then you may disable this feature
by using option -disable_safe_leveling together with
-level -create_configuration.
But be aware that file movements and deletions cannot be detected that way
and will therefore not be included in the higher backup levels. Such a
backup is still a good source for recovering from limited data accidents
but it may be unable to accurately reconstruct the most recent state of
your backup area.
-
Because of the restricted number of fileaddresses possible with older
versions of mkisof there may be whole directories included in the backup
despite they contain some older files.
Therefore, ./CONFIGURE advises you to use locally installed cdrtools
if their mkisofs mentions "-path-list" in its help text.
If you agree, you will get a very high -max_items limit
and should have no problem with above issue.
to scdbackup homepage
to top of page
Restoring a multilevel backup
There are moments when a backup area needs to be restored from scratch.
Possibly on a completely new computer system. This example describes how
to restore a big $HOME tree by using a bash shell with standard commands
and programs.
In complicated disk situations you will have to prepare the mount tree below
$HOME in advance. It is up to you to provide the disk space necessary to take
the backup's content.
-
Get a start directory outside your $HOME :
This example assumes a $HOME entirely on the same filesystem as its parent
and enough room on that filesystem to take the whole backup. So we will
rebuild $HOME below "$HOME"_RESTORED and later move it to its destination.
Have your superuser prepare a directory for you
- $ TARGET_ROOT="$HOME"_RESTORED
- $ echo "Dear superuser, please make me directory $TARGET_ROOT"
Your superuser then is so kind to mkdir the desired
directory and to chown it for you. So you now can begin
to create own files.
- $ TARGET="$TARGET_ROOT"/$(basename "$HOME")
- $ mkdir "$TARGET"
- $ export SCDBACKUP_RESTORE_ROOT="$TARGET"
-
Level 0:
Copy level 0 to your disk. With a CD set from the above example that would
be for each CD :
- $ mount /cdrom
- $ cp -a /cdrom/* "$TARGET"
- $ umount /cdrom
It is necessary to make all directories fully accessible because there may
be the need to create or delete files during restore of the next level.
We will restore the original directory permissions at the end of this
example.
- $ find "$TARGET" -type d -exec chmod 0755 '{}' ';'
In cases of extreme self denial of access rights you may have to do this
after each single CD. Watch out for complaints of the cp command.
-
Level 1:
Copy level 1 to your disk. For each CD :
- $ mount /cdrom
- $ cp -a /cdrom/* "$TARGET"
- $ umount /cdrom
Again set directory permissions
- $ find "$TARGET" -type d -exec chmod 0755 '{}' ';'
Remove the files that vanished between level 0 and level 1 :
- $ gunzip <"$TARGET"/added_by_scdbackup/VANISHED.sh.gz | /bin/sh
-
Further levels:
Repeat the steps of level 1 for any higher level you got
-
Now restore the directory permissions as they were when the last level
was recorded.
- $ gunzip <"$TARGET"/added_by_scdbackup/DIR_PERMS.sh.gz | /bin/sh
-
Clean up and have your superuser do the move for you. The existing $HOME is
to be moved to a new name and then replaced by the newly built tree. Finally
the start directory gets removed.
- $ rm "$TARGET"/added_by_scdbackup/*
- $ rmdir "$TARGET"/added_by_scdbackup
- $ echo "Dear superuser, please do for me :"
- $ echo " mv $HOME ${HOME}_REPLACED"
- $ echo " mv $TARGET $HOME"
- $ echo " rmdir $TARGET_ROOT"
Dear superuser, be aware of the ill effects if $HOME contains blanks or other
characters meaningful to the shell. If you allowed this, now is the time
to call yourself names.
to top of page
CD mount options
The /cdrom line in my /etc/fstab looks like this
- /dev/cdrom /cdrom iso9660 ro,user,noauto,unhide,exec
to top of page |
to scdbackup homepage