to scdbackup homepage
scdbackup
- Examples -
to top of page
A sample configuration with both a CD and a DVD writer
This is an installation for a normal user. Provided this user can already run
programs cdrecord and growisofs, or cdrskin, or xorriso, there will be no
urgent need for superuser action.
Nevertheless the superuser should later follow these steps to create a
private installation for system backups and eventual bundled user backups.
The user's actions are shown in bold kbd typeface.
Step 1 of the installation procedure (see README) is
to unpack the archive in an arbitrary directory (here: $HOME/scdbackup) and
to go to the newly created directory scdbackup-0.9.2 :
- $ mkdir $HOME/scdbackup
- $ cd $HOME/scdbackup
- $ tar xzf ..../scdbackup-0.9.2.tar.gz
- $ cd scdbackup-0.9.2/inst
I read README paragraph "Planning" to prepare myself for
the coming questions:
I found out the device file address of my DVD writer and briefly tested it
with xorriso. I decided not to need a split directory for overly large files
and no buffer file for avoiding failed burns with legacy hardware.
Now i start the configuration script for CD (step 2a).
If you are solely interested in DVD you may jump to
Step 2b , DVD configuration.
- $ ./CONFIGURE_CD
-
- scdbackup for CD installation script 0.9.2
-
- Evaluating availability of ISO 9660 formatter programs:
- ...
- Evaluating availability of CD burn programs:
- ...
- ========================================================================
- Note: Empty input leaves the current setting unchanged
-
- Note: Choose a ISO 9660 formatter program. See list in () brackets.
- You may add "-J" and "-joliet-long" after the program name.
-
-
What ISO 9660 formatter program to use ? ( xorriso genisoimage mkisofs )
- current setting  : ""
- Your input please:
- xorriso
-
- Note: Choose a CD burn program. Available local programs are:
- cdrskin cdrecord xorriso wodim
- ...
- "-" disables use of any burn program.
-
- What CD burn program to use ? ( cdrskin cdrecord xorriso wodim )
- current setting  : ""
- Your input please:
- xorriso
The chosen CD burn program will then show an overview of the burner devices.
-
----------------------------------------------------- devices
/dev/sr0 : 'TSSTcorp' 'CDDVDW SH-S203B'
/dev/sr1 : 'PHILIPS ' 'SPD3300L'
/dev/hda : 'HL-DT-ST' 'DVD-ROM GDR8162B'
/dev/hdc : 'LITE-ON ' 'LTR-48125S'
----------------------------------------------------- end of device list
Note: Adresses of detected drives are shown in the () list.
Enter "-" to skip this input and disable burning for now.
Address of CD recorder ? ( /dev/sr0 /dev/sr1 /dev/hda /dev/hdc )
current setting  : ""
Your input please:
/dev/sr0
Evaluating CD burner at address /dev/sr0 . This may last some seconds.
Burner evaluation done
Speed ? (e.g. 2)
current setting  : ""
Your input please:
48
Note: erasing CDs is usable for rewritable CD-RW media only
Automatically erase CD before writing ? (yj1/n0)
current setting  : ""
Your input please:
y
Note: Files which do not fit on the media usually are excluded from
the backup. But a directory outside the backup area
may be used to buffer pieces of such files which then become
part of the backup. The directory has to be empty now or has
already to contain a file scdbackup_split_file_directory_mark.
Its filesystem has to be able to carry a full media content
(650MB). Take care not to use the same directory as with DVD !
If you enter "-" then large files are excluded.
Your local xorriso is suitable for "-xorriso_cut_out"
which needs no extra disk space.
Large file split directory ? ( "-" = exclude large files)
current setting  : ""
Your input please:
-xorriso_cut_out
Note: Recording CDs without buffering at higher speed may fail
if the system is slow or its load by other processes is high.
Compressed afio (system) backups may produce arbitrarily sparse
output which strongly increases the risk of such a failure.
Therefore compression usually is disabled without buffer.
On the other hand, the buffer is as large as the CD (~ 650MB)
and must be outside the backup area or explicitely excluded.
Take care not to use the same file as with DVD !
Special name "-" disables buffering. "-afio_compress" does the
same but enables afio compression (needs burnproof hardware).
Buffer file name ? ( or "-" or "-afio_compress" )
current setting  : ""
Your input please:
-afio_compress
Note: if you enter 0 then the directory size is determined
by a very slow method which previews the compressed
afio sizes.
Estimated capacity for a compressed afio CD ? (eg. 1024m)
current setting  : ""
Your input please:
1200m
Exclusion list of directories for system backup ?
current setting  : "/cdrom /cdrw /floppy /home /mnt /opt /proc /tmp"
Your input please:
Normal users just press Enter here. This input is only of interest
for the superuser.
Please note that -if ever- a single line is expected here, despite your
browser right now may show more than one.
- unchanged: "/cdrom /cdrw /floppy /home /mnt /opt /proc /tmp"
The interogation ends now and the parameters get written to their files.
In future runs of CONFIGURE_CD you may confirm your settings by empty input
lines and only have to enter those values which you want to change.
The content of those files may be temporarily overriden by shell variables.
See README Appendix CONFIGURATION for a definition of the files' content
(if it isn't obvious by name and sequence :)
- writing file scdbackup_use_genisoimage_value
- done
- writing file scdbackup_use_cdrskin_value
- done
...
- Now applied : permission scheme 'owner_only'
-
- Note: You may change the permission scheme by ./SET_PERMISSIONS
-
- scdbackup for CD 0.9.2 : First stage of installation done.
- --------------------------------------------------------
- The commands of your scdbackup installation are now available in :
- /home/thomas/scdbackup/scdbackup-0.9.2/cmd
-
- You may want to add this address to your PATH variable
- or ask your superuser to run ./SUPERUSER_CONFIGURE in directory
- /home/thomas/scdbackup/scdbackup-0.9.2/inst
- --------------------------------------------------------
If you got no DVD writer or if you want to test CD in advance,
then you may skip now to Step 3
Now for DVD configuration (step 2b) :
- $ ./CONFIGURE_DVD
-
- scdbackup for DVD installation script 0.9.2
-
- Evaluating availability of ISO 9660 formatter programs:
- ...
- Evaluating availability of DVD burn programs:
- ...
- ========================================================================
- Note: Empty input leaves the current setting unchanged
-
- Note: Choose a ISO 9660 formatter program. See list in () brackets.
- You may add "-J" and "-joliet-long" after the program name.
-
-
What ISO 9660 formatter program to use ? ( xorriso genisoimage mkisofs )
- current setting  : ""
- Your input please:
- xorriso
-
- Note: Choose a DVD burn program. Available local programs are:
growisofs cdrskin xorriso cdrecord wodim
...
What DVD burn program to use ? ( growisofs cdrskin xorriso cdrecord wodim )
current setting  : "xorriso"
Your input please:
xorriso
----------------------------------------------------- devices
...
Address of DVD recorder ? ( /dev/sr0 /dev/sr1 /dev/hda /dev/hdc )
current setting  : ""
Your input please:
/dev/sr0
Note: Speed is an upper limit. The drive will eventually curb it.
By default the burn program will do fast DVD-RAM and BD
recording (option stream_recording=on).
Option stream_recording=off causes full slow checkreading.
Speed ? (e.g. any , 1)
current setting  : ""
Your input please:
16
Note: Erasing DVDs is needed for rewritable DVD-RW media only
Automatically erase DVD before writing ? (yj1/n0)
current setting  : ""
Your input please:
y
Note: Files of size 2GB or larger are likely to cause trouble when
read from an ISO filesystem. A directory outside the backup area
may be used to buffer pieces of such files which then become
part of the backup. The directory has to be empty now or has
already to contain a file scdbackup_split_file_directory_mark.
Its filesystem has to be able to carry a full media content
(4.7GB). Take care not to use the same directory as with CD !
If you enter "-" then large files are excluded. If you enter
"-allow_large_files" then files just have to fit on the media.
Your local xorriso is suitable for "-xorriso_cut_out"
which needs no extra disk space.
Large file split directory ? ( "-" = exclude large files)
current setting  : ""
Your input please:
-xorriso_cut_out
Note: Compressed afio (system) backups may produce arbitrarily sparse
output which is quite demanding to the writer hardware.
Therefore compression usually is disabled without buffer.
On the other hand, the buffer is as large as the DVD (~ 4.7 GB)
and must be outside the backup area or explicitely excluded.
Take care not to use the same file as with CD !
Special name "-" disables buffering. "-afio_compress" does the
same but enables afio compression (works fine with my DVD+RW).
Buffer file name ? ( or "-" or "-afio_compress" )
current setting  : ""
Your input please:
-afio_compress
Note: if you enter 0 then the directory size is determined
by a very slow method which previews the compressed
afio sizes.
Estimated capacity for a compressed afio DVD ? (eg. 8000m)
current setting  : ""
Your input please:
8000m
Exclusion list of directories for system backup ?
current setting  : "/cdrom /cdrw /floppy /home /mnt /opt /proc /tmp"
Your input please:
Normal users just press Enter here. This input is only of interest
for the superuser.
Please note that -if ever- a single line is expected here, despite your
browser right now may show more than one.
Now the parameters get written to their files.
In future runs of CONFIGURE_DVD you may confirm your settings by empty input
lines and only have to enter those values which you want to change.
The content of those files may be temporarily overriden by shell variables.
See README Appendix CONFIGURATION for a definition of the files' content (if it
isn't obvious by name and sequence :)
- writing file scdbackup_use_genisoimage_value
- done
...
- scdbackup for DVD 0.9.2 : First stage of installation done.
- --------------------------------------------------------
- The commands of your scdbackup installation are now available in :
- /home/thomas/scdbackup/scdbackup-0.9.2/cmd
-
- You may want to add this address to your PATH variable
- or ask your superuser to run ./SUPERUSER_CONFIGURE in directory
- /home/thomas/scdbackup/scdbackup-0.9.2/inst
- --------------------------------------------------------
A first test :
- $ $HOME/scdbackup/scdbackup-0.9.2/cmd/scdbackup_home -version
- scdbackup 0.9.2 2009.12.13.072542
The user could now extend the variable PATH in order to enable the shell
to find the commands of scdbackup :
- $ PATH="$PATH":$HOME/scdbackup/scdbackup-0.9.2/cmd
This will only help with that single shell session. So the user now has to
find a place in the shell's startup scripts where to make this enhancement
permanently.
If there was already scdbackup-0.8 or older installed, then the superuser
will have to remove the old public commands.
As an alternative for both (PATH and command removal), the superuser may
install the commands in a public command directory where they can be found
without such precautions. In case of an upgrade installation this has
to be the same directory as before, of course.
So, i performStep 3 of the installation procedure.
- $ su
- Password:
- # ./SUPERUSER_CONFIGURE
-
- scdbackup command installation script 0.9.2
-
- This script may install following commands owned by the superuser
- in a public command directory of your choice :
- scdbackup access: a+rx,u+w,go-w
- scdbackup_home access: a+rx,u+w,go-w
- scdbackup_afio access: a+rx,u+w,go-w
- scdbackup_verify access: a+rx,u+w,go-w
- scdbackup_blank access: a+rx,u+w,go-w
- scdbackup_askme access: a+rx,u+w,go-w
- scdbackup_clear access: a+rx,u+w,go-w
- scdbackup_sys access: u+rwx,go+r,go-wx
- For use with the DVD writer it installs accordant commands: sdvdbackup*
- like: sdvdbackup , sdvdbackup_home , sdvdbackup_verify , ...
-
- Public command directory ? (e.g. /usr/bin , "-" = do not install)
- current setting: "/usr/local/bin"
- /usr/local/bin
- confirmed
... the installed commands and their permissions get shown ...
- The superuser should have an own scdbackup installation. To get one
- follow steps 1 and 2 of README "Installation".
- But instead of step 1b (./FETCH_CONFIGURATION) do :
- ./SUPERUSER_FETCH_CONF /home/thomas/scdbackup/scdbackup-0.9.2
- Do not forget step 2c : ./ADD_USER
-
Since i want to run some backups as superuser, i will follow this advice.
I read README paragraph "Superuser Planning" to decide
what to put into the system backup and what therfore has to be excluded.
For the sake of security and because i cannot guarantee for all the
software which is run by myself as a desktop user, i now will make a
designated installation for the superuser. It is a bit more inconvenient
than to simply perform ./ADD_USER now. But it will help to keep
up the system security in the same quality as it was before scdbackup's
installation.
-
- # mkdir $HOME/scdbackup
- # cd $HOME/scdbackup
- # tar xzf ..../scdbackup-0.9.2.tar.gz
- # cd scdbackup-0.9.2/inst
- # chown root ../../scdbackup-0.9.2/*/* ../../scdbackup-0.9.2/* ../../scdbackup-0.9.2
- # ./SUPERUSER_FETCH_CONF /home/thomas/scdbackup/scdbackup-0.9.2
- Fetching files ...
- scdbackup_use_genisoimage_value :
- xorriso
- ...
- done
- # ./CONFIGURE_CD
No fear, you will mainly have to press the Enter key and confirm the offered
settings.
Only with the questions for "Large file split directory" and
"Buffer file name" you will have to invent new directory and file addresses,
where you used some with the desktop user's installation. Do not use the
same addresses as other users. This would be a security hole.
Make sure the superuser's disk buffer objects are properly protected by
restrictive access permissions. Best is to have no such buffers.
This time you also will have to make an appropriate input to question
"Exclusion list of directories for system backup ?". The README paragraph
"Superuser Planning" gives the necessary information about what addresses
belong into this list (everything that's volatile or owned by normal users).
- ...
- # ./CONFIGURE_DVD
- ...
- # ./ADD_USER
- publishing directory address : /root/scdbackup/scdbackup-0.9.2 in /root/.scdbackup_dir
- done
- # exit
And again a test :
- $ scdbackup_home -version
- scdbackup 0.9.2 2009.12.13.072542
If this yields a result different from the first test
then the user has to find the obtrusive old command's location by help of
which scdbackup_home. The superuser then will have to
take appropriate action.
I advise against running old versions of scdbackup for a prolonged time.
Since i do not use the cdrecord_prog which comes with scdbackup,
there is no need to run ./SUPERUSER_CONFIGURE again. The public
commands will stay valid until a user's ./CONFIGURE_* run
with a future scdbackup version might complain about them being outdated.
Note that other users may now install scdbackup on their own - provided
they got read access to scdbackup-0.9.2.tar.gz . Nevertheless that will
not give them any more permissons than they would have had using the
system installed DVD burn programs directly.
to top of page |
to scdbackup homepage
Backup run with scdbackup_home
The installation of this very antique example was made in /home/test .
Usage of scdbackup_home has not changed since then.
The user's actions (1 command, 5 Enter keys) are shown in bold typeface
- $ scdbackup_home
The disk is rattling for about five minutes now.
My box was a K6/350 and below my HOME directory were about 3 GB.
From time to time one of these information lines appears:
- ... splitting .
- ... splitting ./p
- ... splitting ./p/sorten
- ########### 1 (9 items 638.8m bytes)
- ########### 2 (16 items 634.3m bytes)
- ########### 3 (10 items 638.6m bytes)
- ########### 4 (335 items 579.3m bytes)
- ... splitting ./.netscape
- ########### 5 (4 items 411.1m bytes)
- ########## part 1 of 5 (9 items 638.8m bytes)
- insert CD and press Enter
The distribution plan for five CDs is made and the result script took over control now.
This is the time to insert a CD-R and press the Enter key.
The following messages come from mkisofs and cdrecord:
- Total extents scheduled to be written = 326878
- 1.53% done, estimate finish Sat Nov 27 22:37:54 1999
- ...
- 99.43% done, estimate finish Sat Nov 27 22:28:10 1999
- ...
- 326878 extents written (638 Mb)
- Cdrecord release 1.6 Copyright (C) 1995-1998 Jörg Schilling
- ...
- /home/test/scdbackup-0.5/cdrecord_prog: fifo was 0 times empty and 20071 times full, min fill was 96%.
- Fixating time: 62.121s
The first CD is done, gets ejected and scdbackup asks for the next one:
- ########## part 2 of 5 (16 items 634.3m bytes)
- insert CD and press Enter
This is again the time to insert a CD-R and press the Enter key.
- ...
- ########## part 3 of 5 (10 items 638.6m bytes)
- insert CD and press Enter
Press Enter
- ...
- ########## part 4 of 5 (335 items 579.3m bytes)
- insert CD and press Enter
Press Enter
- ...
- ########## part 5 of 5 (4 items 411.1m bytes)
- insert CD and press Enter
Press Enter
- $
It is done. Five CDs. I watched two movies in TV meanwhile and changed CDs every half hour. (Speed factor 4)
I label my CD-RWs at first use with a unique id number. After each use
i write a leaflet with this id number, the date , the format ("ISO" or "afio"),
the content ("system", "music collection", whatever). Very important is the
volume number and the total number of volumes ("1 of 5", "2 of 5" ...).
With incremental backups i additionaly record level number and date of
the previous level.
Leaflet and CD are stored together (most time).
As soon as a CD-RW gets blanked i destroy the old leaflet to avoid any
confusion.
to scdbackup homepage
to top of page
Resuming an interrupted backup run
After a backup run has been interrupted, it can be resumed without distributing
the data over the CDs again. This guarantees that the new CDs match the old
ones. Care should be taken that not too many new files have been created
meanwhile. Otherwise particular CDs might overflow.
If you expect an interupted backup then you should decrease the size of
the pieces in order to get more buffer space for subsequent size increases.
Use -max_size 600m to reserve 50 MB on each CD (with 650 MB
capacity).
If you want to interrupt a lengthy backup session then kill the script
when it asks for the next CD (e.g. by pressing CTRL+C rather than Enter).
The interrupted backup can be resumed until you start a new backup.
Let us assume that the run was interrupted before or during the forth piece:
- $ scdbackup_home -resume
- ...
- ########## part 4 of 5 (335 items 579.3m bytes)
- insert CD and press Enter
Press Enter
- ...
- ########## part 5 of 5 (4 items 411.1m bytes)
- insert CD and press Enter
Press Enter
- ...
- $
It is done again. Two missing CDs have been created.
If a certain volume shall be done it can be addressed by an optional
volume number like in :
- $ scdbackup_home -resume 2
to scdbackup homepage
to top of page
Verifying a result CD
Some CD recorders and some CD media are not as reliable as they should be.
It may be necessary to verify the readabilty and correctness of the backup
CDs.
Be aware that this check does not ensure the completeness of your
backup. Any faults of scdbackup or yourself will not be detected. Only the
success of your CD recorder's efforts, the individual CD and your CD reading
device will be tested.
So you should look at the total backup size wether it is plausible. After
verifying the single CDs you should also use ASKME to locate some files and
check wether they really are on their CD.
The most convenient way to verify a CD is to put it into the tray of /dev/cdrom
and to execute :
- $ scdbackup_verify -auto
- scdbackup_verify 0.8.8 en
- Loaded 994 checksum records from /home/thomas/scdbackup/scdbackup-0.8.8/logs/backup_log
- / 103.5m bytes
-
- Found matching record :
- 1_1 A40904.234957 109447168 695bc386687f9594e2885f87e9983c68
-
- Note: Checksum tag found by scanning confirms match.
-
- Note: read 340464 surplus bytes
-
- OK. Matching Record found . Passender Record gefunden .
-
- 104.4m bytes
So this CD is part 1 of 1, written 2004 September 4 23:49:57,
with 109447168 bytes.
The note about the scanned checksum tag means that not only the matching
list record was found and verified but also an identical checksum tag
on the backup media. It would have been sufficient to verify the media
if there had been no matching record in the list file.
The note about surplus bytes is typical for CD reading devices. To avoid
problems, scdbackup appends a generous amount of padding data when writing.
An amount of more than 400000 with a CD would be quite suspicious, though.
In case of failed verification there would be no "OK" but :
- / 103.5m bytes
-
- -search_md5 : NO MATCHING RECORD FOUND. KEIN PASSENDER RECORD GEFUNDEN :
- - - -
-
- 104.4m bytes
In such a case you should hurry to burn a new CD #1 by
scdbackup -resume 1 .
Do this as long as the data from the disk still fit on the CD.
If you want to verify at a device other than /dev/cdrom, give that
address as first argument :
- $ scdbackup_verify /dev/cdrecorder -auto
With a previously used rewritable DVD (and command
sdvdbackup_verify
) one may get a lot of surplus data
which are the remainder of previous media contents.
This may be annoying. So one may use option -auto_end which stops
reading after the first match. Check well what record has been found and
better rerun with -auto if size or timestamp are not as expected.
You may also compare the CD against the most recent checksum record in the
list. This will not complain about surplus bytes and eliminates the very small
risk to incidentially match a faulty CD with an older record.
- $ scdbackup_verify
-
- Comparing with record :
- 1_1 A30110.222002 260765696 c30d13d53977cecb87017896d72616ae
-
- OK. Record and data input match. Daten passen zum Record.
-
- 248.7m bytes
In case of failure there would be no "OK" but for example :
- SIZE MISMATCH. FALSCHE GROESSE :
- record=260796416 stdin=255420416
or
- MD5 CHECKSUM MISMATCH. FALSCHE MD5 PRUEFSUMME :
- record= c30d13d53977cecb87017896d72616ae
- stdin = c7a17c2d8e7de65e82756527fe498c75
Note that normally you have to be the same user who burned the media
in order to have access to the recorded checksums of that user.
With option -auto and a backup written by scdbackup-0.8.4 or
newer, you will not need those records.
If you want to verify at /dev/cdrecorder the first volume of the most recent
three volume backup then execute
- $ scdbackup_verify /dev/cdrecorder 1_3
More options of scdbackup_verify are explained by
- $ scdbackup_verify -help
(or in file scdbackup_verify_help )
to scdbackup homepage
to top of page
Searching and restoring a particular file or directory
From time to time, i spoil some of my files. Linux is quite safe, i am not.
For example, i lost the file $HOME/gifs/milleniumbug.gif
But i have got my backup !
First i mount one of the CDs (it doesn't matter which one)
- $ mount /cdrom
Since i have got the "exec" option in the /cdrom line of /etc/fstab, i can execute the ASKME script directly from CD.
- $ /cdrom/ASKME gifs/milleniumbug.gif
- 5 gifs gifs/milleniumbug.gif
From this answer i learn that the whole directory "gifs" is on CD number 5.
Now i have to mount this CD and can restore my lost file.
- $ umount /cdrom
I insert CD number 5 and execute
- $ mount /cdrom
- $ cp /cdrom/gifs/milleniumbug.gif $HOME/gifs
and the milleniumbug (75 K GIF movie) is back on my disk.
For some more info about ASKME see ASKME -help
For locating files within incremental backups you will either have to use
scdbackup option -info_script -full at backup time or rely on the
much more suitable command scdbackup_askme.
to scdbackup homepage
to top of page
Erasing backup media with confidential data
Backups often contain data which in no case shall be read by others. On a CD
anybody can be superuser and read anything. Therefore it is desirable to make
the stored data permanently unreadable before a media is reused or disposed.
Rewritable media with confidential data should be overwritten before they
are refilled with new data and possibly handed to other people.
This is important especially with DVD because remainders of older backups
can be read from the media without using any special means. With CD-RW
this is not so easy but still not impossible.
The following command overwrites a DVD+RW with the amount of 4.7 billion bytes
which is usually advertised on the DVD's label.
- $ sdvdbackup_blank shred 4.7e9
- sdvdbackup_blank 0.8.8 en
-
- This program blanks a DVD-RW to make it ready for re-use.
- After blanking, the data currently stored on the media will be unreadable.
- Blanking DVD+RW makes only sense together with option "shred".
- Note: DVD-RW media formatted in mode "Restricted Overwrite"
- and DVD+RW media are rewriteable without blanking.
-
- Beware: Blanking other media may produce any kind of damage.
-
- insert media and press Enter
Press the Enter key.
- evaluating media type ...
- Detected media type : DVD+RW
-
- Note: Blanking/formatting disabled because of media type DVD+RW
- Note: Triggered by option "shred" the following will happen :
- The media will get filled with 4.7e9 bytes of random
- data in order to erase old content. You may verify these
- data by sdvdbackup_verify.
- The media will be overwriteable afterwards.
-
- press Enter to continue
Press the Enter key again.
- beginning to write random data
The usual messages of the DVD burn program follow.
After the writing is done, it is possible to check the result with command
sdvdbackup_verify . This is a quality check for the media, too.
With CD-RW the capacity is advertised in binary MB (= 1024*1024) .
So for the usual 700 MB media use :
- $ scdbackup_blank shred 700m
- scdbackup_blank 0.8.8 en
-
- This program blanks a CD-RW to make it ready for re-use.
- After blanking, the data currently stored on the media will be unreadable.
-
- Beware: Blanking other media may produce any kind of damage.
-
- insert media and press Enter
Press the Enter key.
- evaluating media type ...
- Detected media type : CD-RW
-
- Note: Triggered by option "shred" the following will happen :
- The media will get blanked by cdrecord option "blank=all".
- The media will get filled with 700m bytes of random
- data in order to erase old content. You may verify these
- data by scdbackup_verify.
- You will need another blanking run without "shred"
- to make the media writable again.
- press Enter to continue
Press the Enter key again.
- beginning to blank media ...
- ...
- beginning to write random data
You will see the usual messages of cdrecord blanking and writing the media.
After repeated treatment with these commands even first class equipment should
have difficulties to read the original content.
With non rewritable media it is best to destroy them physically. Be aware
that sophisticated precautions are made on the media for reconstruction of
damaged data. Therefore i burn the recording layer and soften the media with
a gas lighter, bend the disks and break them into pieces when cold again.
(CAUTION : Hazard to burn or cut your fingers.)
to scdbackup homepage
to top of page
Incremental backups
If you got a substantial amount of data, it becomes annoying to backup the old
data with each new run. To avoid this (within certain limitations) one may
use an incremental backup scheme.
Be aware, though, that incremental backups increase complexity when it comes
to restoring a whole backup area. So only decide to make incremental backups
after you experience that full backups are unbearable.
(See example below)
For an incremental backup a configuration needs to be created. We simply
use the $HOME backup but one could also record an arbitrarily complicated
set of data sources and options. After the configuration is created, only
its name and the desired update level have to be given with the scdbackup
command.
- $ scdbackup_home \
- -conf_dir $HOME/home_backup \
- -level -create_configuration
This creates the directory $HOME/home_backup
(and fails if it already exists),
records the parameters of the backup (which are set within script
scdbackup_home) and enables recording of the state
of the backup area (i.e your $HOME with its sub directory trees).
Also it starts the initial base backup which is called -level 0.
This backup is like any other run of scdbackup_home (see above)
besides it may take quite some time to obtain CRC fingerprints from all
the files in your $HOME tree. Also it eats some of your disk space by
writing file
$HOME/home_backup/level_0/content_file_list.gz
The difference comes with the update run a few days later:
- $ scdbackup \
- -conf_dir $HOME/home_backup \
- -level 1
This command causes the backup configuration
$HOME/home_backup to be read
and updated on level 1. Level 1 backups all files that were changed or newly
created since the last run of level 0. By comparing the current state with
the one recorded from level 0, it also detects files which have vanished or
got moved.
So from now on, you do base backups with -level 0. These base
backups include all files without any time constraint. Updates of the base
backups are done by -level 1 in regular intervals.
You may also do short term updates which refer to the last run of level 1
by running a -level 2 backup.
There is no upper limit of level numbers but you may only increase them by
one with each run. That is, after a level 0 run was done, you cannot do
a level 2 run but only level 1. The special level -next can be
used to get the highest permittable level.
A backup scheme which from monday to saturday records only the
daily changes and makes a full backup each sunday:
Monday : -conf_dir $HOME/home_backup -level -next
...
Saturday: -conf_dir $HOME/home_backup -level -next
Sunday : -conf_dir $HOME/home_backup -level 0
Now the media of monday to saturday may be recycled.
A scheme which makes quarterly (13 weeks) full backups, four-weekly
and weekly updates (-conf_dir same as above). This scheme lets you
retrieve old file versions for quite a long time.
Week 1 : -level 1
Week 2 : -level 2
Week 3 : -level 2
Week 4 : -level 2
Week 5 : -level 1
Week 6 : -level 2
Week 7 : -level 2
Week 8 : -level 2
Week 9 : -level 1
Week 10 : -level 2
Week 11 : -level 2
Week 12 : -level 2
Week 13 : -level 0
For week 14 hop to week 1.
It is advised to keep levels 0 and 1 for a longer time.
To combine both schemes shown, simply replace the sunday run of
scheme 1 by the weekly run of scheme 2.
If you want to change the configuration parameters, you may either run
the altered command with special level -update_configuration:
- $ scdbackup_home \
- -not trashdir newsdir \
- -conf_dir $HOME/home_backup \
- -update_configuration
or you may directly edit the parameter list file
$HOME/home_backup/specific_arguments
In this file, each single argument is recorded in a single line. The list
begins with arguments automatically set by scdbackup and ends with arguments
given by you (if any).
Be aware what the options mean before you delete or change them. Also
perform a new -level 0 backup afterwards.
Incremental backups include the ASKME scripts of lower levels. If you use
the ASKME script of level 2 it will try to execute the ASKMEs of level 1
and level 0 if it cannot find a file by itself. The option -levelall
causes all levels to be searched in any case.
Since a large multilayer backup may produce very large and slow ASKME scripts
there is a specialized binary program which can extract the information from
the scripts and can perform searches much faster. (See
scdbackup_askme_help
)
These features are not restricted to scdbackup_home but may also
be used with scdbackup and scdbackup_afio. One may
have an abritrary number of configurations in distinct directories.
Now for some restrictions:
-
If you cannot afford the disk space and additional computation time for
the recording of the backup area state, then you may disable this feature
by using option -disable_safe_leveling together with
-level -create_configuration.
But be aware that file movements and deletions cannot be detected that way
and will therefore not be included in the higher backup levels. Such a
backup is still a good source for recovering from limited data accidents
but it may be unable to accurately reconstruct the most recent state of
your backup area.
-
Because of the restricted number of fileaddresses possible with older
versions of mkisof there may be whole directories included in the backup
despite they contain some older files.
Therefore, ./CONFIGURE_* advises you to use locally installed cdrtools
if their mkisofs mentions "-path-list" in its help text.
If you agree, you will get a very high -max_items limit
and should have no problem with above issue.
-
If quick and simple restore is essential, then you should refrain from
incremental backups as long as this does not prevent you from making
backups in a sufficient frequency.
Consider to purchase a fast DVD recorder or to divide your data into
a small frequent and a larger unfrequent backup (take care that both
backups do not share any data content).
to scdbackup homepage
to top of page
Locating files in a multilevel backup and restoring them
Usually the ASKME scripts of an incremental backup can
just tell their date, level and volume. Although they can at backup time
be enabled for searching by option -info_script -full they are
of very limited use when it comes to locating files in the heap of backup
levels and volumes.
Much faster than ASKME is the command scdbackup_askme which uses a
binary program to load and explore the content model of an incremental
backup configuration. (It can also load ASKME scripts but in that case its
search capabilities are nearly as limited as the ones of the script.)
The content model can be read from the -conf_dir directory of backup or it
can be read from directory /added_by_scdbackup of the last volumes of the
available backup levels.
Beginning with scdbackup version 0.8.7, non-incremental ISO-9660 backups,
too, have such a content list on the last volume.
Queries can be submitted either as program arguments or in dialog mode which
usually offers (via libreadline) an editable input line with history.
Since the load process lasts some time the dialog mode is to prefer if
one does not have a clearly predefined set of queries.
- $ scdbackup_askme $HOME/home_backup -dialog -page:20
- scdbackup_askme 0.8.8 en
- reading /home/thomas/home_backup/level_0/content_file_list.gz
- 484262 lines read
- reading /home/thomas/home_backup/level_0/complete_dir_list.gz
- 2136 lines read
- reading /home/thomas/home_backup/level_1/vanished_file_list.gz
- 12224 lines read
- reading /home/thomas/home_backup/level_1/content_file_list.gz
- 61481 lines read
- reading /home/thomas/home_backup/level_1/complete_dir_list.gz
- 2254 lines read
- reading /home/thomas/home_backup/level_2/vanished_file_list.gz
- 10461 lines read
- reading /home/thomas/home_backup/level_2/content_file_list.gz
- 18790 lines read
- reading /home/thomas/home_backup/level_2/complete_dir_list.gz
- 2366 lines read
- sorting directories of file tree model
- sorting done
- sorting directories of file target model
- sorting done
The program starts up in mode -search_sh. Search patterns will work similar
to the parser of the shell: one may use wildcards *,?,[] which cannot match
character '/', nevertheless.
Search patterns which begin with '/' describe absolute addresses. Other
patterns get prepended the working directory, which was current at the
time of -level -create_configuration.
Without further precautions, search is done on the source addresses.
- =============================
- enter option or search text :
- -pwd
- ==============================================================
- current working directory:
- /home/thomas/
- =============================
- enter option or search text :
- gifs/milleniumbug.gif
- ==============================================================
- +++ on backup level 0
- 7 /gifs/milleniumbug.gif = /home/thomas/gifs/milleniumbug.gif
- ============================= 1 matches in 0.003 seconds
- =============================
- enter option or search text :
- gifs
- ==============================================================
- +++ on backup level 2
- 1 /gifs/ = /home/thomas/gifs
- +++ on backup level 1
- 3 /gifs/ = /home/thomas/gifs
- +++ on backup level 0
- 7 /gifs/ = /home/thomas/gifs
- ============================= 3 matches in 0.001 seconds
- =============================
- enter option or search text :
- gifs/*
- ==============================================================
- +++ on backup level 2
- 1 /gifs/best_before.gif = /home/thomas/gifs/best_before.gif
- ... further matches completing a 20 line page ...
-
-
.. [Press Enter to continue. @,Enter avoids further stops. @@@ aborts] ..
- @
- ...
- +++ on backup level 1
- ...
- +++ on backup level 0
- ...
- ============================= 433 matches in 0.041 seconds
Let us assume we got no idea where our millenium thingee might have been
stored. We only have a dim memory about its name.
Mode -search_shname will search on the single filenames only and ignore
the preceeding paths.
- =============================
- enter option or search text :
- -search_shname
- ==============================================================
- =============================
- enter option or search text :
- *millenium*
- +++ on backup level 2
- 3 /gifs/millenium.gif = /home/thomas/gifs/millenium.gif
- +++ on backup level 0
- 7 /gifs/milleniumbug.gif = /home/thomas/gifs/milleniumbug.gif
- 10 /misc/milleniumparty/ = /home/thomas/misc/milleniumparty
- 11 /photo/happy_millenium.jpg = /home/thomas/photo/happy_millenium.jpg
- ============================= 3 matches in 2.324 seconds
(Run times are of a 32 bit AMD 2600+ working on a tree of 540,000 files.)
A description of the search rules and a complete list of options are provided
by
- =============================
- enter option or search text :
- -help
- ==============================================================
- usage: scdbackup_askme infosource [options|searchtexts]
- ... further helptext...
It is possible to use scdbackup_askme for restoring data from ISO-9660
backups.
This offers the advantage that only those files are restored from older levels
which are still valid. It is also more convenient and less error prone than
restoring data by help of shell commands.
With long lasting restore operations, the user guidance and the ability to
resume interrupted restore runs come in handy.
Initially the ability to restore is disabled in order to avoid inadverted
mishaps. By option -restore_mode it can be enabled and restore rules can
be set. It is also necessary to tell the mount point of the backup media.
- =============================
- enter option or search text :
-
-media_mode:load_eject:mount_umount
- =============================
- enter option or search text :
-
-restore_mode:overwrite_off:on_error_ignore
- =============================
- enter option or search text :
- -restore_from:/cdrom
The following two options replace in the original addresses the start text
/home/thomas by /home/test/thomas. After all it is only a test and by
overwrite_off existing files are protected against overwriting.
Directory /home/test/thomas does already exist and and is owned by the user
who runs this session.
(Have your superuser create such a playground and chown it to you, or use
as -restore_to: an empty directory within your own realm on the
disk.)
- =============================
- enter option or search text :
- -restore_skip:/home/thomas
- =============================
- enter option or search text :
- -restore_to:/home/test/thomas
Now extract /home/thomas/gifs from the levels. Access permissions and
adjustable timestamps will be restored, too.
- =============================
- enter option or search text :
- -restore:/home/thomas/gifs
- ==============================================================
-
############################################################################
-
################## Please insert: LEVEL 0 VOLUME 7 #################
-
############################################################################
-
.................. [ Press enter when done. @@@ = Abort ] ..................
Enter-key ... files get copied ...
-
############################################################################
-
################# Please insert: LEVEL 1 VOLUME 3 ##################
-
############################################################################
-
.................. [ Press enter when done. @@@ = Abort ] ..................
Enter-key ... files get copied ...
-
############################################################################
-
################# Please insert: LEVEL 2 VOLUME 1 ##################
-
############################################################################
-
.................. [ Press enter when done. @@@ = Abort ] ..................
Enter-key ... files get copied ...
-
Restored:
732 file objects,
4152k data bytes
-
===============================
3 matches in 57.843 seconds
The quite long run time is due to three mount operations which with DVD
may well last 10 to 15 seconds each.
Because -page:20 is still active, a prompt appears after a
few files are copied and reported. It is ok to enter @
in order to disable this paging temporarily.
One may also disable it permanently by -page:0.
to scdbackup homepage
to top of page
Full restore of a multilevel backup
There are moments when a backup area needs to be restored from scratch.
Possibly on a completely new computer system.
In complicated disk situations you will have to prepare the mount tree below
$HOME in advance. It is up to you to provide the disk space necessary to take
the backup's content.
If it is possible to use scdbackup_askme then the restore
can be done quite conveniently. If not, then one has to follow the somewhat
more cumbersome path of
restoring by hand using shell commands only.
The previous paragraph demonstrated how to load a
backup configuration from a healthy disk and use it for search and restore.
For this example it is assumed that this configuration directory is among
the victims of the data accident. Nothing is left but the set of backup media.
The operating system and scdbackup are assumed operational, nevertheless.
- $ scdbackup_askme /cdrom/added_by_scdbackup \
-
-dialog \
-
-logfile:I:/tmp/scdbackup_restore_log \
-
-media_mode:load_eject:mount_umount \
-
-restore_mode:overwrite_off:on_error_skip \
-
-restore_from:/cdrom
The backup of /home/thomas consists of Level 0: 14 DVDs,
Level 1: 3 DVDs und Level 2: 1 DVD .
At first the model of the backup area has to be read from the backup media.
The necessary files are on the last volume of each level.
The program prompts:
-
#####################################################################
-
############# Please insert: LEVEL 0 last VOLUME ##############
-
#####################################################################
-
.......... [ Press enter when done. @@@ = All levels done ] .........
Insert DVD , Enter-key.
- reading /cdrom/added_by_scdbackup/content_file_list.gz
- 484262 lines read
reading /cdrom/added_by_scdbackup/complete_dir_list.gz
2136 lines read
#####################################################################
############# Please insert: LEVEL 1 last VOLUME ##############
#####################################################################
.......... [ Press enter when done. @@@ = All levels done ] .........
I do this with the last DVDs of level 1 and 2 until finally i get
asked for the non-existing level 3.
-
#####################################################################
-
############# Please insert: LEVEL 3 last VOLUME ##############
-
#####################################################################
-
.......... [ Press enter when done. @@@ = All levels done ] .........
- @@@
-
... [@@@ = request to end registered. Really done reading ? (y/n) ] ..
- j
-
--- ( request to end confirmed )
- sorting directories of file tree model
- sorting done
- sorting directories of file target model
- sorting done
Now the model is read in and can be exploited.
My restore wish is a very general one:
- =============================
- enter option or search text :
- -restore:/home/thomas
- ==============================================================
-
#####################################################################
-
############### Please insert: LEVEL 0 VOLUME 1 ###############
-
#####################################################################
-
.............. [ Press enter when done. @@@ = Abort ] ...............
18 Enter-keys and six and a half hours later:
-
Restored:
560464 file objects,
65.5g data bytes
-
===============================
18 matches in 20661.667 seconds
The plain restore time is 5.7 hours. About 45 minutes in total the program
was waiting for me to put in the next volume.
After everything went well one may delete log file
/tmp/scdbackup_restore_log.
scdbackup_askme will neither restore files below its own installation
directory nor files below the eventually redirected directories conf, tmp,
or logs. An attempt will cause messages but is not considered to be an error.
Resuming an interrupted restore run
With runs that last several hours one has to expect interruptions.
While restoring files the program is tracking the current search position
within the scdbackup tmp directory. At the start of each restore
run the complete setup status of the program is recorded.
To resume a restore run after the program ended more or less neatly it
gets restarted with the pseudo information source "-resume".
- $ scdbackup_askme -resume
The program will start the -restore run and eventually prompt for media.
If it is not desired to resume at the exact position where the last restore
run stopped, one may add the level number and also a volume number:
- $ scdbackup_askme -resume:1
- $ scdbackup_askme -resume:1:4
The usual abort handling strives for leaving no
incompletely restored file objects and to eventually reinstate the
properties of existing directories in the path of the aborted
file object.
So -resume usually can start restoring seamlessly.
If the previous restore run was interrupted very roughly or if abort handling
was not able (or allowed) to clean up an incomplete restore, then it is
possible that the affected file object on disk is in a questionable condition.
It will then be
recorded in the status files as "touched but not completed" unless the
operating system failed to perform those status writes (e.g. because of a
power failure).
If such an indication of an incomplete file restore is found then the
program warns:
-
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
-
!! WARNING: The last run left a file object in QUESTIONABLE state !!
-
!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
-
The address of the questionable file object is:
-
/home/test/thomas/gifs/hastenichgesehn.gif
Some hints about potential problems and their potential solutions follow.
If a dialog session is resumed then a prompt appears:
-
If everything is fixed, enter text "go on"
Fixing might mean to remove the file or to rectify directory permissions.
One has to examine the particular situation.
-
go on
Restoring usually resumes with the file that was reported as questionable.
Nevertheless one should check this file after the restore is done and
eventually restore it as single file once again.
When resuming a run which had been performed in non-dialog mode, a questionable
file causes abort in any case. The user gets gets told to resolve any problem
and then to empty the file which caused the warning warning by this shell
command:
echo > $(scdbackup -where tmp)/askme_resume_state_"$UID"_dirty
Restoring by hand using shell commands only
This example describes how
to restore a big $HOME tree by using a bash shell with standard commands
and programs.
-
Get a start directory outside your $HOME :
This example assumes a $HOME entirely on the same filesystem as its parent
and enough room on that filesystem to take the whole backup. So we will
rebuild $HOME below "$HOME"_RESTORED and later move it to its destination.
Have your superuser prepare a directory for you
- $ TARGET_ROOT="$HOME"_RESTORED
- $ echo "Dear superuser, please make me directory $TARGET_ROOT"
Your superuser then is so kind to mkdir the desired
directory and to chown it for you. So you now can begin
to create own files.
- $ TARGET="$TARGET_ROOT"/$(basename "$HOME")
- $ mkdir "$TARGET"
- $ export SCDBACKUP_RESTORE_ROOT="$TARGET"
Variable SCDBACKUP_RESTORE_ROOT is needed for the vanish scripts of levels
above 0.
-
Level 0:
Copy level 0 to your disk. With a CD set from the above example that would
be for each CD :
- $ mount /cdrom
- $ cp -a /cdrom/* "$TARGET"
- $ umount /cdrom
It is necessary to make all directories fully accessible because there may
be the need to create or delete files during restore of the next level.
We will restore the original directory permissions at the end of this
example.
- $ find "$TARGET" -type d -exec chmod 0755 '{}' ';'
In cases of extreme self denial of access rights you may have to do this
after each single CD. Watch out for complaints of the cp command.
-
Level 1:
Execute the vanish script of level 1. This has to be done before copying
the backup content in case file objects have changed their types between
level 0 and level 1. Such files might be impossible to overwrite by the
copy command and thus have to be removed by the script.
Insert the last CD of level 1 and execute :
- $ mount /cdrom
- $ export SCDBACKUP_RESTORE_PREP_LEVEL=1
- $ gunzip </cdrom/added_by_scdbackup/VANISHED.sh.gz | /bin/sh
- $ umount /cdrom
Variable SCDBACKUP_RESTORE_PREP_LEVEL set to the current level number
prevents an eventual abort which occurs if the vanish script
really has to handle file object type changes.
This is done to avoid unnoticed faulty removals which would occur if the
script is run according to the outdated restore procedure which was prescribed
in the examples of scdbackup previous to 0.8.5.
Now copy level 1 to your disk. For each CD :
- $ mount /cdrom
- $ cp -a /cdrom/* "$TARGET"
- $ umount /cdrom
Again set directory permissions
- $ find "$TARGET" -type d -exec chmod 0755 '{}' ';'
-
Further levels:
Repeat the steps of level 1 for any higher level you got. (Set variable
SCDBACKUP_RESTORE_PREP_LEVEL to the according number.)
-
Finally:
Restore the directory permissions as they were when the last level
was recorded.
- $ gunzip <"$TARGET"/added_by_scdbackup/DIR_PERMS.sh.gz | /bin/sh
Clean up and have your superuser do the move for you. The existing $HOME is
to be moved to a new name and then replaced by the newly built tree. Finally
the start directory gets removed.
- $ rm "$TARGET"/added_by_scdbackup/*
- $ rmdir "$TARGET"/added_by_scdbackup
- $ echo "Dear superuser, please do for me :"
- $ echo " mv $HOME ${HOME}_REPLACED"
- $ echo " mv $TARGET $HOME"
- $ echo " rmdir $TARGET_ROOT"
Dear superuser, be aware of the ill effects if $HOME contains blanks or other
characters meaningful to the shell. If you allowed this, now is the time
to call yourself names.
to top of page
CD mount options
The /cdrom line in my /etc/fstab looks like this
- /dev/cdrom /cdrom iso9660 ro,user,noauto,unhide,exec,nosuid,nodev
Note that there are program interpreters which on their own account
acknowledge the dangerous setuid-permissions and then act accordingly
and irresponsibly. E.g. program suidperl.
If you allow mount option exec you have to make sure that those interpreters
are deactivated.
to top of page |
to scdbackup homepage