Backing Up HDD to External Drive over USB 2.0

Technical issues/questions of an intermediate or advanced nature.
Bogomips
Full of knowledge
Full of knowledge
Posts: 2522
Joined: 25 Jun 2014, 15:21
Distribution: 3.2.2 Cinnamon & KDE5
Location: London

Re: Backing Up HDD to External Drive over USB 2.0

Post#16 by Bogomips » 26 Jan 2017, 23:39

Cloning Internal HDD of 250 GB (< 233GiB) to External Drive Partition

External Disk has Optimal I/O Size: 256MiB-4Kib i.e. 256MiB less one Sector.
Four records of this size would transfer 1GiB-16KiB. Therefore doing Tests and Timings on the Transfer of Four Records. To this end, using dd to read Sectors of 512 bytes from Internal Drive, and second dd instance to write the Optimal Size O/P Record.

There are two ways of achieving this: either through use of a PIPE, or by using a FIFO file:
  • In the Environment

    Code: Select all

    guest@porteus:~$ cat /proc/cmdline
    quiet from=/dev/sda10/tmp/iso/3.2.2/i586/Porteus-CINNAMON-v3.2.2-i586.iso copy2ram ramsize=25% noauto rootcopy=/dev/sda4/cin_rcp noswap nonetwork noload=002;003 3
    
    
    guest@porteus:~$ free -m
                  total        used        free      shared  buff/cache   available
    Mem:            879          14         642         116         222         622
    Swap:             0           0           0
    
    guest@porteus:~$ df -h .
    Filesystem      Size  Used Avail Use% Mounted on
    aufs            220M  4.3M  216M   2% /
    
  • FIFO Explained here: viewtopic.php?f=94&t=4566&p=49705#p49642

    Code: Select all

    root@porteus:/home/guest# time dd of=/dev/sdb2 bs=268431360 iflag=fullblock count=4 < <(dd if=/dev/sda)
    4+0 records in
    4+0 records out
    1073725440 bytes (1.1 GB, 1.0 GiB) copied, 50.4661 s, 21.3 MB/s
    
    real    0m50.485s
    user    0m1.277s
    sys     0m4.411s
    
  • PIPE

    Code: Select all

    root@porteus:/home/guest# time dd if=/dev/sda | dd of=/dev/sdb2 bs=268431360 iflag=fullblock count=4
    4+0 records in
    4+0 records out
    1073725440 bytes (1.1 GB, 1.0 GiB) copied, 50.4014 s, 21.3 MB/s
    
    real    0m50.439s
    user    0m2.304s
    sys     0m8.782s
    
Under normal circumstances with GUI running, times of FIFO are half that of PIPE, but in Real Time slower, 58" to 52". However this seems to even out under the more restricted conditions. That with one subshell taking about half
the time of the two subshell process, but still slower by 0.056 sec.

Estimation of Cloning Time

Time taken to Transfer 1GiB less 16KiB estimate at 50.5 secs. Therefore transfer of 233GiB less 4MiB comes to 233 x 50.5 = 11766.5 secs. This should work out at 3hr 16min 6sec. :roll:

CLONING
  • ENVIRONMENT

    Code: Select all

    guest@porteus:~$ uname -rm
    4.9.0-porteus i686
    guest@porteus:~$ cat /proc/cmdline
    quiet from=/dev/sda10/tmp/iso/3.2.2/i586/Porteus-CINNAMON-v3.2.2-i586.iso copy2ram ramsize=25% noauto rootcopy=/dev/sda4/cin_rcp noswap nonetwork noload=002;003 3
    
    # Non standard /rootcopy dir:
    /dev/sda4/cin_rcp
    
    # Modules activated during boot time:
    /memory/copy2ram/000-kernel.xzm
    /memory/copy2ram/001-core.xzm
    
    ISO=/mnt/sda10/tmp/iso/3.2.2/i586/Porteus-CINNAMON-v3.2.2-i586.iso
    
    guest@porteus:~$ free -m
                  total        used        free      shared  buff/cache   available
    Mem:            879          14         643         116         221         622
    Swap:           644           0         644
    guest@porteus:~$ /sbin/swapon -s
    Filename                                Type            Size    Used    Priority
    /dev/sda11                              partition       130044  0       -1
    /dev/sda7                               partition       530108  0       -2
    
    guest@porteus:~$ sudo /sbin/swapoff /dev/sda11
    guest@porteus:~$ sudo /sbin/swapoff /dev/sda7
    guest@porteus:~$ free -m
                  total        used        free      shared  buff/cache   available
    Mem:            879          14         642         116         222         622
    Swap:             0           0           0
    
    guest@porteus:~$ df -h .
    Filesystem      Size  Used Avail Use% Mounted on
    aufs            220M  4.3M  216M   2% /
    
  • STATISTICS

    Code: Select all

    root@porteus:/home/guest# time dd of=/dev/sdb2 bs=268431360 iflag=fullblock status=progress < <(dd if=/dev/sda conv=noerror,sync)
    249909596160 bytes (250 GB, 233 GiB) copied, 11941.7 s, 20.9 MB/s
    488397168+0 records in
    488397168+0 records out
    250059350016 bytes (250 GB, 233 GiB) copied, 11944.7 s, 20.9 MB/s
    250059350016 bytes (250 GB, 233 GiB) copied, 11949.2 s, 20.9 MB/s
    931+1 records in
    931+1 records out
    250059350016 bytes (250 GB, 233 GiB) copied, 11949.2 s, 20.9 MB/s
    
    real    199m9.261s
    user    5m40.106s
    sys     16m45.068s
    
Actual time was 11949.2 - 11766.5 = 182.7 secs longer than estimated. Works out at 3' 3". Or going by the timer 182.8" longer, so 3' 3" longer than estimated.
Last edited by Bogomips on 27 Jan 2017, 23:37, edited 2 times in total.
Reason: Actual Clone Time
Linux porteus 4.4.0-porteus #3 SMP PREEMPT Sat Jan 23 07:01:55 UTC 2016 i686 AMD Sempron(tm) 140 Processor AuthenticAMD GNU/Linux
NVIDIA Corporation C61 [GeForce 6150SE nForce 430] (rev a2) MemTotal: 901760 kB MemFree: 66752 kB

Bogomips
Full of knowledge
Full of knowledge
Posts: 2522
Joined: 25 Jun 2014, 15:21
Distribution: 3.2.2 Cinnamon & KDE5
Location: London

Re: Backing Up HDD to External Drive over USB 2.0

Post#17 by Bogomips » 14 Feb 2017, 20:11

RSYNC

0. Overview Fixed Format Blocks Represented by <>.

RSYNC Backup
  1. Inode Quantity for Ext4 Partition
      • Number of Files (reiserfs,vfat) and Used Inodes (ext2) on HDD
        • Prolog<>
        • inodes()<>
        • All FSs (sda_infi.txt)<>
        • Epilog<> :shock:
      • Parametrisizing Inode Creation
        • Usage Types<>
        • Inode Numbers<>
      • Requesting 640k Inodes (655340):<>
    • Creating a Directory for Each Relevant HD Partition <>
    • VFAT Partitions RSYNCed
      • sda8<>
        • <>
        • <>
      • sda3<>
        • <>
      • Linux Partitions to RSYNC
        • Arch Way<>
          • <>
          • <>
        • Whittled down rsync Options for Consideration<>
      • Traversing the RSYNC Options
        • Clear Options<>
        • Symbolic Link Options. All Requiring Clarification:<>
        • Test<>Would seem to indicate no Explicit Symbolic Link Options required.
          • Finalised RSYNC Parameter Superset<>
        RSYNC Backup
        1. Inode Quantity for Ext4 Partition
            • Number of Files (reiserfs,vfat) and Used Inodes (ext2) on HDD
              • Prolog

                Code: Select all

                # Recognized devices:
                /dev/sda1: UUID="ca6ce5e1-1bbd-4170-87f7-5fd0448e5f56" TYPE="reiserfs" 
                /dev/sda3: SEC_TYPE="msdos" UUID="4B46-EEEA" TYPE="vfat" 
                /dev/sda4: UUID="966b254d-20de-4376-b736-ddf89150a6b3" TYPE="reiserfs" 
                /dev/sda5: LABEL="kix" UUID="e737efd6-32f4-4a80-bbc1-a6ad3114b4d5" TYPE="ext2" 
                /dev/sda6: UUID="f554b881-2089-4c0f-8de2-70900bdc4e14" TYPE="ext2" 
                /dev/sda7: UUID="0d2c9e21-5e47-4dd3-a279-38551711f3dd" TYPE="swap" 
                /dev/sda8: UUID="AF61-DAE9" TYPE="vfat" 
                /dev/sda9: LABEL="iso" UUID="5bf4078e-3d9c-46e6-8a43-2dfc0eef3aeb" TYPE="ext2" 
                /dev/sda10: LABEL="p_10" UUID="ed3f74cd-26bb-47bd-86b0-5e690663dd55" TYPE="ext2" 
                /dev/sda11: UUID="6a0167f6-5cf7-412d-b85f-0076ca590371" TYPE="swap" 
                
              • inodes()

                Code: Select all

                inodes is a function
                inodes () 
                { 
                    sudo /sbin/tune2fs -l /dev/$1 | awk -F: ' \
                    /^Block count:/ { blocks = $2 } \
                    /^Inode count:/ { inodes = $2 } \
                    /^Block size:/ { block_size = $2 } \
                    /^Free inodes:/ { free_i = $2 } \
                    END { blocks_per_inode = blocks/inodes; \
                          print "blocks per inode:\t", blocks_per_inode, \
                                "\nbytes per inode:\t", blocks_per_inode * block_size, \
                                "\nInode count:\t",inodes, "\nFree:\t\t",free_i,"\t\tUsed:\t",inodes-free_i}'
                }
                
              • All FSs (sda_infi.txt)

                Code: Select all

                guest@porteus:~$ time sudo find /mnt/sda1 -xdev -printf "\n" | wc -l       TYPE="reiserfs"
                311915
                real    1m6.721s
                user    0m0.475s
                sys     0m2.905s
                
                TYPE="vfat"
                guest@porteus:~$ time sudo find /mnt/sda3 -xdev -printf "\n" | wc -l 
                32
                real    0m0.052s
                user    0m0.018s
                sys     0m0.007s
                
                guest@porteus:~$ time sudo find /mnt/sda4 -xdev -printf "\n" | wc -l       TYPE="reiserfs"
                132709
                real    0m18.027s
                user    0m0.225s
                sys     0m1.307s
                
                guest@porteus:~$ inodes sda5                                        TYPE="ext2"
                blocks per inode:        3.99351 
                bytes per inode:         16357.4 
                Inode count:                   262944 
                Free:                          262617           Used:    327
                guest@porteus:~$ time sudo find /mnt/sda5 -xdev -printf "\n" | wc -l 
                331
                real    0m0.889s
                user    0m0.012s
                sys     0m0.191s
                
                guest@porteus:~$ inodes sda6                                        TYPE="ext2"
                blocks per inode:        3.99812 
                bytes per inode:         16376.3 
                Inode count:                   122880 
                Free:                          121634           Used:    1246
                guest@porteus:~$ time sudo find /mnt/sda6 -xdev -printf "\n" | wc -l  
                1247
                real    0m0.638s
                user    0m0.016s
                sys     0m0.059s
                
                TYPE="vfat"
                guest@porteus:~$ time sudo find /mnt/sda8 -xdev -printf "\n" | wc -l
                13
                real    0m0.044s
                user    0m0.018s
                sys     0m0.006s
                
                guest@porteus:~$ inodes sda9                                        TYPE="ext2"
                blocks per inode:        4 
                bytes per inode:         16384 
                Inode count:                   5120000 
                Free:                          5108239          Used:    11761
                guest@porteus:~$ time sudo find /mnt/sda9 -xdev -printf "\n" | wc -l
                11725
                real    0m6.724s
                user    0m0.050s
                sys     0m0.331s
                
                guest@porteus:~$ inodes sda10                                        TYPE="ext2"
                blocks per inode:        3.99905 
                bytes per inode:         16380.1 
                Inode count:                   5324800 
                Free:                          5256657          Used:    68143
                guest@porteus:~$ time sudo find /mnt/sda10 -xdev -printf "\n" | wc -l
                68559
                real    0m23.372s
                user    0m0.204s
                sys     0m1.323s
                
              • Epilog

                Code: Select all

                guest@porteus:~$ awk -F: '/Used:/{inodes+=$3;} END{print "Total Used Inodes =",inodes;}'  sda_infi.txt 
                Total Used Inodes =  81477
                Reiserfs Files:     444624
                Vfat Files:             45
                Total Inode Reqmt:  526146
                
                guest@porteus:~$ awk -F: '/count:/{all+=$2;} END{print "Total Allocated:\t",all;}'  sda_infi.txt 
                Total Allocated:         10830624
                
                Overkill    =            10749147
                Lost to Unused Inodes = 1312MiB+159104B = 1GiB+288MiB+155KiB
                
                :shock:
            • Parametrisizing Inode Creation
              • Usage Types

                Code: Select all

                small, floppy, news, largefile, and largefile4
                The news filesystem type creates one inode per 4-kb block. The largefile filesystem type creates one inode per megabyte. The largefile4 filesystem type creates one inode per 4-MB block.
              • Inode Numbers

                Code: Select all

                root@porteus:/home/guest# mkfs.ext4 -b 4096 -m 0 -T huge -O ^has_journal /dev/sdb3 
                mke2fs 1.43.1 (08-Jun-2016)
                Creating filesystem with 79167488 4k blocks and 4947968 inodes
                blocks per inode:        16 
                bytes per inode:         65536
                
                root@porteus:/home/guest# mkfs.ext4 -b 4096 -m 0 -T largefile -O ^has_journal /dev/sdb3 
                mke2fs 1.43.1 (08-Jun-2016)
                guest@porteus:~$ inodes sdb3
                blocks per inode:        256 
                bytes per inode:         1048576 
                Inode count:                   309248 
                Free:                          309237           Used:    11
                
                root@porteus:/home/guest# mkfs.ext4 -b 4096 -m 0 -T largefile4 -O ^has_journal /dev/sdb3 
                mke2fs 1.43.1 (08-Jun-2016)
                Creating filesystem with 79167488 4k blocks and 77312 inodes
                blocks per inode:        1024 
                bytes per inode:         4194304
                
                root@porteus:/home/guest# time mkfs.ext4 -b 4096 -m 0 -N 4096 -O ^has_journal  /dev/sdb3
                mke2fs 1.43.1 (08-Jun-2016)
                /dev/sdb3 contains a ext4 file system
                        created on Tue Jan 24 17:17:04 2017
                Proceed anyway? (y,n) y
                Creating filesystem with 79167488 4k blocks and 38656 inodes
                
                blocks per inode:        2048
                bytes per inode:         8388608
                
            • Requesting 640k Inodes (655340):

              Code: Select all

              root@porteus:/home/guest# time mkfs.ext4 -b 4096 -m 0 -N 655340 -O ^has_journal  /dev/sdb1
              mke2fs 1.43.1 (08-Jun-2016)
              /dev/sdb1 contains a ext2 file system
                      last mounted on Wed Feb  1 11:48:23 2017
              Proceed anyway? (y,n) y
              Creating filesystem with 61079552 4k blocks and 656128 inodes
              Filesystem UUID: 1088cffa-7046-4a2a-b3bc-748d06932797
              Superblock backups stored on blocks: 
                      32768, 98304, 163840, 229376, 294912, 819200, 884736, 1605632, 2654208, 
                      4096000, 7962624, 11239424, 20480000, 23887872
              
              Allocating group tables: done                            
              Writing inode tables: done                            
              Writing superblocks and filesystem accounting information: done     
              
              real    0m28.004s
              user    0m0.032s
              sys     0m0.436s
              
              
              guest@porteus:~$ inodes sdb1
              blocks per inode:        93.0909 
              bytes per inode:         381300 
              Inode count:                   656128 
              Free:                          656117           Used:    11
              
              Space Reserved for Extra Inodes = 15MiB+888KiB ~ 16MiB
              
          • Creating a Directory for Each Relevant HD Partition

            Code: Select all

            guest@porteus:~$ mkdir x; sudo mount /dev/sdb1 x
            guest@porteus:~$ df -i x
            Filesystem     Inodes IUsed  IFree IUse% Mounted on
            /dev/sdb1      656128    11 656117    1% /home/guest/x
            guest@porteus:~$ df -h x
            Filesystem      Size  Used Avail Use% Mounted on
            /dev/sdb1       233G   60M  233G   1% /home/guest/x
            guest@porteus:~$ sudo mkdir x/sda{1,{3..6},{8..10}}
            
            guest@porteus:~$ ls  x
            lost+found/  sda1/  sda10/  sda3/  sda4/  sda5/  sda6/  sda8/  sda9/
            guest@porteus:~$ sudo chmod a+w x/sda{1,{3..6},{8..10}}
            
          • VFAT Partitions RSYNCed
            • sda8

              Code: Select all

              guest@porteus:~$ rsync -aAXv  /mnt/sda8/   x/sda8                
              sending incremental file list
              rsync: failed to set times on "/home/guest/x/sda8/.": Operation not permitted (1)
              ./
              lm/
              lm/cd/
              lm/vd/
              lm/vd/casper-rw
              por/
              por/savbg.dat
              rc5/
              rc5/porteus/
              rc5/porteus/porteus-v3.2-i586.sgn
              rc5/porteus/base/
              rc5/porteus/base/000-kernel.xzm
              rc5/porteus/modules/
              
              sent 1,115,175,690 bytes  received 217 bytes  24,509,360.59 bytes/sec
              total size is 1,114,902,965  speedup is 1.00
              rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1178) [sender=3.1.2]
              
              • Code: Select all

                guest@porteus:~$ tree -n x/sda8
                x/sda8
                ├── lm
                │   ├── cd
                │   └── vd
                │       └── casper-rw
                ├── por
                │   └── savbg.dat
                └── rc5
                    └── porteus
                        ├── base
                        │   └── 000-kernel.xzm
                        ├── modules
                        └── porteus-v3.2-i586.sgn
                8 directories, 4 files
                
              • Code: Select all

                guest@porteus:~$ ls -lh x/sda8//por/savbg.dat 
                -rwxr-xr-x 1 guest users 0 Aug 15 13:02 x/sda8//por/savbg.dat*
                guest@porteus:~$ ls -lh p8//por/savbg.dat 
                -rwxr-xr-x 1 root root 0 Aug 15 13:02 p8//por/savbg.dat*
                
                guest@porteus:~$ ls -lh x/sda8/rc5/porteus/base/000-kernel.xzm 
                -rwxr-xr-x 1 guest users 40M Jul  2  2016 x/sda8/rc5/porteus/base/000-kernel.xzm*
                guest@porteus:~$ ls -lh p8/rc5/porteus/base/000-kernel.xzm 
                -rwxr-xr-x 1 root root 40M Jul  2  2016 p8/rc5/porteus/base/000-kernel.xzm*
                
                guest@porteus:~$ ls -lh x/sda8/lm/vd/casper-rw 
                -rwxr-xr-x 1 guest users 1.0G Aug 14 02:26 x/sda8/lm/vd/casper-rw*
                guest@porteus:~$ ls -lh p8/lm/vd/casper-rw 
                -rwxr-xr-x 1 root root 1.0G Aug 14 02:26 p8/lm/vd/casper-rw*
                
            • sda3

              Code: Select all

              guest@porteus:~$ rsync -aAXv  /mnt/sda3/   x/sda3
              guest@porteus:~$ sending incremental file list
              rsync: failed to set times on "/home/guest/x/sda3/.": Operation not permitted (1)
              ./
              XF86Config-4
              bzimage
              initrfs.gz
              ploplinux/
              ploplinux/bin.sqfs
              ploplinux/copying
              ploplinux/etc.sqfs
              ploplinux/lib.sqfs
              ploplinux/license
              ploplinux/ploplinux
              ploplinux/sbin.sqfs
              ploplinux/usr.sqfs
              ploplinux/myscripts/
              ploplinux/myscripts/8x16
              ploplinux/myscripts/8x8
              ploplinux/myscripts/autologin
              ploplinux/myscripts/eth
              ploplinux/myscripts/mountopt
              ploplinux/myscripts/readme.txt
              ploplinux/myscripts/runme.sh
              ploplinux/myscripts/setwol
              ploplinux/myscripts/startbluetooth
              ploplinux/myscripts/startdmcrypt
              ploplinux/myscripts/startnamed
              ploplinux/myscripts/startnfs
              ploplinux/myscripts/startproftpd
              ploplinux/myscripts/startsamba
              ploplinux/myscripts/startsshd
              ploplinux/myscripts/startsyslogd
              ploplinux/myscripts/useavast
              ploplinux/myscripts/usefprot
              
              sent 52,915,120 bytes  received 668 bytes  6,225,386.82 bytes/sec
              total size is 52,900,397  speedup is 1.00
              rsync error: some files/attrs were not transferred (see previous errors) (code 23) at main.c(1178) [sender=3.1.2]
              
              real    0m8.207s
              user    0m0.420s
              sys     0m0.240s
              
              [1]+  Exit 23                 time rsync -aAXv  /mnt/sda3/   x/sda3
              
              • Code: Select all

                guest@porteus:~$ tree -nd x/sda3
                x/sda3
                └── ploplinux
                    └── myscripts
                2 directories
                
            • Linux Partitions to RSYNC
              • Arch Way

                Code: Select all

                # rsync -aAXv --exclude={"/dev/*","/proc/*","/sys/*","/tmp/*","/run/*","/mnt/*","/media/*","/lost+found"} / /path/to/backup/folder                                             
                • Code: Select all

                  The --exclude option causes files that match the given patterns to be
                   excluded. The contents of /dev, /proc, /sys, /tmp, and /run are excluded in
                   the above command, because they are populated at boot, although the folders 
                  themselves are not created. /lost+found is filesystem-specific.
                  
                • Code: Select all

                      Using many hard links, consider adding the -H option, which is turned off by default due to its memory expense
                       
                      Using sparse files, such as virtual disks, Docker images and similar, add the -S option.
                      
                      The --numeric-ids option will disable mapping of user and group names; instead, numeric group and user IDs will be transfered. This is useful when backing up over SSH or when using a live system to backup different system disk.
                      
                      Choosing --info=progress2 option instead of -v will show the overall progress info and transfer speed instead of the list of files being transferred.
                  
                      Restore backup, using the same rsync command that was executed but with the source and destination reversed. 
                  
              • Whittled down rsync Options for Consideration

                Code: Select all

                 -v, --verbose               increase verbosity
                     --info=FLAGS            fine-grained informational verbosity
                     --debug=FLAGS           fine-grained debug verbosity
                 **    --msgs2stderr           special output handling for debugging
                     --archive               archive mode; equals -rlptgoD (no -H,-A,-X)
                
                 -l, --links            When symlinks are encountered, recreate the symlink on the destination.
                -L, --copy-links            transform symlink into referent file/dir
                     --copy-unsafe-links     only "unsafe" symlinks are transformed
                     --safe-links            ignore symlinks that point outside the tree
                     --munge-links           munge symlinks to make them safer
                 -k, --copy-dirlinks         transform symlink to dir into referent dir
                 -K, --keep-dirlinks         treat symlinked dir on receiver as dir
                 -H, --hard-links            preserve hard links
                 
                 -E, --executability         preserve executability
                     --chmod=CHMOD           affect file and/or directory permissions
                 -A, --acls                  preserve ACLs (implies -p)
                 -X, --xattrs                preserve extended attributes
                
                 -S, --sparse                handle sparse files efficiently
                 -n, --dry-run               perform a trial run with no changes made
                 -x, --one-file-system       do not cross filesystem boundaries
                
                 -m, --prune-empty-dirs      prune empty directory chains from file-list
                 -T, --temp-dir=DIR          create temporary files in directory DIR
                
                     --link-dest=DIR         hardlink to files in DIR when unchanged
                
                     --exclude=PATTERN       exclude files matching PATTERN
                     --exclude-from=FILE     read exclude patterns from FILE
                     --include=PATTERN       do not exclude files matching PATTERN
                     --include-from=FILE     read include patterns from FILE
                     --files-from=FILE       read list of source-file names from FILE
                
                 -h, --human-readable        output numbers in a human-readable format
                     --progress              show progress during transfer
                     --log-file=FILE         log what we are doing to the specified FILE
                
            • Traversing the RSYNC Options
              • Clear Options

                Code: Select all

                --archive Taken for granted
                --hard-links            preserve hard links
                --executability         preserve executability
                
                 -A, --acls                  preserve ACLs (implies -p)
                 -X, --xattrs                preserve extended attributes
                
                 -S, --sparse                handle sparse files efficiently
                 -n, --dry-run               perform a trial run with no changes made
                 -x, --one-file-system       do not cross filesystem boundaries
                
                [ -m, --prune-empty-dirs      prune empty directory chains from file-list]  Prefer to see Directory Structure
                [ -T, --temp-dir=DIR          create temporary files in directory DIR]  Case of Remote Transfer?
                
                 --exclude=PATTERN       exclude files matching PATTERN
                  --exclude="/lost+found"
                
                 -h, --human-readable        output numbers in a human-readable format
                [     --progress              show progress during transfer] Replaced by --info
                     --log-file=FILE         log what we are doing to the specified FILE
                
                --numeric-ids   disables mapping of user and group names; when using a live system to backup different system disk.
                    
                --info=progress2   shows the overall progress info and transfer speed
                
                
              • Symbolic Link Options. All Requiring Clarification:

                Code: Select all

                 -l, --links            When symlinks are encountered, recreate the symlink on the destination.
                -L, --copy-links            transform symlink into referent file/dir
                     --copy-unsafe-links     only "unsafe" symlinks are transformed
                     --safe-links            ignore symlinks that point outside the tree
                     --munge-links           munge symlinks to make them safer
                 -k, --copy-dirlinks         transform symlink to dir into referent dir
                 -K, --keep-dirlinks         treat symlinked dir on receiver as dir
                
              • Test

                Code: Select all

                guest@porteus:~$ ls -l /mnt/sda5/rsync/tst                      
                total 4
                lrwxrwxrwx 1 guest users   49 Dec 25 20:54 05-devel.xzm -> /mnt/sda10/tmp/iso/3.2.2/i586/kernel/05-devel.xzm
                lrwxrwxrwx 1 guest users   29 Feb  6 23:20 Pictures -> /home/guest/p10/tmp/Pictures//
                -rw-r--r-- 1 guest users 4096 May 21  2016 blacklist.xzm
                lrwxrwxrwx 1 guest users   57 Dec 26 03:12 crippled_sources.xzm -> /mnt/sda10/tmp/iso/3.2.2/i586/kernel/crippled_sources.xzm
                
                guest@porteus:~$ rsync -aAXv /mnt/sda5/rsync  p10/tmp/tst
                sending incremental file list
                rsync/
                rsync/tst/
                rsync/tst/.directory
                rsync/tst/05-devel.xzm -> /mnt/sda10/tmp/iso/3.2.2/i586/kernel/05-devel.xzm
                rsync/tst/Pictures -> /home/guest/p10/tmp/Pictures/
                rsync/tst/blacklist.xzm
                rsync/tst/crippled_sources.xzm -> /mnt/sda10/tmp/iso/3.2.2/i586/kernel/crippled_sources.xzm
                
                sent 4,663 bytes  received 75 bytes  9,476.00 bytes/sec
                total size is 4,290  speedup is 0.91
                
                guest@porteus:~$ ls -l p10/tmp/tst/rsync/tst/
                total 4
                lrwxrwxrwx 1 guest users   49 Dec 25 20:54 05-devel.xzm -> /mnt/sda10/tmp/iso/3.2.2/i586/kernel/05-devel.xzm
                lrwxrwxrwx 1 guest users   29 Feb  6 23:20 Pictures -> /home/guest/p10/tmp/Pictures//
                -rw-r--r-- 1 guest users 4096 May 21  2016 blacklist.xzm
                lrwxrwxrwx 1 guest users   57 Dec 26 03:12 crippled_sources.xzm -> /mnt/sda10/tmp/iso/3.2.2/i586/kernel/crippled_sources.xzm
                
                Would seem to indicate no Explicit Symbolic Link Options required.
                • Finalised RSYNC Parameter Superset

                  Code: Select all

                  -aAXh --log-file= --info=progress2 --hard-links --executability  --numeric-ids    --one-file-system --sparse --exclude="/lost+found" exclude=GiBswap [ --exclude="/tmp/*"] [ --dry-run]
                   
              Do make sure to Watch Next Exciting Episode of RSYNC Backup 8)
              Linux porteus 4.4.0-porteus #3 SMP PREEMPT Sat Jan 23 07:01:55 UTC 2016 i686 AMD Sempron(tm) 140 Processor AuthenticAMD GNU/Linux
              NVIDIA Corporation C61 [GeForce 6150SE nForce 430] (rev a2) MemTotal: 901760 kB MemFree: 66752 kB

              Post Reply