[HOWTO] Sync 2 Folders

Post tutorials, HOWTO's and other useful resources here.
User avatar
FURRY_NOVA
Black ninja
Black ninja
Posts: 84
Joined: 11 Aug 2016, 17:26
Distribution: Porteus XFCE v3.2 i586
Location: United Kingdom

[HOWTO] Sync 2 Folders

Post#1 by FURRY_NOVA » 21 Sep 2016, 04:01

Intro~
Syncing files has become an important role in todays computing. If it's either keeping 2 devices up-to-date, or backing up and recovering. This small tutorial will teach you some basic methods.

Methods~
There are 4 Linux methods. They are:
  1. rsync Terminal - A command-line built in Porteus Terminal.
    Image
    Here's the command I use for syncing 2 directories. Be aware that the destination will match the source. Even if ya delete and replace one in destination, source will have top priority what the changes will be.

    Code: Select all

    rsync --delete --modify-window=1 --progress --recursive --times --update "/home/guest/Desktop/1/" "/home/guest/Desktop/2/"
    The first directory is the source /home/guest/Desktop/1/.
    The second directory is the destination /home/guest/Desktop/2/.
  2. .desktop Launcher - A method of having a saved double-click button on your desktop to run a stored rsync command.
    Image
    This involves making a .desktop application launcher file. The process is simple. Open a text editor such as Mousepad and copy the following code inside it. Then edit source (A="/home/guest/Desktop/a/") and destination (B="/home/guest/Desktop/b/").When you're happy, save the file as any name you want as long as it ends in ".desktop" without the quotes.

    Code: Select all

    [Desktop Entry]
    Name=Sync
    Exec=/bin/bash -c 'A="/home/guest/Desktop/a/"; B="/home/guest/Desktop/b/"; TXT_GREEN="\033[1;32m"; TXT_RED="\033[1;31m"; TXT_NONE="\033[0m"; echo "Running sync simulation..."; echo -e "${TXT_GREEN}${A}${TXT_NONE} >> ${TXT_RED}${B}${TXT_NONE}"; echo ""; rsync --delete --dry-run --modify-window=1 --progress --recursive --times --update "${A}" "${B}"; echo "Simulation completed."; echo ""; read -rsp "Press any key to perform operation, else close window to abort...\n" -n1 key; echo ""; rsync --delete --modify-window=1 --progress --recursive --times --update "${A}" "${B}"; echo "Operation completed."; echo ""; read -rsp "Press any key to exit...\n" -n1 key;'
    Type=Application
    Icon=emblem-synchronizing
    Comment="Rsync from directory A to B, specified in this launcher."
    Path=
    Terminal=true
    StartupNotify=false
    
  3. grsync GUI - An application which can be downloaded from USM. It's less that 1mb and requires no dependencies requirement.
    Image
  4. syncthing GUI - Another application. Although I've never used it before. More info can be found here:
    - Re: [HOWTO] Sync 2 Folders: syncthing (Post by Bogomips #49253)
    - Re: [HOWTO] Sync 2 Folders (Post by mocabilly #49584)
Which is best?~
Personally I used the .desktop Launcher version. Works exactly the same as rsync Terminal which is basically what it is. The grsync is buggy from my experiance and syncthing I hadn't tried.

Help~
You can get a list of all the rsync perimeters usable in the command-line by just entering rsync or man rsync.
Last edited by FURRY_NOVA on 01 Nov 2016, 05:29, edited 2 times in total.
New to Linux but learning. Hoping to help out Porteus community. :)

Bogomips
Full of knowledge
Full of knowledge
Posts: 2564
Joined: 25 Jun 2014, 15:21
Distribution: 3.2.2 Cinnamon & KDE5
Location: London

Re: [HOWTO] Sync 2 Folders

Post#2 by Bogomips » 21 Sep 2016, 11:51

Nice presentation. Rsync has ginormous options list, and fact that distilled down to a handful makes it easy to build on..

An enhancement could be running a cron job.
Linux porteus 4.4.0-porteus #3 SMP PREEMPT Sat Jan 23 07:01:55 UTC 2016 i686 AMD Sempron(tm) 140 Processor AuthenticAMD GNU/Linux
NVIDIA Corporation C61 [GeForce 6150SE nForce 430] (rev a2) MemTotal: 901760 kB MemFree: 66752 kB

User avatar
FURRY_NOVA
Black ninja
Black ninja
Posts: 84
Joined: 11 Aug 2016, 17:26
Distribution: Porteus XFCE v3.2 i586
Location: United Kingdom

Re: [HOWTO] Sync 2 Folders

Post#3 by FURRY_NOVA » 29 Sep 2016, 03:10

Update
I've added a Shortcut Method w/ Simulation.

Although I'm sure it should be possible to shorten the code by applying the directories once with variables but my knowledge is lacking in Linux terminal.

Example of my explanation. (Not an actual code example.)

Code: Select all

a="/home/guest/Desktop/1/";
b="/home/guest/Desktop/2/";
/bin/bash -c 'echo "Initializing rsync..."; rsync --delete --modify-window=1 --progress --recursive --times --update a b; echo "Operation completed."; read -rsp $"Press any key to close...\n" -n1 key'
New to Linux but learning. Hoping to help out Porteus community. :)

User avatar
mocabilly
White ninja
White ninja
Posts: 21
Joined: 12 Jan 2011, 17:53
Distribution: Mint-Porteus
Location: Belgium

Re: [HOWTO] Sync 2 Folders

Post#4 by mocabilly » 29 Sep 2016, 19:51

How about using syncthing?
Especially when you want to sync between multiple devices/servers/..

Been using it now for a couple of days and I'm just loving it..
Not sure though there is a module for Porteus.. :S
There are only 10 kind of people in this world, those who can read binary and those who can't.

User avatar
francois
Contributor
Contributor
Posts: 6434
Joined: 28 Dec 2010, 14:25
Distribution: xfce plank porteus nemesis
Location: Le printemps, le printemps, le printemps... ... l'hiver s'essoufle.

Re: [HOWTO] Sync 2 Folders

Post#5 by francois » 29 Sep 2016, 23:03

Un revenant. :shock:
Prendre son temps, profiter de celui qui passe.

User avatar
FURRY_NOVA
Black ninja
Black ninja
Posts: 84
Joined: 11 Aug 2016, 17:26
Distribution: Porteus XFCE v3.2 i586
Location: United Kingdom

Re: [HOWTO] Sync 2 Folders

Post#6 by FURRY_NOVA » 29 Sep 2016, 23:25

I tried checking it out. It isn't on USM. Also I tried downloading the source but I don't know how to compile it into a xzm module. I tried that src2pkg that comes with the Devel module, even done the "src2pkg --setup" and still get errors trying to "src2pkg /dir_to_source_file". :(
New to Linux but learning. Hoping to help out Porteus community. :)

User avatar
brokenman
Site Admin
Site Admin
Posts: 6105
Joined: 27 Dec 2010, 03:50
Distribution: Porteus v4 all desktops
Location: Brazil

Re: [HOWTO] Sync 2 Folders

Post#7 by brokenman » 01 Oct 2016, 02:46

Mocabilly! Great to see you here!

Gem of an application you posted about. Looks really interesting, kind of how cloud should be.
How do i become super user?
Wear your underpants on the outside and put on a cape.

Bogomips
Full of knowledge
Full of knowledge
Posts: 2564
Joined: 25 Jun 2014, 15:21
Distribution: 3.2.2 Cinnamon & KDE5
Location: London

Re: [HOWTO] Sync 2 Folders: syncthing

Post#8 by Bogomips » 03 Oct 2016, 18:10

SYNCTHING
  • Code: Select all

    USM version:  3.2.0
    DB version:  160320
    Current storage: /home/guest/p10/Por/tmp/usm
    Current config: /etc/usm/usm.conf
    Current platform: Porteus-v3.2rc5
    Current arch: i686
    Current Slackware 14.2
    Current kernel 4.7.2-porteus
    Current distros: slackware slackwarepatches slacky salix alien ponce slackonly
    
    USM offered to build. No thanks. Via pkgs.org downloaded 14.1 package from slackonly.
  • After making module and activating, checked for outstanding dependencies using modified form of ralcoer's script (http://forum.porteus.org/viewtopic.php? ... 247#p49224) for just the one bin/lib, ckdeps.sh (non-final version):

    Code: Select all

    #!/bin/bash
    #	03.10.2016 16:04:57
    # Check for outstanding dependencies. Ralcoer's script modified for just one bin/lib.
    # ckdeps.sh	<bin/lib>	[Directory holding bin/lib. Default: executable in /usr/bin]
    ckdeps()
    {
    dir=${2:-/usr/bin};
    if [[ ! -d ${dir} ]]; then
       echo "Parameter is not a directory."
       return 1
    fi
    [[ ! $1 ]] && return 2;
    file=${dir%/}/$1
    if [[ -e  $file ]]; then
        if [ ! -d ${file} ]; then
          elf=`file ${file} | grep ELF`
          if [ "${elf}" != "" ]; then
              missing=`ldd ${file} | grep "not found"`
              if [ "${missing}" != "" ]; then
                echo "-------"
                echo "${file}"
           echo "${missing}"
              fi
          fi
        fi
    else
    	echo File $1 not found in $dir; return 1
    fi
    }
    ckdeps $*
    
    Found no outstanding dependencies for syncthing.
  • Ran in terminal and after what seemed like taking an eternity to make the RSA key on my m/c, Light browser came up with a start page, despite the fact that Palemoon was already up and running.. Checking preferred applications showed Light as browser app, so presumably syncthing had invoked a browser application. Then at a loss how to proceed further.
  • At next effort. when Pomodoro Timer recommended a short break, started up syncthing and gave it some ten minutes to make the RSA key. Upon returning, Light browser was up on start page. This time, having recalled reading in documentation about listening on some local port, inserted a link from terminal output into browser,

    Code: Select all

    guest@porteus:~$ syncthing 
    [monitor] 15:01:17 INFO: Starting syncthing
    [start] 15:01:17 INFO: Generating RSA key and certificate for syncthing...
    [2BUIR] 15:01:53 INFO: syncthing v0.11.10 (go1.4.2 linux-386 default) unknown-user@syncthing-builder 2015-06-21 09:45:54 UTC
    [2BUIR] 15:01:53 INFO: My ID: 2BUIRFS-CUGAPJI-W22XN6Q-6G3SDKW-52UWCPK-GE4ANRP-A2F2FOW-XHB5HAX
    [2BUIR] 15:01:53 INFO: No config file; starting with empty defaults
    [2BUIR] 15:01:53 INFO: Edit /home/guest/.config/syncthing/config.xml to taste or use the GUI
    [2BUIR] 15:01:53 INFO: Database block cache capacity 8192 KiB
    [2BUIR] 15:01:53 OK: Ready to synchronize default (read-write)
    [2BUIR] 15:01:53 INFO: Starting web GUI on http://127.0.0.1:8384/
    [2BUIR] 15:01:53 INFO: Loading HTTPS certificate: open /home/guest/.config/syncthing/https-cert.pem: no such file or directory
    [2BUIR] 15:01:53 INFO: Creating new HTTPS certificate
    [2BUIR] 15:01:53 INFO: Generating RSA key and certificate for porteus...
    [2BUIR] 15:01:53 INFO: Completed initial scan (rw) of folder default
    [2BUIR] 15:02:11 INFO: Starting local discovery announcements
    [2BUIR] 15:02:11 INFO: Starting global discovery announcements
    [2BUIR] 15:02:11 INFO: Device 2BUIRFS-CUGAPJI-W22XN6Q-6G3SDKW-52UWCPK-GE4ANRP-A2F2FOW-XHB5HAX is "porteus" at [dynamic]
    [2BUIR] 15:02:11 INFO: API listening on 127.0.0.1:8384
    [2BUIR] 15:02:21 INFO: New UPnP port mapping: external port 31406 to local port 22000.
    
    and Admin GUI came up:Image
  • Module just 3.2 MB, and syncthing looks like not very demanding on memory, not at least at the start:

    Code: Select all

    guest@porteus:~$ ls -lh syncthing-0.11.10-i486-1_slack.xzm
    -rw-r--r-- 1 root root 3.2M Oct  2 18:25 syncthing-0.11.10-i486-1_slack.xzm
    
    # Running with LightPalemoon, Nemo, Geany, Terminator:
    guest@porteus:~$ free -m
                  total        used        free      shared  buff/cache   available
    Mem:            879         492          87          80         299         145
    Swap:           644         304         340
    
    All above+flash plugin-container:
    free -m
                  total        used        free      shared  buff/cache   available
    Mem:            879         496          68          56         314         165
    Swap:           644         396         248
    
It seems, unlike rsync, a change in one synced folder is reflected in all!
Linux porteus 4.4.0-porteus #3 SMP PREEMPT Sat Jan 23 07:01:55 UTC 2016 i686 AMD Sempron(tm) 140 Processor AuthenticAMD GNU/Linux
NVIDIA Corporation C61 [GeForce 6150SE nForce 430] (rev a2) MemTotal: 901760 kB MemFree: 66752 kB

User avatar
mocabilly
White ninja
White ninja
Posts: 21
Joined: 12 Jan 2011, 17:53
Distribution: Mint-Porteus
Location: Belgium

Re: [HOWTO] Sync 2 Folders

Post#9 by mocabilly » 12 Oct 2016, 19:56

Memory & processing time will increase with the number of files and sub-directories you sync with Syncthing.
It will take a lot files and subdirectories before it consumes to much of the resources..
There are only 10 kind of people in this world, those who can read binary and those who can't.

Bogomips
Full of knowledge
Full of knowledge
Posts: 2564
Joined: 25 Jun 2014, 15:21
Distribution: 3.2.2 Cinnamon & KDE5
Location: London

Re: [HOWTO] Sync 2 Folders: Limiting Case of Two Files

Post#10 by Bogomips » 04 Nov 2016, 01:06

Syncing Two or More Files

Folders could not be synced, as belonging to different architecture. Best could do was to resort to Links or Symlinks. With this approach, one file has to hold the real contents, while all the others are links to it. No danger of accidental deletion, as it will only be a link that is deleted.

Question then arises of where to hold the real contents, best could come up with was to have the file in a directory, difficult to accidentally delete, a root directory.

Creating a Symlink:

Code: Select all

ln  -s  /path/to/file  /path/to/symlink
# Important
ln  -s  <Absolute Path to File>  <Absolute Path to Symlink> 
Got caught here, as there is no error message output when relative paths given, and in certain cases symlink created, actually works.

Example
  • Salt away Contents

    Code: Select all

    guest@porteus:~$ sudo mkdir /mnt/sda4/safe
    
    guest@porteus:~$ ls -ld /mnt/sda4/safe/
    drwxr-xr-x 2 root root 48 Nov  3 19:58 /mnt/sda4/safe/
    guest@porteus:~$ sudo chmod a+w /mnt/sda4/safe
    guest@porteus:~$ ls -ld /mnt/sda4/safe/
    drwxrwxrwx 2 root root 48 Nov  3 19:58 /mnt/sda4/safe/
    
    guest@porteus:~$ touch /mnt/sda4/safe/"The files in this directory are reasonably safe from accidental deletion"
    
    guest@porteus:~$ ln /mnt/sda4/PaleMoon/pmProfile/contentblock-regex.txt  /mnt/sda4/safe/
    guest@porteus:~$ ls /mnt/sda4/safe/
    The\ files\ in\ this\ directory\ are\ reasonably\ safe\ from\ accidental\ deletion
    contentblock-regex.txt
    
  • Sync the Files

    Code: Select all

    guest@porteus:~$ mv /mnt/sda1/PaleMoon/pmProfile/contentblock-regex.txt /mnt/sda1/PaleMoon/pmProfile/bak_contentblock-regex.txt
    guest@porteus:~$ ln  -s  /mnt/sda4/safe/contentblock-regex.txt    /mnt/sda1/PaleMoon/pmProfile/
    
    guest@porteus:~$ ls -l  /mnt/sda1/PaleMoon/pmProfile/contentblock-regex.txt
    lrwxrwxrwx 1 guest users 42 Nov  3 21:26 /mnt/sda1/PaleMoon/pmProfile/contentblock-regex.txt -> /mnt/sda4/safe/contentblock-regex.txt
    
    guest@porteus:~$ tail -n3  /mnt/sda1/PaleMoon/pmProfile/contentblock-regex.txt
    terraclicks.com
    # 02.11.2016 19:45:30
    liveadexchanger.com
    
Maybe there's a better way? :unknown:
Linux porteus 4.4.0-porteus #3 SMP PREEMPT Sat Jan 23 07:01:55 UTC 2016 i686 AMD Sempron(tm) 140 Processor AuthenticAMD GNU/Linux
NVIDIA Corporation C61 [GeForce 6150SE nForce 430] (rev a2) MemTotal: 901760 kB MemFree: 66752 kB

rych
Warlord
Warlord
Posts: 622
Joined: 04 Jan 2014, 04:27
Distribution: Porteus 5.0 x64 OpenBox
Location: NZ
Contact:

[HOWTO] Sync 2 Folders

Post#11 by rych » 20 Oct 2022, 13:54

I like FreeFileSync so far: installs in the directory of your choice in a 2 seconds, integrates with the desktop, etc. Beautiful yet lightning fast GUI. Start by downloading their latest single binary installer for Linux.

Post Reply