In this tutorial, you will learn how to synchronize data using lsyncd on Ubuntu 22.04. Lsyncd is a daemon that monitors changes from directories and sub-directories and synchronizes them to a given location. The name lsyncd stands for Live Syncing Mirror Daemon.
Lsyncd watches local directory trees through an event monitor interface (inotify, fsevents). It aggregates and combines events for a few seconds and then spawns one or more processes to synchronize the changes. By default this is rsync. Rsync+ssh is an advanced action configuration that uses a SSH to act file and directory moves directly on the target instead of re-transmitting the move destination over the wire.
Synchronize data using Lsyncd on Ubuntu 22.04
Lsyncd is not installed by default on Ubuntu. Thus, in order to use it to synchronize data using Lsyncd, you first have to install it.
It’s a good practice to make sure Linux packages are updated before fresh installation, thus;
sudo apt -y updateOnce the package update is done, install lsyncd on Ubuntu;
sudo apt -y install lsyncdYou can verify installed version of Lsyncd by executing the command;
lsyncd --versionSample output
Version: 2.2.3Configuring Lsyncd on Ubuntu
Lsyncd can be configured for both local and remote syncing.
Remote syncing involves syncing files to a server while local syncing is for data within your local computer.
Configuring Lsyncd local syncing
To demonstrate this, let’s create a source folder where my data to be synced will be stored.
I will make a new directory called “data” under my documents directory
mkdir -p $HOME/Documents/dataAdditionally, I will create the destination directory as follows
mkdir -p $HOME/sync_documents/backupNext, create Lsyncd status and log files
sudo mkdir /var/log/lsyncd
sudo touch /var/log/lsyncd/lsyncd.{log,status}Now let’s create a directory to store Lsyncd configurations
sudo mkdir /etc/lsyncdNext, create Lsyncd local syncing configuration file.
sudo vim /etc/lsyncd/lsyncd.conf.luaThen add the information below. Replace the source and destination paths accordingly.
settings {
logfile = "/var/log/lsyncd/lsyncd.log",
statusFile = "/var/log/lsyncd/lsyncd.status"
}
sync {
default.rsync,
source = "/home/itnixpro/Documents/data",
target = "/home/itnixpro/sync_documents/backup"
}
Save and exit the file.
Restart Lsyncd to effect the changes above:
sudo systemctl restart lsyncdCheck the status;
systemctl status lsyncd● lsyncd.service - LSB: lsyncd daemon init script
Loaded: loaded (/etc/init.d/lsyncd; generated)
Active: active (running) since Thu 2022-03-24 19:20:14 EAT; 5s ago
Docs: man:systemd-sysv-generator(8)
Process: 50152 ExecStart=/etc/init.d/lsyncd start (code=exited, status=0/SUCCESS)
Tasks: 1 (limit: 2306)
Memory: 812.0K
CPU: 40ms
CGroup: /system.slice/lsyncd.service
└─50159 /usr/bin/lsyncd -pidfile /var/run/lsyncd.pid /etc/lsyncd/lsyncd.conf.lua
Mac 24 19:20:14 ubuntu2204 systemd[1]: Starting LSB: lsyncd daemon init script...
Mac 24 19:20:14 ubuntu2204 lsyncd[50152]: * Starting synchronization daemon lsyncd
Mac 24 19:20:14 ubuntu2204 lsyncd[50157]: 19:20:14 Normal: --- Startup, daemonizing ---
Mac 24 19:20:14 ubuntu2204 lsyncd[50152]: ...done.
Mac 24 19:20:14 ubuntu2204 systemd[1]: Started LSB: lsyncd daemon init script.
That is it! We have set our directories to automatically sync our data upon modification to our source folder.
To demonstrate how this works, let’s create some test files on the source directory we created above, $HOME/Documents/data and see whether it gets copied to the target destination directory, $HOME/sync_documents/backup;
for i in {1..20}; do touch $HOME/Documents/data/file$i; doneOnce the command run, check the Lsyncd status and log files;
sudo tail -50 /var/log/lsyncd/lsyncd.logThu Mar 24 19:37:59 2022 Normal: --- TERM signal, fading --- Thu Mar 24 19:37:59 2022 Normal: --- Startup, daemonizing --- Thu Mar 24 19:37:59 2022 Normal: recursive startup rsync: /home/itnixpro/Documents/data/ -> /home/itnixpro/sync_documents/backup/ Thu Mar 24 19:37:59 2022 Normal: Startup of /home/itnixpro/Documents/data/ -> /home/itnixpro/sync_documents/backup/ finished. Thu Mar 24 19:38:41 2022 Normal: Calling rsync with filter-list of new/modified files/dirs /file1 / /file2 /file3 /file4 /file5 /file6 /file7 /file8 /file9 /file10 /file11 /file12 /file13 /file14 /file15 /file16 /file17 /file18 /file19 /file20 Thu Mar 24 19:38:41 2022 Normal: Finished a list after exitcode: 0
Check the status log;
sudo tail -50 /var/log/lsyncd/lsyncd.statusLsyncd status report at Thu Mar 24 19:38:51 2022 Sync1 source=/home/itnixpro/Documents/data/ There are 0 delays Filtering: nothing. Inotify watching 1 directories 1: /home/itnixpro/Documents/data/
From the logs, it is evident that files have been copied to the backup directory;
ls -1 /home/itnixpro/sync_documents/backup/file1 file10 file11 file12 file13 file14 file15 file16 file17 file18 file19 file2 file20 file3 file4 file5 file6 file7 file8 file9
Configuring Lsyncd remote syncing
So how can you use Lsyncd to sync local directories/files to a remote host? The process of remote configuration is similar to the local configuration except that the target is a remote host.
If you want to be able to automatically sync files, then you have to set up passwordless SSH authentication using a public/private key pair.
Create SSH public/private key pair using the ssh-keygen command.
ssh-keygenWhen prompted for password, press ENTER to set blank password.
Then use the ssh-copy-id command to copy the public key to the remote host. Replace the username and host accordingly.
ssh-copy-id [email protected]On the remote host;
- create a destination folder. In my setup, I have created a directory called
/server/sync_documents/backup
sudo mkdir -p /server/sync_documents/backup- Next, ensure the remote user has rights to write to the remote backup directory. Be sure to use correct user;
chown -R johndoe: /server/sync_documents/backupOn the local source host, the source of the data to be synchronized;
- edit or create a new configuration file;
sudo vim /etc/lsyncd/lsyncd.conf.luasettings {
logfile = "/var/log/lsyncd/lsyncd.log",
statusFile = "/var/log/lsyncd/lsyncd.status",
statusInterval = 20,
nodaemon = false,
}
sync {
default.rsyncssh,
source = "/home/itnixpro/sync_documents/backup",
host = "192.168.124.16",
targetdir = "/server/sync_documents/backup",
rsync = {
owner = true,
group = true,
perms = true,
rsh = "/usr/bin/ssh -l johndoe -i /home/itnixpro/.ssh/id_rsa -o StrictHostKeyChecking=no"
}
}
- Ensure rsync command is installed on both source and destination host;
sudo apt install rsyncsudo yum install rsync- Lastly, restart lsyncd;
sudo systemctl restart lsyncdCheck the status to ensure all is well;
systemctl status lsyncdAlso tail the logs;
sudo tail -f /var/log/lsyncd/lsyncd.logOn the remote host, you can watch the directory to see as files are copied;
watch -d ls -alh /server/sync_documents/backupEvery 2.0s: ls -alh /server/sync_documents/backup debian11: Thu Mar 24 20:35:35 2022 total 8.0K drwxrwxr-x 2 johndoe johndoe 4.0K Mar 24 19:38 . drwxr-xr-x 3 root root 4.0K Mar 24 19:47 .. -rw-r--r-- 1 johndoe johndoe 0 Mar 24 19:38 file1 -rw-r--r-- 1 johndoe johndoe 0 Mar 24 19:38 file10 -rw-r--r-- 1 johndoe johndoe 0 Mar 24 19:38 file11 -rw-r--r-- 1 johndoe johndoe 0 Mar 24 19:38 file12 -rw-r--r-- 1 johndoe johndoe 0 Mar 24 19:38 file13 -rw-r--r-- 1 johndoe johndoe 0 Mar 24 19:38 file14 -rw-r--r-- 1 johndoe johndoe 0 Mar 24 19:38 file15 -rw-r--r-- 1 johndoe johndoe 0 Mar 24 19:38 file16 -rw-r--r-- 1 johndoe johndoe 0 Mar 24 19:38 file17 -rw-r--r-- 1 johndoe johndoe 0 Mar 24 19:38 file18 -rw-r--r-- 1 johndoe johndoe 0 Mar 24 19:38 file19 -rw-r--r-- 1 johndoe johndoe 0 Mar 24 19:38 file2 -rw-r--r-- 1 johndoe johndoe 0 Mar 24 19:38 file20 -rw-r--r-- 1 johndoe johndoe 0 Mar 24 19:38 file3 -rw-r--r-- 1 johndoe johndoe 0 Mar 24 19:38 file4 -rw-r--r-- 1 johndoe johndoe 0 Mar 24 19:38 file5 -rw-r--r-- 1 johndoe johndoe 0 Mar 24 19:38 file6 -rw-r--r-- 1 johndoe johndoe 0 Mar 24 19:38 file7 -rw-r--r-- 1 johndoe johndoe 0 Mar 24 19:38 file8 -rw-r--r-- 1 johndoe johndoe 0 Mar 24 19:38 file9
How fantastic and simple is that!
Conclusion
In this tutorial, you have learned how to synchronize data using lsyncd on Ubuntu 22.04 both locally and to remove hosts.
More Example Configurations on;
man lsyncd