Features

Edit on Gitlab

Table of Contents

  • Automatic backup of databases: A set of database backup scripts detects exsiting locally running database services and puts a compressed dump file per database scheme to a local backup directory.
  • local encryption of all data before transferring to backup storage
  • no agent, no sattelite, no central backup server. We deploy the software and a config set to a server and the backup will run itself
  • you can add custom scripts in many hooks to react if actions failed or were ok
  • supported restore actions:
    • restore files files from backup storage
    • restic on linux: mount restic repository with fuse - then you can browse through snapshots and data like in a normal filesystem
    • restore a single database with some comfort
  • a check script shows the local Backupstatus

How does it work?

For backup it makes

  • the dump of all found local databases and then
  • transfer all files with a backup tool:
    • locally encrypted files will be transferred to a mounted dir, ssh, rsync, https, (the available backaneds depend on the used backup tool) … to a backup target system
    • on the backup target each system has its own subdirectory or repository to store its data
graph LR Start((Start)) --> | Cronjob<br>in the night | backup.sh(backup.sh:<br><br>init config) backup.sh --> localdump.sh(localdump.sh:<br><br>Dump<br>local<br>databases) localdump.sh --> transfer.sh(transfer.sh:<br><br>Secure<br>file backup<br>using Restic<br>or Duplicity) transfer.sh --> End((End))

Dump local databases

Before starting the backup of local files to a backup target there is a step to dump a local database. If one of the supported database types is detected it will be dumped.

We try to implement the approach “zero config” if possible.

Supported local databases for backup and restore:

  • Mysql/ Mariadb (requires mysql_dump)
  • postgres (pg_dump)
  • sqlite (by naming files with full path in a config)

Limited support:

  • couchdb (using a config with naming convention)
  • ldap (without restore so far)
graph LR %% --- define nodes ---------- localdump.sh(localdump.sh) loopdbs((Loop over<br>dabase types)) db1[(mysql)] db2[(pgsql)] db3[(...)] db1A[db A] db1B[db B] db1C[db C] subgraph /var/iml-backup/mysql/ dump1A[dump A] dump1B[dump B] dump1C[dump C] end %% --- connect nodes ---------- localdump.sh --> loopdbs loopdbs --> db1 loopdbs --> db2 loopdbs --> db3 db1 --> db1A db1 --> db1B db1 --> db1C db1A --> dump1A db1B --> dump1B db1C --> dump1C

File backup

It is a classic backup a set of directories. Includes and excludes can be defined (but I prefer to backup all files).

The backup target for local database backups is included automatically.

Additionally there is a flag for a local Samba: if enabled all found shares will treated as directory to backup.

graph LR %% --- define nodes ---------- transfer.sh(transfer.sh) loopdirs((Loop over<br>set of dirs)) dir1(/etc) dir2(/home) dir3(/var/www) dirA(/var/iml-backup/mysql) dirB(...) backup(Backup<br>Restic or<br>Duplicity) backuptarget(Remote Backup target<br>in the cloud<br>or on another<br>physical location) verify(Verify<br>written<br>data) %% --- connect nodes ---------- transfer.sh --> loopdirs loopdirs --> dir1 loopdirs --> dir2 loopdirs --> dir3 loopdirs --> dirA loopdirs --> dirB dir1 --> backup dir2 --> backup dir3 --> backup dirA --> backup dirB --> backup backup --> |transfers<br>encrypted<br>files| backuptarget backuptarget --> verify

Hooks

You have several entry points to execute your custom scripts before, after and during th backup process. See Hooks

Backup tools

Restic

  • creates one initial full backup - and then never again (then it starts incremental backups only).
  • encrypts data
  • deduplicates files
  • delete backups by rules to keep a count of hourly, daily, weekly, mothly, yearly backups
  • several backup targets (currently supported protocols: sftp:// https:// and file://)
  • Single binary (written in Go)
  • MS Windows: Backup uses volume shadow copy service (vss)
  • Linux: with Fuse you can browse you can mount the backup repository and browse through all snapshots

Duplicity

  • Incremental and full backups
  • encrypted backups using GPG
  • set size of backup volumes
  • delete old backups by a given time limit
  • several backup targets (currently supported protocols: scp:// rsync:// and file://)
  • Backup script (written in Python) with dependencies and possibly differend versions on different OS

Limit simoultanous backups

As an optional feature you can control the count simultanous written backups. This requires additional effort next to the client installation.