As an IT service provider, I am confronted with various software solutions on a daily basis. Of these, my colleagues in enterprise environments use Veeam and similar software. While this software is super designed for your purpose, I have always been a big fan of open source software. I recently came across the Proxmox Backup Server.
Similar to how other solutions have a focus point, for example in the Veeam and VMWare use case, Proxmox Backup Server also has its optimized area. As you would expect, Proxmox Backup Server is optimized on the Proxmox VE. A complete open source hypervisor. The host backups or actually it should be more accurately called file level backup.
Proxmox Backup Server is a Debian Linux-based solution that can be installed as a virtual appliance (also under Proxmox VE) or as a hardware appliance. The PBS uses a proprietary chunking technique to provide efficient deduplication and compression of data. It provides a web-based user interface that is easy to use and an intuitive method for creating and managing backup jobs. The PBS does not in itself support normal full and incremental backups as they are traditionally known. Rather, it uses a process called chunking.
Chunking
Proxmox Backup Server uses chunking to break large files into small, efficient pieces, much like a jigsaw puzzle. These pieces are then deduplicated and compressed to optimize the backup process. The PBS creates a guide for itself to reassemble these puzzle pieces later.
Chunking is a process that divides large files into small, equal-sized pieces (chunks) that can then be deduplicated and compressed. Proxmox Backup Server uses a variable chunk size that is automatically adjusted to the size of the data to achieve optimal deduplication. This means that the PBS stores only the changes between the chunks, reducing the amount of data and shortening the backup time.
When restoring data, the chunks are reassembled to restore the original file. Because each chunk is assigned a unique hash value, the PBS can also ensure that data integrity is maintained during the backup and restore process.
In addition to this chunking process, Proxmox Backup Server also provides support for encryption and compression of backup data to maximize the security and efficiency of the backup process.
For block-based backups, Proxmox Backup Server uses a fixed chunk size of 4 MiB. This type of backup is usually used for virtual machines and is based on the block level, which means that changes to the entire virtual disk are detected and backed up.
However, for file-based backups, Proxmox Backup Server uses a variable chunk size from 64 KiB to 4 MiB to achieve optimal deduplication of data. File-based backups are based on the file level and allow backing up specific files or directories instead of the entire disk.
File based backups
I currently have the backups in use with two methods, firstly I was amazed that these backups work incredibly well in Docker environments. As well as in cases where I want to backup the contents of a VM, e.g. our mailcow at Netcup.
The host backups from the PBS run via the cli tool proxmox-backup-client which is available from Proxmox. I create a bash script which is accessed via a cronjob.
- pulls data from the source (e.g. from a MySQL server via mysqldump)
- I put this data into a directory like backup
- I lead the
proxmox-backup-clientand specify the directory with the data as source.
In reality, this would look like this.
Backup.sh
1. # !/bin/bash
# Stop script at first error
2. set -e
# If an error occurs, report it to my monitoring server.
3. trap 'curl -fs "https://status.meinuptimekuma.de/api/push/Avf1241sdas?status=down&msg=FAILED"' ERR
# Execute mysqldump, move the backup to the backup folder.
4. /root/backup-database.sh
# Set environment variables
5. export PBS_REPOSITORY=mailcow@[email protected]:datastore
6. export PBS_PASSWORD="xxxx".
7. export PBS_ENCRYPTION_PASSWORD="xxxx"
# Run the proxmox backup client.
8. proxmox-backup-client backup database.pxar:/root/database-backups docker-volumes.pxar:/var/lib/docker/volumes mailcow-config.pxar:/root/mailcow-dockerized --ns mailcow --skip-lost-and-found
# Message to my Uptime Kuma (open source monitoring software) that the backup was successful.
10. curl "https://status.meinuptimekuma.de/api/push/Avf1241sdas?status=up&msg=OK"First of all, this article is not sponsored in any way. Becker Software is not related to Proxmox in any way.
German
English