As an IT service provider, I face the daily challenge of finding the right software for various application scenarios. While in many enterprise environments, solutions like Veeam have their merits and are well-tailored for specific use cases, I have always favored open-source software. This not only offers the flexibility I appreciate as a developer but also provides greater control over the solutions in use.
Recently, I came across the Proxmox Backup Server (PBS), a particularly promising open-source solution in the area of data backup. The PBS is specifically tailored to the Proxmox VE (Virtual Environment), an open-source hypervisor that has proven to be powerful and resource-efficient. But what truly sets the Proxmox Backup Server apart is not only its integration with Proxmox VE but also its innovative chunking method, which ensures particularly efficient deduplication and compression of data.
Compared to traditional backup solutions, the PBS offers some unique features and a user-friendly, web-based interface that simplifies the process of creating and managing backups. However, what fascinates me the most is the flexibility of the system, which can be used for backing up virtual machines, Docker environments, and individual databases.
In this article, I want to share my experiences with the Proxmox Backup Server and explain why I consider it one of the most promising open-source backup solutions. Especially regarding Docker environments and the backup of applications like Mailcow at hosting providers such as Netcup, PBS has proven to be particularly efficient and reliable.
Chunking
The Proxmox Backup Server uses the chunking method to break down large files into small, efficient pieces, much like a puzzle. These pieces are then deduplicated and compressed to optimize the backup process. The PBS creates a guide to piece these puzzle pieces back together later.
Chunking is a method that splits large files into small, equally sized pieces (chunks), which can then be deduplicated and compressed. The Proxmox Backup Server uses a variable chunk size, automatically adjusted to the size of the data to achieve optimal deduplication. This means that PBS only saves the changes between the chunks, thereby reducing data volume and shortening backup time.
When restoring data, the chunks are reassembled to restore the original file. Since each chunk is provided with a unique hash value, PBS can also ensure data integrity during the backup and restoration process.
In addition to this chunking method, the Proxmox Backup Server also offers support for encryption and compression of backup data to maximize the security and efficiency of the backup process.
For block-based backups, the Proxmox Backup Server uses a fixed chunk size of 4 MiB. This type of backup is usually used for virtual machines and is based on the block level, meaning changes to the entire virtual hard disk are detected and secured.
For file-based backups, however, the Proxmox Backup Server uses a variable chunk size of 64 KiB to 4 MiB to achieve optimal deduplication of data. File-based backups are based on the file level and allow the backup of specific files or directories instead of the entire hard disk.
File-Based Backups
Currently, I use two methods for backups, and I was surprised at how well these backups work in Docker environments. Likewise, in cases where I want to back up the content of a VM, such as our Mailcow at Netcup.
The host backups from PBS run through the CLI tool proxmox-backup-client, which is available from Proxmox. I create a bash script that is triggered by a cron job.
- It pulls data from the source (e.g., from a MySQL server via mysqldump)
- I place this data in a directory like backup
- I run the proxmox-backup-client and specify the directory with the data as the source.
In reality, this would look as follows.
Backup.sh
<span class="hljs-number">1.</span> <span class="hljs-comment"># !/bin/bash</span>
<span class="hljs-number">2.</span> set -e
<span class="hljs-number">3.</span> trap <span class="hljs-string">'curl -fs "https://status.myuptimekuma.de/api/push/Avf1241sdas?status=down&msg=FAILED"'</span> ERR
<span class="hljs-comment"># Run Mysql dump to backup into the backup folder</span>
<span class="hljs-number">4.</span> /root/backup-database.sh
<span class="hljs-comment"># Set env vars</span>
<span class="hljs-number">5.</span> export PBS_REPOSITORY=mailcow@[email protected]:datastore
<span class="hljs-number">6.</span> export PBS_PASSWORD=<span class="hljs-string">"xxxx"</span>
<span class="hljs-number">7.</span> export PBS_ENCRYPTION_PASSWORD=<span class="hljs-string">"xxxx"</span>
<span class="hljs-comment"># Run the Proxmox Backup client</span>
<span class="hljs-number">8.</span> proxmox-backup-client backup database.pxar:/root/database-backups docker-volumes.pxar:/<span class="hljs-keyword">var</span>/lib/docker/volumes mailcow-config.pxar:/root/mailcow-dockerized --ns mailcow --skip-lost-<span class="hljs-keyword">and</span>-found
<span class="hljs-comment"># Report to the monitoring solution (in my case uptime kuma, remove if not needed)</span>
<span class="hljs-number">10.</span> curl <span class="hljs-string">"https://status.meinuptimekuma.de/api/push/Avf1241sdas?status=up&msg=OK"</span>
First off, this article is not sponsored in any way. Becker Software has no affiliation with Proxmox.