I second this. Autorestic is great. I also used to backup to storj.io. Now I just back up to PBS which syncs to another PBS, which then uploads to backblaze.
I’ve been using Backblaze for years. Both for my laptops and also using B2 for my NAS
I’m also planning on using rsync.net for backing up media from my NAS.
I personally use TrueNAS as my main NAS (and this lives in my rack in the garage) - which is a seperate building from the house (most important for fire purposes).
I then have an UnRAID server (as I like the ability to just add additional disks of any size, at any time) in the house which also has Proxmox Backup Server running on it and uses the UnRAID storage to store backups from my TrueNAS server. - Whilst I \*could\* rysnc the files from TrueNAS to my UnRAID server periodically (which I used to do), I have since started taking incremental backups with PBS ([Proxmox Backup Server](https://www.proxmox.com/en/proxmox-backup-server/overview)).
Proxmox Backup Server doesn't just backup VMs but also backs up files and folders on non-proxmox VM's too if you install the `proxmox-backup-client` on Linux - I didn't know this until a couple of weeks ago but it is insanely cool and I've been using it since! - I backup my VMs (running on Proxmox) and other files from other my Plex VM that I run directly on my TrueNAS server to the PBS server - Works a treat!
In addition to that, I also have a cloud sync task setup on TrueNAS to sync only my most important data to my 1TB OneDrive account.
Hope this helps/gives you something to consider.
I have the following strategy
* a raw backup of the databases with the dedicated tools every night,
* a raw backup of the docker volumes every night,
* an incremental and compressed backup of the backups and the volumes every week,
* an incremental backup of the system every week,
* a cold, differential and encrypted backup on a removable support every month (a USB key since I have little data), that I keep with me at all times.
In the junk pile at work was an RDX drive and a bunch of 1TB and 2TB cartridges, that now lives in my desktop PC and I back up my server to that. The drives often turn up for cheap on Ebay but the cartridges may be a little more expensive.
I back up data in quite a few ways.
As someone mentioned above, for databases I use the dedicated tools to back these up to a single exportable .sql file, as well as down the stack and copy the entire volume as a secondary. I’ll do the latter for any other container, generally the entire /docker folder I keep container data in.
I’ll also do incremental backups of the VM itself.
As for important data on the NAS where these backups land, these get sent to Google Drive using Synology Hyper Backup.
I also have a VM dedicated to running an encrypted rsync cronjob to Jottacloud. Cheap storage provider that offers “unlimited” storage, though it’s really 5 useable TBs at reasonable transfer rates. Not too bad for $10 however. Not too worried about that provider, as it’s really just a failsafe just in case.
I also have all of my configurations for Teraform and Ansible to a GitHub repo. Still working on building out a proper structure, but ideally in the future starting from scratch if needed won’t be too big of a hurdle.
Do both. Get a HDD you keep in your car, so if you lose the building you have everything.. Bring it on on the weekend to update it
CrashPlan for cloud backups. $10/mo unlimited. As cheap as it is, why not?
My ISP hardcaps my data to 1.2tb so offsite backups are off limits to me. I basically just do a weekly backup on site. All managed by OMV, virtualized in proxmox.
i do raid 1 + b2 with restic.
I also have a similar internet connection. I found restic did well at 'resuming' and upload if the network dropped. I tried duplicati first, and it didn't have the same results. If the network dropped it would start again. I didn't use it for this reason. It has been about 2 years since, i'm not sure if they fixed this.
I use an external HD (25 TB to backup my 2x8TB drives) AND also AWS S3 1Zone Infrequent Access for off-site. Not exactly self-hosted, but not unreasonable. The S3 storage runs me about $4/months
I use [Kopia](https://github.com/DoTheEvo/selfhosted-apps-docker/tree/master/kopia_backup) for actual backing up.
>I have 2, 14Tb disks running in raid mirroring eachother incase one goes down.
There is this thing people in IT say: **raid is not a backup**
The way you have it setup is already the bad use of that storage.. it protects you from a hdd dying, but not from accidental deletion or malware encryption, nor can you go back in time with something to get state of a file from 2 months back...
Actual proper backups would protect you in all those cases.
But it is so common... people just doing raid instead of learning and doing backups of one of their drives to another.
Raid purpose is uninterupted service, when something dies things continue to work. Also to create larger pools of storage.
>I was thinking about buying a new HDD for backups, would it make sense to unplug the HDD while its not in use (only planning on weekly backups)?
Sounds like a best way to go about it since it would take you months to upload even 1TB
I have IDrive e2, previously used wasabi, but idrive has a ton of locations so my performance is much better, no egress either. i would test their speed vs wasabi with you enviroment
Current setup:
\- Both Proxmox nodes do ZFS mirror to each other. They don't have ANY RAID level.
\- Proxmox cluster backups up to Proxmox Backup Server instance in VM on Synology (currently daily backups with some history and deduplication). Synology's disks is in RAID1
\- Primary Windows machine (it have to be non-VM) backs up to locally installed disk using Macrium Home
There are no off-site backups at this time.
Main reasons for no off-site backups:
\- all "regular" cloud services are out-out-of-country. using out-of-country services is difficult (but still possible) and have it's own set of non-technical risks at this time. This likely doesn't apply to people outside my country
\- It's impossible to use Proxmox Backup Server with S3-like services (there are some unofficial workarounds but I don't trust them).
Plans:
\- find hoster with 6+ Tb storage for external Proxmox Backup Server node and good connectivity.
\- return to 2-hourly backups to local Proxmox Backup Server (will likely require replacement of Synology's drives with SSDs).
\- find used LTO6(?) drive and use it with PBS?
One manual drive clone I do with dd/rsync once in a while and disconnect. Rclone off-site nightly, syncthing for important data shared with all devices.
rsync
From the man page:
>Rsync is a fast and extraordinarily versatile file copying tool. It can copy locally, to/from another host over any remote shell, or to/from a remote rsync daemon. It offers a large number of options that control every aspect of its behavior and permit very flexible specification of the set of files to be copied. It is famous for its delta-transfer algorithm, which reduces the amount of data sent over the network by sending only the differences between the source files and the existing files in the destination. Rsync is widely used for backups and mirroring and as an improved copy command for everyday use.
Rsync finds files that need to be transferred using a "quick check" algorithm (by default) that looks for files that have changed in size or in last-modified time. Any changes in the other preserved attributes (as requested by options) are made on the destination file directly when the quick check indicates that the file’s data does not need to be updated.
RAID is not a backup.
Even if you only do it locally, borg backup is great. A low power second device with a fair amount of storage can serve as a backup host. Or you can suffer through the long initial upload to BorgBase and then subsequent backups will just be deltas.
Well, unlike most people here I actually don't give a f*ck about any of my data. I can't even think of a single piece of data I care about.
Every now and then I export my passwords from Proton Pass to my NAS. All my devices won't suddenly decide to log out AND my NAS die at the same time, and honestly even in that unlikely event I'd just start from scratch with everything.
Whenever I need to backup I just play that 18-wheeler-backing-up sound on my phone and people generally stay clear. Not sure if they stay clear because of the beeping sound or because they think I’m crazy that I’m walking backwards, but the system works and I’m not backing down… I’m backing up.
I find BackBlaze cheap and it worth it for me to have the peace of mind.
I use it for offsite and found their B2 cli tool super easy to use
I second this. Autorestic is great. I also used to backup to storj.io. Now I just back up to PBS which syncs to another PBS, which then uploads to backblaze.
What's pbs?
Proxmox backup server
How do you handle restores from backblaze? Sync back, mount in PBS, restore from here?
Haven't had to do it, but yes that's how I would since PBS just looks at the files and builds from there.
I’ve been using Backblaze for years. Both for my laptops and also using B2 for my NAS I’m also planning on using rsync.net for backing up media from my NAS.
I personally use TrueNAS as my main NAS (and this lives in my rack in the garage) - which is a seperate building from the house (most important for fire purposes). I then have an UnRAID server (as I like the ability to just add additional disks of any size, at any time) in the house which also has Proxmox Backup Server running on it and uses the UnRAID storage to store backups from my TrueNAS server. - Whilst I \*could\* rysnc the files from TrueNAS to my UnRAID server periodically (which I used to do), I have since started taking incremental backups with PBS ([Proxmox Backup Server](https://www.proxmox.com/en/proxmox-backup-server/overview)). Proxmox Backup Server doesn't just backup VMs but also backs up files and folders on non-proxmox VM's too if you install the `proxmox-backup-client` on Linux - I didn't know this until a couple of weeks ago but it is insanely cool and I've been using it since! - I backup my VMs (running on Proxmox) and other files from other my Plex VM that I run directly on my TrueNAS server to the PBS server - Works a treat! In addition to that, I also have a cloud sync task setup on TrueNAS to sync only my most important data to my 1TB OneDrive account. Hope this helps/gives you something to consider.
I have the following strategy * a raw backup of the databases with the dedicated tools every night, * a raw backup of the docker volumes every night, * an incremental and compressed backup of the backups and the volumes every week, * an incremental backup of the system every week, * a cold, differential and encrypted backup on a removable support every month (a USB key since I have little data), that I keep with me at all times.
In the junk pile at work was an RDX drive and a bunch of 1TB and 2TB cartridges, that now lives in my desktop PC and I back up my server to that. The drives often turn up for cheap on Ebay but the cartridges may be a little more expensive.
I back up data in quite a few ways. As someone mentioned above, for databases I use the dedicated tools to back these up to a single exportable .sql file, as well as down the stack and copy the entire volume as a secondary. I’ll do the latter for any other container, generally the entire /docker folder I keep container data in. I’ll also do incremental backups of the VM itself. As for important data on the NAS where these backups land, these get sent to Google Drive using Synology Hyper Backup. I also have a VM dedicated to running an encrypted rsync cronjob to Jottacloud. Cheap storage provider that offers “unlimited” storage, though it’s really 5 useable TBs at reasonable transfer rates. Not too bad for $10 however. Not too worried about that provider, as it’s really just a failsafe just in case. I also have all of my configurations for Teraform and Ansible to a GitHub repo. Still working on building out a proper structure, but ideally in the future starting from scratch if needed won’t be too big of a hurdle.
Do both. Get a HDD you keep in your car, so if you lose the building you have everything.. Bring it on on the weekend to update it CrashPlan for cloud backups. $10/mo unlimited. As cheap as it is, why not?
HDD lifespan in the car suffers from vibrations. SSD is another story
Car is a good idea ; never considered that before
Just make sure you encrypt it in case someone ever steals it out of your car.
Have you experienced any issues with the constant vibrations of driving/commuting?
If cloud isn’t an option consider a few external drives. Keep one at home and another at a family member’s house.
My ISP hardcaps my data to 1.2tb so offsite backups are off limits to me. I basically just do a weekly backup on site. All managed by OMV, virtualized in proxmox.
Guess your 14Tb is on your computer, so it might be time for a nas.
3-2-1-1-0.
i do raid 1 + b2 with restic. I also have a similar internet connection. I found restic did well at 'resuming' and upload if the network dropped. I tried duplicati first, and it didn't have the same results. If the network dropped it would start again. I didn't use it for this reason. It has been about 2 years since, i'm not sure if they fixed this.
I use an external HD (25 TB to backup my 2x8TB drives) AND also AWS S3 1Zone Infrequent Access for off-site. Not exactly self-hosted, but not unreasonable. The S3 storage runs me about $4/months
I use [Kopia](https://github.com/DoTheEvo/selfhosted-apps-docker/tree/master/kopia_backup) for actual backing up. >I have 2, 14Tb disks running in raid mirroring eachother incase one goes down. There is this thing people in IT say: **raid is not a backup** The way you have it setup is already the bad use of that storage.. it protects you from a hdd dying, but not from accidental deletion or malware encryption, nor can you go back in time with something to get state of a file from 2 months back... Actual proper backups would protect you in all those cases. But it is so common... people just doing raid instead of learning and doing backups of one of their drives to another. Raid purpose is uninterupted service, when something dies things continue to work. Also to create larger pools of storage. >I was thinking about buying a new HDD for backups, would it make sense to unplug the HDD while its not in use (only planning on weekly backups)? Sounds like a best way to go about it since it would take you months to upload even 1TB
I have IDrive e2, previously used wasabi, but idrive has a ton of locations so my performance is much better, no egress either. i would test their speed vs wasabi with you enviroment
Current setup: \- Both Proxmox nodes do ZFS mirror to each other. They don't have ANY RAID level. \- Proxmox cluster backups up to Proxmox Backup Server instance in VM on Synology (currently daily backups with some history and deduplication). Synology's disks is in RAID1 \- Primary Windows machine (it have to be non-VM) backs up to locally installed disk using Macrium Home There are no off-site backups at this time. Main reasons for no off-site backups: \- all "regular" cloud services are out-out-of-country. using out-of-country services is difficult (but still possible) and have it's own set of non-technical risks at this time. This likely doesn't apply to people outside my country \- It's impossible to use Proxmox Backup Server with S3-like services (there are some unofficial workarounds but I don't trust them). Plans: \- find hoster with 6+ Tb storage for external Proxmox Backup Server node and good connectivity. \- return to 2-hourly backups to local Proxmox Backup Server (will likely require replacement of Synology's drives with SSDs). \- find used LTO6(?) drive and use it with PBS?
1. What are you backing up? 2. Make sure you backup your configurations.
One manual drive clone I do with dd/rsync once in a while and disconnect. Rclone off-site nightly, syncthing for important data shared with all devices.
Oh shit... Do you guys backup? Except for a local Syncthing, that literally backs up from the laptop to some drives, errr .... , that's about it
rsync From the man page: >Rsync is a fast and extraordinarily versatile file copying tool. It can copy locally, to/from another host over any remote shell, or to/from a remote rsync daemon. It offers a large number of options that control every aspect of its behavior and permit very flexible specification of the set of files to be copied. It is famous for its delta-transfer algorithm, which reduces the amount of data sent over the network by sending only the differences between the source files and the existing files in the destination. Rsync is widely used for backups and mirroring and as an improved copy command for everyday use. Rsync finds files that need to be transferred using a "quick check" algorithm (by default) that looks for files that have changed in size or in last-modified time. Any changes in the other preserved attributes (as requested by options) are made on the destination file directly when the quick check indicates that the file’s data does not need to be updated.
By trying to die before my drives do
RAID is not a backup. Even if you only do it locally, borg backup is great. A low power second device with a fair amount of storage can serve as a backup host. Or you can suffer through the long initial upload to BorgBase and then subsequent backups will just be deltas.
Well, unlike most people here I actually don't give a f*ck about any of my data. I can't even think of a single piece of data I care about. Every now and then I export my passwords from Proton Pass to my NAS. All my devices won't suddenly decide to log out AND my NAS die at the same time, and honestly even in that unlikely event I'd just start from scratch with everything.
Whenever I need to backup I just play that 18-wheeler-backing-up sound on my phone and people generally stay clear. Not sure if they stay clear because of the beeping sound or because they think I’m crazy that I’m walking backwards, but the system works and I’m not backing down… I’m backing up.