Right now I’m creating a nightly restic snapshot of my server on borgbase and on some spare drives, and keeping the snapshots forever. There’s a ton of daily churn in my data though, between databases and logs and other files that don’t constitute ‘real data’ but might be needed to restore the app.
Always last daily, weekly, monthly. The others are pointers (hardlinks). How many years, i’m not there yet.
For configs I use Home Manager on a public Git repo. And for everything else I use Pika Backup.
I know I’ll get shot for this, but I stopped backing up a long time ago.
I was doing it at home because I do it at work and it’s the done thing. But when I stopped to think about it, I realised I don’t have anything at home that actually matters. If I lose my Plex library it’s ok.
For frequently changing data hourly up to 24, daily up to 7, weekly upto 4, monthly up to 12.
For daily changing data daily up to 7 and etc. as above.
For seldom changing data a backup every 3 days with rotation based on a storage quota, and retention of manually held samples.
Backups?
Daily for 10 days, 4 weekly, 4 monthly.
That covers me from broken updates, generally.
I was backing up multiple places on my LAN but than half of my place flooded, so lamenat and carpet was ripped up, one of my pi 5’s was misplaced with a NVMe drive and another was lost. Than just down the street 3 house went up in flames had the wind been blowing just a little south my condo complex would have been burnt to the ground. After the fire fighters were telling us to get back further and further I had my main pi 5 that had all the backups of everything important on another NVMe drive and a few other things ready to go if the firefighters started banging on my door telling me it was time to leave. I wish I could honestly say that was the day I decided to get a VPS and start my back up, but I was still hoping my father would let me set up a pi with a drive big enough to do back ups at his place. It sounds like he will but he is concerned about security risks about my VPN, funny thing is my back up pi would be the most secure thing hooked up to his modem/router and I wouldn’t want his stuff touching my stuff and with how my LAN and VPN are set up they would have a hard time doing it. Anywho I have a domain hosted by EasyDNS, and just went and looked at their VPS options and got some year long plan and set up cron jobs to take the stuff I want to backup and use age to encrypt it and than it is sent to my VPS each age container is than deleted after there is something of the same name but 3 days iterations older and deletes it.
How long do you keep your backups?
Yes, I have that exernal HDD lying around somewhere near the PC, and it was at least 4 or 5 times that I backed up a lot of files on it.
I think I keep it for more than 10 years now.
7 backups, though the frequency they’re taken (daily Vs weekly Vs monthly) depends on the service.
Why don’t you exclude logs from your backups? databases sure yes that’s where the real info is (not sure why you don’t consider it real data) but logs are useless outside of the current running instance.
I didn’t really want to poke through each containers files to figure out where it stores logs, plus it adds risk that I’m not backing up something I should.
When I say databases aren’t real data, I’m more talking about them changing every day despite nothing happening. Even if I haven’t taken any photos or opened the app, today’s db dump of immich is not the same as yesterday’s db dump so restic stores it as a new unique file.
I did mean to ask, why restic into Borg instead of just Borg?
on the logs thing that’s fair enough, most of the stuff I use has a clear “logs” folder I can exclude, and I checked what the backup contains Vs what the container data folder has before calling it a day
I’m not doing any sort of backup conversion. Borgbase is a website that focuses on hosting borg backups, but they also natively support restic. I chose restic out of convenience for my local backups because it came preinstalled on truenas and figured it made sense to use the same for my cloud backups.
As of now, i keep them basically for ever, but granularity gets less over time. (restic forget)
Before you go nuts about files you want to exculde the “churn” as you said it, go ahead and actually check how much space this actually takes up. Text and especially logs are highly compressible and will most likely end up taking up almost no space.
Be sure you backup everything you actually need, rather backup a file more then a file too less.
Comparing daily snapshots I see 100mb+ daily churn (I assume restic diff is pre compression), the borgbase repository space usage chart shows it increasing about 50mb daily.
I just buy new HDDs.
Unironically how I use LTO-6 tapes.
2.5TB of storage for ~$5/tape on ebay.
Never delete anything. If the authorities ever come after me, discovery is going to be hilarious.
What’s the catch, why don’t we use those? I/O speed? Limited read/write cycles? I imagine it sucks if you need files on opposite ends of the tape.
The drives are crazy expensive if you want to use a somewhat recent generation.
My local backup drives still have room (it’s my main pool with all bluray backups that’s begging for more space), but borgbase is approaching the point where I’ll have to bump to a higher tier or pay per gb. Looking at my usage chart every day adds at a minimum 50mb so something has to change.
Removed by mod
My VPS gets a weekly rsync to my home server. I keep ten deduplicated copies for weeks gone by. The current week also gets synced off-site with encryption.
Removed by mod
[forget] keep-daily = 10 keep-weekly = 4Daily backups are cheap because of de-duplication. I keep a few weekly backups in case it takes me a bit to notice something went wrong. I don’t usually back up logs.
I could probably drop the weekly backups without much risk, but I’ve got the space.