Hi All,

Ultra-nerdy post occasioned by some Twitter discussion about backup strategies. I’ve been in the business nearly 20 years and have had to transition my backups to new hardware many times. So far (touch wood) I have not lost anything.

I thought it might help newer producers and especially model-producers who aren’t used to having to do digital imaging tech jobs the way that most webmasters are.

Basic Principles

1. You don’t have it until you have three copies of it.
2. Don’t delete it from the card until you’ve backed it up at least twice.
3. One of the copies has to be physically separate from the others. If your laptop gets stolen, it does you no good if your backup drives all went with it.
4. Figure out what you can afford to lose and plan accordingly.
5. You have to have a routine.

Three Copies

Hard drives fail often (see https://www.backblaze.com/b2/hard-drive-test-data.html) about 2% of drives used by industrial backup company Backblaze fail per year. If that 2% includes your single 8 TB external drive with everything you’ve ever shot on it… YOU ARE FUCKED.

Want to start selling all your old clips on a new store system? You’re out of luck.

At least in fetish porn, there’s a LONG sales tail – stuff I shot two decades ago still sells and a lot of the profit I generate comes from these steady sales of older clips, since the costs are all paid any sale is pure profit.

That’s why everyone needs at least three copies of your work. Hard drives are cheap compared with the lost sales or the cost of production.

Don’t be cheapskate. Buy an extra hard drive or three. Think how many hours of work it took to create the data. It quickly adds up to tens or hundreds of thousands of pounds, and many work months or years. Suddenly £100 here and there on more hard drives sounds like a fucking BARGAIN considering how much it cost to shoot what’s on there.

Don’t delete it from the card until you’ve backed it up twice

Your footage is at its most vulnerable when there’s just the one fragile, easy to reformat, easy to drop down the loo camera card. Don’t leave it un-backed-up for a second longer than you have to.

Newer cameras are starting to have a second SD card slot so you can record two copies simultaneously in case a card fails. Follow that maxim- get a second backup as soon as you can. Don’t just read the footage in to your laptop – copy it to an external hard drive immediately too. Two external hard drives is better.

I take small, cheap, external portable drives with me on location shoots because I have to format the cards during the week. Two or three £50 drives might be all that stands between me and losing a the footage from a trip that took thousands of pounds and many person-hours to put together.

Cards are cheap – buy more, I’m doing this now the fast ones have come down in price so hopefully next location trip I won’t even have to wipe the cards during the week.

One copy offsite

If your house burns down, that’s grim. If you can afford it, you’ll be insured. But if your house burns down with the only copy of years of filming on it, you might also be out of business. And if someone steals your laptop on the road, make sure all the copies of yesterday’s shoot aren’t with it.

There’s a limit to what you can do here but tucking one of your portable external HD copies in your suitcase or your car glove box rather than having them all in the laptop bag is worth doing.

When you get home, the same principle applies. But how do you get that additional off-site copy?

Hard drive in a friend’s house works well. (Encrypt the hard drive so said friend can’t pirate your stuff even if they were so untrustworthy as to try). I did this for ages, swapping a big box of hard drives with my ex-wife who lived around the corner (and who is my best friend).

It’s a pain, though. What about a more sensible and automated strategy?

Raw images and unedited video footage are huge. Backing everything up to a cloud provider would be prohibitively expensive for 20 years of raw footage (it comes to 100 TB or so for me!) More importantly the bandwidth of domestic internet connections means it would take a decade to upload.

So you need to be smarter.

What can you afford to lose?

This is the bit of analysis I’ve not seen much in generic “back up everything you have” advice by data security companies. I’ve got more data in my study than many medium-sized enterprise computer companies, but I don’t have the budgets to match. We generate huge data sets which we have to deal with using cheap consumer methods.

Work out how much each class of data is worth to you. Concentrate on backing up the high value stuff. For me, this goes as follows:

1. Unedited footage and unprocessed images. This is highest of high value because I have invested all the time and money into shooting them but not got any return yet. That’s why you need to be ultra-paranoid of data straight out of the camera: you can’t afford to lose it, especially once everyone has been paid and gone home.
2. Finished clips and edited photos, in the best quality possible. Let’s call this “archival quality”. This stuff is useful to hold on to because in future you might wish to make a more compressed version is another format. I always keep a “master copy” of edited videos in full resolution, using a high bit-rate codec like Apple ProRes. If you’re shooting in 4K but selling clips in 1080p HD, you’re throwing away the opportunity to sell in 4K in two years’ time if everyone now demands it. Ditto if you kept WMV copies, and everyone now prefers MP4, you’ve lost resale value. Export very high quality JPEGs of images in full resolution too. When I started, 1 megapixel was a huge image. Now people like to download the full 42 megapixels. I wish I’d exported full resolution JPEGs of my earliest images. I’ve frequently gone back to my archival edited copies to export in a new format when fashions change; we’re probably all going to swap to HEVC/H.265 formats in the next few years and people will increasingly be wanting 4K. It’s easy enough to be prepared, it just takes some disk space.
3. “Sale-quality” images and clips. This is the heavily-compressed full HD MP4 files and JPEGs you’re probably selling today. These are your actual products, and you MUST at least back these up somewhere so when “NewClipsStoreEveryoneLoves.com” opens up you can upload your back catalogue. On the plus side, you might have multiple off-site backups of these already if you use something like the FTP interface to Clips4Sale – you could download all your old clips from there if you needed to. So long as C4S haven’t gone bust of course, so don’t count on it one hundred percent, have another independent offsite backup as well.
4. Raw footage for sets you’ve processed and put on sale. It’s vaguely worth hanging on to this if you like to allow later reprocessing, but I’ve hardly ever done that in two decades. So the raw footage goes from being the most valuable data on your hard drive to the least valuable at the point where you’ve exported the video and put it on sale.
5. Maintaining your edits. Saving things like Final Cut or Premiere Pro projects or Lightroom catalogues. In my experience I might go back to an edit after a few days because someone pointed out we left in an “action/cut” call, but beyond that, I never re-edit stuff. I keep a copy for video out of paranoia, but strip out large files like proxies. I don’t bother keeping anything for stills, not least because the skin smoothing software I use is works on rendered pixels not parametrically like Lightroom. So just high quality final JPEGs is OK for stills. I could save 16 bit TIFFs but they are huge so I don’t.

It’s that change in the value of the raw footage that makes our backup strategy different from a lot of other businesses.

We’ve got very high value unprocessed raw data. Very high value sale-quality data that we’re about to put on sale. High value sale-quality data which has been on sale a while. Medium value archival quality data that we might want to strike a new format export from one day. Then low value old raw footage and projects for films we’ve already edited an exported.

To keep costs sane and to automate the process, I use a cloud backup services for my offsite backups these days but I only back up the unprocessed raw data and the sale-quality data. I want to be able to stretch to the archival “master” video files one day, but right now my bandwidth won’t cope with 60+ GB a week.

When you run out of space on your current backup hardware, which you will because file sizes keep growing, migrate your unprocessed data first, then your sale-quality data, then your archival data and only back up the old raw footage and projects if you think there’s any chance of going back to them to re-edit later.

I keep that stuff on general principles, but I’m relaxed about not having offsite backups of the old raw footage in particular.

Routine

Automated backups which happen without you remembering are much better.

I use Carbon Copy Cloner to keep my local backups up to date and Backblaze to automatically backup to unprocessed stuff and sale-quality stuff to the cloud. The only thing I trigger manually is a “push” of data from my working disk to my archival RAID arrays once I’ve offloaded the cards and organised photos and videos into sets at the end of shoot day. Then the archival RAID automatically backs up to a second one daily in the small hours.

My Advice

I hope this has helped a bit to think about your backup strategy. Think about how much of a disaster it would be to lose some of your footage at different parts of your workflow and plan accordingly. £50 spent on another hard drive might literally save you thousands of pounds of lost work and lost sales.