crayfellow

Family/Business Backup Strategy

Recommended Posts

Hi all,

Thanks for this great forum.

I have a pretty common scenario - 14+ years of family data with a spotty backup/sharing strategy, looking to do it right finally.

Here's what we have to work with. I'll give as much detail as I can just in case it is useful.

'client' hardware

* MacBook Pro (my business laptop) 512GB SSD

* MacBook Air (new, wife/kids/homeschool) 256GB SSD

* Various iOS devices

'server'

* Mac Mini (media server in utility room) 256GB SSD (new, replaces an old Mini)

- used to stream video to Apple TV and other iOS, music to various AirPlay, and as an Internet-accessible family web server

storage

* USB2 1.5TB HDD

* USB2 1TB HDD

* USB3 2x4TB RAID1

The most valuable data we have is digital photo/video from SLR's and iOS devices going back 14 years.

What I currently have is:

* MacBook Pro backup via Time Machine to the USB3 2x4TB RAID1

* Current year photo/video on MacBook Pro system drive (backed up only by virtue of Time Machine)

* Photos/Videos up to current year stored solely on the USB3 2x4TB RAID1

* Old Mac Mini backup via Time Machine to the USB2 1TB HDD

Issues include:

* No offsite backup

* No reliable way to access photo/video archive (we make family movies for the kids' birthdays that sometimes include past years). The MacBook Pro is the designated "editing station".

* The RAID1 is both Time Machine and sole storage location for a priceless photo/video archive

To resolve these, here is what I am thinking:

* Purchase a fast (Thunderbolt 2?) DAS for photo/video editing station

- Which one? Considering LaCie 5big, Promise Pegasus R4/R6, and OWC ThunderBay 4. I have seen you guys recommend buying empty chassis and HGST enterprise grade HDD's. Is that still the recommendation?

- Which RAID level? RAID5 to balance performance and some semblance of safety, or RAID0 for maximum performance then clone to a second identical unit as a backup?? HW or SW RAID?

- What capacity?

- Do not quite understand yet what exactly is backed up from this and to where. Just the raw source files?

* Maybe centralize the USB3 2x4TB RAID1 on the Mini and use it as a Time Machine backup of the Mini, MacBook Pro, and the MacBook Air as well as a central place to put business/tax/legal documents. Maybe separate partitions?

* Use the 1TB or 1.5TB USB2 as a clone of the boot drive on the MacBook Pro

* Purchase a cheap ~$30 USB3 HDD dock and 2 or 3 4TB-or-so HDD's to connect to the Mini and use as a rotating offside backup

- what will go on this? from where and can it be automated somehow?

- I think this should be business/tax/legal documents, the photo/video library, and ??

I know these solutions potentially have a significant cost, but I feel like I have a 'debt' of years of putting this off, so I want to pay the debt and move forward with something we'll feel good about.

Thanks in advance for your help guys!

Patrick

Edited by crayfellow

Share this post


Link to post
Share on other sites

My standard reply for this is a NAS, you can do everything you want except the fast local scratch space, which can be done with LaCie solution or other. We like LaCie quite a bit and G-Tech, but they're both expensive which is why the chassis and your own disks is a nice option if you're handy enough to manage it. But with the NAS you get the streaming, Time Machine backup and option to sync to another NAS or web service like Amazon.

Share this post


Link to post
Share on other sites

My standard reply for this is a NAS, you can do everything you want except the fast local scratch space, which can be done with LaCie solution or other. We like LaCie quite a bit and G-Tech, but they're both expensive which is why the chassis and your own disks is a nice option if you're handy enough to manage it. But with the NAS you get the streaming, Time Machine backup and option to sync to another NAS or web service like Amazon.

So even though I have the Mini and 4TBx2 RAID you think I would benefit from a NAS? If so, which one? Alternately this could be a DAS on the Mini - right?

As for chassis+disks, what chassis do you recommend? What workflow/tools would make sense to get the media library sync'd to the network storage while still being accessible on the fast local scratch space?

Edited by crayfellow

Share this post


Link to post
Share on other sites

Your problem with the Mini is it's attached to one machine (no external access) and isn't being replicated anywhere else. You also have more complexity than you need with backup disks for all the other machines which are also not being replicated. Classic issue of what happens when the house burns down? At least the iOS devices are probably using iCloud for backup though you may want to check that for sure.

So I like the NAS to unify all of that stuff. I use Synology, but QNAP is fine too. If you need simplicity more than anything, there are other options, WD has good choices in that case. As to which one, it's a math/performance problem. Sounds like a 4-bay is all you need at this point with 3 or 4TB NAS disks inside.

http://www.storagereview.com/synology_ds415_nas_review

http://www.storagereview.com/qnap_ts451_nas_review

Those two are about the same price or were at least at the time of review.

For moving media around, what kind of files are you dealing with? Pulling them down locally to your machine from a NAS might not be that big of a deal. The NAS units have USB 3 in most cases to when direct attached makes more sense.

Share this post


Link to post
Share on other sites

Your problem with the Mini is it's attached to one machine (no external access) and isn't being replicated anywhere else. You also have more complexity than you need with backup disks for all the other machines which are also not being replicated. Classic issue of what happens when the house burns down? At least the iOS devices are probably using iCloud for backup though you may want to check that for sure.

So I like the NAS to unify all of that stuff. I use Synology, but QNAP is fine too. If you need simplicity more than anything, there are other options, WD has good choices in that case. As to which one, it's a math/performance problem. Sounds like a 4-bay is all you need at this point with 3 or 4TB NAS disks inside.

Yes, the complexity is why I'm looking at centralizing everything. The Mini is in a basement utility room wired directly to the main router so could potentially be made accessible from the outside. I guess I don't understand well enough what sort of external access NAS's provide in order to speak to how the Mini+DAS might do the same.

The iOS devices are backed up to iCloud.
The Synology DS415+ with 3/4TB HGST disks sounds good, set up in the utility room next to the Mini. I worry it would make the Mini redundant, but as an engineer and software/firmware developer, I like having direct access to a central server vs. the "black box" a NAS appears to be, even for media streaming purposes.
How do you like to organize a NAS between "business" files, "family" files, TM backups of various computers, etc.?

For moving media around, what kind of files are you dealing with? Pulling them down locally to your machine from a NAS might not be that big of a deal. The NAS units have USB 3 in most cases to when direct attached makes more sense.

The local (RAID0?) I had envisioned using the LaCie for is the initial place DSLR photos/videos would import to, then likewise where editing (Lightroom/FCP) would happen. I just don't have an ideal workflow sorted to make sure whatever ends up on there gets replicated and likewise backed up. Onto the NAS somehow, right? This would be happening in a different room potentially only accessible to the NAS via the LAN.

thanks,

Patrick

Share this post


Link to post
Share on other sites

On the storage side, I really do like the KISS approach that a NAS provides. Very simple OS designed for only storage. Allows it to be more stable in most cases (fewer things to go wrong), use less power (doesnt need a larger CPU or more RAM for a more complex OS) and generally provides more bang for your buck with more of the hardware costs going towards storage instead of compute hardware.

With systems such as Synology models, they provide SSH access to tinker if you must, but at their core they just run a simple Linux variant. Not really a black box approach, just an embedded system one.

With the Mini, it is most likely a fine system, but internally it can support 1 or two disks max, with everything else being a Thunderbolt or USB3 addon. You could build it into a NAS, but it would end up costing more than just getting the NAS originally.

Share this post


Link to post
Share on other sites

On the storage side, I really do like the KISS approach that a NAS provides. Very simple OS designed for only storage. Allows it to be more stable in most cases (fewer things to go wrong), use less power (doesnt need a larger CPU or more RAM for a more complex OS) and generally provides more bang for your buck with more of the hardware costs going towards storage instead of compute hardware.

With systems such as Synology models, they provide SSH access to tinker if you must, but at their core they just run a simple Linux variant. Not really a black box approach, just an embedded system one.

With the Mini, it is most likely a fine system, but internally it can support 1 or two disks max, with everything else being a Thunderbolt or USB3 addon. You could build it into a NAS, but it would end up costing more than just getting the NAS originally.

Yep - that's all true. It's just that I already own the Mini for business purposes.

Now just to figure out how/what/where/when if I add a Synology NAS as Time Machine for all the machines and as a central storage for the important files and media considering a potential RAID0 for initial media import and photo/video editing.

Seems like in this scenario I wouldn't need the HDD cradle and HDD's going offsite as long as the NAS was backing up to "cloud", S3, Google Drive, etc.?

Share this post


Link to post
Share on other sites

I think you're on the right track now, yes. You can segment the NAS with folders and keep things separate with access restrictions too if needed. Then there's always the "extra" stuff it can do like home surveillance. There are also hundreds (???) of third party apps for Synology and QNAP too that allow for a ton more functionality.

Also, if you don't love the cloud idea you can get a second NAS at another site and sync to it.

Share this post


Link to post
Share on other sites

I think you're on the right track now, yes. You can segment the NAS with folders and keep things separate with access restrictions too if needed. Then there's always the "extra" stuff it can do like home surveillance. There are also hundreds (???) of third party apps for Synology and QNAP too that allow for a ton more functionality.

Also, if you don't love the cloud idea you can get a second NAS at another site and sync to it.

Would you think I'm nuts if I got 8 HGST Ultrastar 4TB HDD's and 2 OWC ThunderBay enclosures? I'd use one in RAID0 as the fast local scratch connected to the MacBook Pro, and the other in RAID1 connected to the Mini as a central backup. Would that be silly since I'd lose the fancy partition management available with a NAS?

It would be kind of nice to have the same type of sled and identical HDD's so I could have extras on hand for either.

What RAID level do you guys typically use on a NAS being used for backup?

Edited by crayfellow

Share this post


Link to post
Share on other sites

Yes you'd be crazy. It wouldnt be anywhere near as reliable as a NAS.

It would also end up costing you more than if you just bought a NAS and have less functionality.

Personally I used RAID1 at home (two disk setup that backs up to a third disk). For more capacity where performance doesnt matter as much RAID6 and where performance does matter RAID10. I completely avoid RAID5 these days.

Share this post


Link to post
Share on other sites

How much critical data do you actually have to backup? For offsite backup, an external drive is easy. Bring it home once a month, run your backup, take it back (to work, to your mother-in-law's house, to a safe-deposit box, etc.) With 6TB hard drives going for under $300, it's a simple solution.

Share this post


Link to post
Share on other sites

Yes you'd be crazy. It wouldnt be anywhere near as reliable as a NAS.

It would also end up costing you more than if you just bought a NAS and have less functionality.

Personally I used RAID1 at home (two disk setup that backs up to a third disk). For more capacity where performance doesnt matter as much RAID6 and where performance does matter RAID10. I completely avoid RAID5 these days.

I think that's precisely what I am missing. I missed the NAS bandwagon so I am clueless as to what I'd be missing. What would I need that I don't already get backing up to a simple 2-HDD RAID1 connected directly to the Mini, since it's already doing all the fancy media streaming, web serving, etc.? Or a 3- or 4-HDD enclosure to cover the scenario you mention, 2xHDD in RAID1 and a third for backup. I am sure I am missing something major. I also can't see how the NAS is cheaper.

I would imagine whatever it is would be partitioned to set aside, say, 2TB as Time Machine for the Mini (256GB SSD), MacBook AIR (256GB SSD), and MacBook Pro (512GB SSD), then leave the rest for the "other": the photo/video libraries, business/legal/tax, etc.

How much critical data do you actually have to backup? For offsite backup, an external drive is easy. Bring it home once a month, run your backup, take it back (to work, to your mother-in-law's house, to a safe-deposit box, etc.) With 6TB hard drives going for under $300, it's a simple solution.

It's pretty funny, I checked the DSLR photo/video libraries and they only consume 550GB as of today. But going forward I want to be much freer to capture the (4) kids, create media for our business, etc. as well as include photo/video coming from other devices such as our phones so all can easily be used in the editing process.

All together I would think 4TB should be sufficient. For offsite something like a HGST Deskstar should be OK, no?

So using Kevin's example, if I had a 2xHDD RAID1 and a third drive to back that up for offsite, assuming I have sufficient space, I'd be pretty close, no?

Then the "fast access" system could be a Thunderbay, LaCie 5big, Pegasus, or whatever. Yes, for that purpose an 8-bay in RAID6/10 would be pretty awesome and great performance/capacity/protection balance, but that's really getting up there in cost.

Edited by crayfellow

Share this post


Link to post
Share on other sites

The NAS I recommended are $600 or so bare, so if you can do it all on a NAS, you buy that and one set of disks, rather than two sets of disks in your proposed DAS model.

Share this post


Link to post
Share on other sites

The NAS I recommended are $600 or so bare, so if you can do it all on a NAS, you buy that and one set of disks, rather than two sets of disks in your proposed DAS model.

Sure - that would make the most sense by a long shot if it were fast enough for video editing and directly connected as opposed to in the basement by the network stuff.

How do you guys feel about the OWC Thunderbay 4 and dependency on SoftRAID for RAID5? This would be an inexpensive way to go for an photo/video edit station vs. 5big, Pegasus, etc. where I'd still want to have a spare disk on hand (right?).

Share this post


Link to post
Share on other sites

For fast local storage, use SSD. Push finished projects and raw files off to the network file server. To a large extent, it sounds like that's what you're doing now. Is the problem that you don't have sufficient storage on the server, or that you need more space on your workstation?

I think the NAS recommendation above is for the role of networked storage. Personally, I'm not a fan of consumer NAS devices that you typically buy for under $800. They're neat, simple solutions for most people, but their performance and value is poor. I'd just as soon put together a server with more CPU power and more memory for the same amount of money. That, or spend 3-5 times as much for a NAS with some power and expandability.

Share this post


Link to post
Share on other sites

We haven't looked at anything from OWC in some time, mostly we just don't have time. The gear from them we've seen before though is generally pretty good and a nice alternative to the more expensive branded solutions. Also have had good experience with CalDigit.

Share this post


Link to post
Share on other sites

For fast local storage, use SSD. Push finished projects and raw files off to the network file server. To a large extent, it sounds like that's what you're doing now. Is the problem that you don't have sufficient storage on the server, or that you need more space on your workstation?

I had in mind that SSD was still too costly, but considering the investment we have been looking at from a macro level, that's a great suggestion. What would you recommend?

The problem today is we have a big place for the libraries, big enough to hold the raw media and the project work, but because it's also serving as the "first line of defense" backup it is RAID1 and thus not fast enough for editing. That one RAID1 is trying to do it all.

So what I had in mind was one place that was FAST and big enough for the initial media import, editing, and even to hold the whole library, then some place on the network where the raw files are backed up. Depending on how often they are backed up, this could be RAID0 (back up immediately) or maybe RAID5.

The other problems are (1) I don't understand FCP well enough to know what can be backed up vs. the working projects and (2) I like the idea of editing from a place that has access to our entire media library, FCP for video and Lightroom for photos. The workflow is the big disconnect. So if the actual media ARCHIVE were on the network, could you see a FCP workflow where it could browse raw files from there, have a local scratch with proxy files for all the media that is actually on the network share, and still essentially have access to "everything" without it having to be on the local drive?

If that sounds like it could work, then I could put a big fast-ish RAID5 (Thunderbay 4 with SoftRAID?) on the Mini, access it over the network, and use an external SSD as the scratch as you suggest. In that scenario, maybe 1TB would be enough? The workflow is definitely the disconnect for me.

I think the NAS recommendation above is for the role of networked storage. Personally, I'm not a fan of consumer NAS devices that you typically buy for under $800. They're neat, simple solutions for most people, but their performance and value is poor. I'd just as soon put together a server with more CPU power and more memory for the same amount of money. That, or spend 3-5 times as much for a NAS with some power and expandability.

As a UNIX guy myself, yeah, they're Linux PC's with software RAID with a nice UI. A really neat idea, but I would lean toward something simpler using the infrastructure I already have as much as possible (like the Mini) which is already doing everything I need except for sane storage/backup.

Thank you for your time guys!

Share this post


Link to post
Share on other sites

For fast local storage, use SSD. Push finished projects and raw files off to the network file server. To a large extent, it sounds like that's what you're doing now. Is the problem that you don't have sufficient storage on the server, or that you need more space on your workstation?

if I can figure out a workflow where I can have access to "everything", but the raw media can still live on the network, something like a Thunderbay 4 Mini ($379) 4x480GB OWC Mercury Extreme 6G SSD ($379 each) would give close to 2TB all SSD for less than $1900.

What would you think of using something like that in a RAID0, then using spinning disks on the network for backups and to hold all the raw media?

Edited by crayfellow

Share this post


Link to post
Share on other sites

I'm no authority on such products, especially in a Mac environment, so I couldn't really say. Do you really need 2TB locally? You can get 1TB SSDs for under $500.

From everything that I've read, using RAID0 with SSD gives you only marginal gains in speed in exchange for the increased expense and data insecurity. Made more sense (purely from a performance standpoint) with HDDs, IMO.

Share this post


Link to post
Share on other sites

I'm no authority on such products, especially in a Mac environment, so I couldn't really say. Do you really need 2TB locally? You can get 1TB SSDs for under $500.

From everything that I've read, using RAID0 with SSD gives you only marginal gains in speed in exchange for the increased expense and data insecurity. Made more sense (purely from a performance standpoint) with HDDs, IMO.

OK, that makes sense. I wasn't sure of the reliability of SSD vs HDD's.

I really don't know how much space I need. I will have to look at SSD performance numbers to (for instance) have a single external 1TB SSD or Thunderbay 4 Mini with a single 1TB unit for future expansion.

Share this post


Link to post
Share on other sites

For what it's worth, there are few ready made SSD portable storage solutions. That will change next week though with a more value centric offering from a major brand.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now