Jump to content


Photo

Need Backup Solution for Home Data for years


  • You cannot start a new topic
  • Please log in to reply
16 replies to this topic

#1 nibble123

nibble123

    Member

  • Member
  • 2 posts

Posted 14 July 2014 - 10:50 AM

Hello , 

 

As we all have large home media files and family data of years however , how are we keeping them safe so that we do not lost them.

As per my understanding the requirement is to have a backup solution which is easy and fast to access when required and also the data should not be lost when the hardware is crashed.

I do not want to use Online Cloud Solutions due to below reasons -

       1. I do not want to pay any online service for size of data ( which will keep on increasing with the time) 

       2. Access to cloud is dependent on good internet bandwidth , which is still a challenge in my country.

        3. Moreover , i do not want to move from one cloud vendor to another in future say 5-10 years ahead.

 

I have gone through various forums and was not able to find the correct answer. Do remember its a home solution therefore, it should not also in expensive.

 

Kind Regards,

JR

 


#2 Brian

Brian

    SR Admin

  • Admin
  • 5,389 posts

Posted 14 July 2014 - 10:58 AM

Seems like a multi-bay NAS is what you need. Provides the local redundancy you want and does not have to be expensive. I would suggest backing it up however to a device that can be taken off site (USB 3 drive) or using a cloud service for backup only. 


Brian

Publisher- StorageReview.com
Twitter - @StorageReview

 

#3 anywhere

anywhere

    Member

  • Member
  • 93 posts

Posted 16 July 2014 - 02:28 PM

I'm using a traditional atx case with 6 5.25 bays and a handful of 3.5" internal bays. A couple of 3to4 3.5" enclosures, that has a 120mm fan to chill the 4 drives.

12 3.5 hdds @ 3tb stuffed in that dude. A USB stick 4gb booting Debian and mdadm for managing the arrays. Debian tuned to use swap on a small sdd. No gui. I ssh into it. Garbage onboard video.

Cheap setup. Amd64 @ 2.4ghz. Reads around 600mb writes around 200 sustained. $40 pci4x cards. Flexibility too, Linux doesn't care of chopset, devices, makes, models... ,

Sent from my rooted HTC Supersonic using Tapatalk 2 Pro


#4 anywhere

anywhere

    Member

  • Member
  • 93 posts

Posted 16 July 2014 - 02:43 PM

Can't edit.

Flexible as in you can't switch anything and everything, Linux doesn't care. No driver issues if swapping mobo, ethernet, chipset.

For example swapping an amd setup for a Intel, it's still called cpu0,cpu1,etc, or swapping nics, Eth0,eth1,...

Makes life so much easier. I spilled beer on my mobo, swapped over, array came up, mounted, plex resumed, roku continued playing the 1080. About murders a 100mbit link though. Need to go gigabit soon and a ps3 for a dlna client.



Sent from my rooted HTC Supersonic using Tapatalk 2 Pro


#5 Brian

Brian

    SR Admin

  • Admin
  • 5,389 posts

Posted 17 July 2014 - 09:12 AM

Still thinking NAS is your best path. 


Brian

Publisher- StorageReview.com
Twitter - @StorageReview

 

#6 anywhere

anywhere

    Member

  • Member
  • 93 posts

Posted 18 July 2014 - 05:59 AM

What Nas is available with 36tb under $1100?



Sent from my rooted HTC Supersonic using Tapatalk 2 Pro


#7 anywhere

anywhere

    Member

  • Member
  • 93 posts

Posted 18 July 2014 - 06:06 AM

I needed massive local home data storage, no cloud services, zero Internet. And some type of redundancy. And didn't feel like buying MORE motherboards, cpus, ram, etc etc. Used what I had. Why buy new? Stuff is already outdated even when you do buy the newest latest and greatest. Only investment I needed was platter storage.

Any serious stuff stuff gets copied to a 2nd box with 2x2tb drives in two separate raid1 arrays via mdadm, yes, 4 copies. I'd have all parity and mirroring arrays in one box, but don't have case enough.



Sent from my rooted HTC Supersonic using Tapatalk 2 Pro


#8 Brian

Brian

    SR Admin

  • Admin
  • 5,389 posts

Posted 18 July 2014 - 10:28 AM

You started by asking about an available backup solution...I really can't tell what your ultimate goals are. Maybe someone else less dense than I can help you. 


Brian

Publisher- StorageReview.com
Twitter - @StorageReview

 

#9 chuckleb

chuckleb

    Member

  • Member
  • 15 posts

Posted 19 July 2014 - 12:15 AM

@nibble123: Can you give more details on what you have? How much data, does NAS work for you or something local, etc? There are lots of options depending on how much data you need and what tools.

 

I have a main file server with about 16TB online. I use rsync to rsync important data to an external 4TB that I swap to the bank every couple of months. I also have Crashplan dumping to a locally attached HDD as well as the cloud (optional). 

 

It helps to have more details to think of ideas. Lots of ways to prioritize data as well. Many of my media content I would be annoyed to lose, but not worth it to make multiple copies.


#10 nibble123

nibble123

    Member

  • Member
  • 2 posts

Posted 21 July 2014 - 10:18 AM

@

 

@nibble123: Can you give more details on what you have? How much data, does NAS work for you or something local, etc? There are lots of options depending on how much data you need and what tools.

 

I have a main file server with about 16TB online. I use rsync to rsync important data to an external 4TB that I swap to the bank every couple of months. I also have Crashplan dumping to a locally attached HDD as well as the cloud (optional). 

 

It helps to have more details to think of ideas. Lots of ways to prioritize data as well. Many of my media content I would be annoyed to lose, but not worth it to make multiple copies.

Hi , 

 

I have around 2 TB as of now , which I think will be grown to 10 TB in next 5 years. I am just thinking How people store and safe their home made media files  as obviously that would be easily  more that 2TB which is in my case.

Secondly, In case I have to build a NAS , I would not like to spend more that 250 $ on that.

 

Kind Regards,


I needed massive local home data storage, no cloud services, zero Internet. And some type of redundancy. And didn't feel like buying MORE motherboards, cpus, ram, etc etc. Used what I had. Why buy new? Stuff is already outdated even when you do buy the newest latest and greatest. Only investment I needed was platter storage.

Any serious stuff stuff gets copied to a 2nd box with 2x2tb drives in two separate raid1 arrays via mdadm, yes, 4 copies. I'd have all parity and mirroring arrays in one box, but don't have case enough.



Sent from my rooted HTC Supersonic using Tapatalk 2 Pro

Hi Anywhere,

Can you send me your hardware config and how have you setup it along with cost involved. In case it suits me I will also setup the same way.


#11 chuckleb

chuckleb

    Member

  • Member
  • 15 posts

Posted 21 July 2014 - 10:56 AM

I can see a few solutions that I'd recommend. I wasn't sure if your $250 budget number includes disks or not, but if it does, that makes it hard. I also don't know how you are currently serving them - from a server or just your desktop.

 

1) Manual + cheapest. Buy a set of HDDs and set a calendar reminder every week to copy all your contents over. Swap one to the bank and one offsite. If windows, you can use robocopy or something similar, if Linux, rsync

 

2) Manual + cheapest + automated. Plug in a drive and use Crashplan to automate the backups. I don't recommend swapping the crashplan store though since you may end up missing content. Leave it plugged in. Crashplan is free to backup to a local media.

 

3) All NAS setups will cost you more than $250 unless you have all the parts left over from a previous computer. Well, there may be exceptions, there was a deal at Microcenter for a MB+CPU for $100, then you'd need RAM and disk still.

 

For disks, since these are backups, you could get by with desktop-grade disks. You can get 3TB drives for about $100 bare and an external disk reader for $30 or so. If you wanted to, you could do 4TB white label drives for about $145 as well. 

 

The problem is easier when you're taking about what will fit on one disk. If you have more than 1 disk work of content, then you have to figure out how to break it up or span multiple disks together to get the capacity you need. That's when you really need NAS solutions or something else. Also NAS would give you automated backups so you don't forget vs going the cheap route and setting a calendar event.

 

 

@

 

Hi , 

 

I have around 2 TB as of now , which I think will be grown to 10 TB in next 5 years. I am just thinking How people store and safe their home made media files  as obviously that would be easily  more that 2TB which is in my case.

Secondly, In case I have to build a NAS , I would not like to spend more that 250 $ on that.

 


#12 anywhere

anywhere

    Member

  • Member
  • 93 posts

Posted 21 July 2014 - 12:22 PM

@
 
Hi , 
 
I have around 2 TB as of now , which I think will be grown to 10 TB in next 5 years. I am just thinking How people store and safe their home made media files  as obviously that would be easily  more that 2TB which is in my case.
Secondly, In case I have to build a NAS , I would not like to spend more that 250 $ on that.
 
Kind Regards,

Hi Anywhere,
Can you send me your hardware config and how have you setup it along with cost involved. In case it suits me I will also setup the same way.


Will do when I get home, phone is almost dead.

Sent from my rooted HTC Supersonic using Tapatalk 2 Pro


@
 
Hi , 
 
I have around 2 TB as of now , which I think will be grown to 10 TB in next 5 years. I am just thinking How people store and safe their home made media files  as obviously that would be easily  more that 2TB which is in my case.
Secondly, In case I have to build a NAS , I would not like to spend more that 250 $ on that.
 
Kind Regards,

Hi Anywhere,
Can you send me your hardware config and how have you setup it along with cost involved. In case it suits me I will also setup the same way.


Sent from my rooted HTC Supersonic using Tapatalk 2 Pro



Sent from my rooted HTC Supersonic using Tapatalk 2 Pro


#13 anywhere

anywhere

    Member

  • Member
  • 93 posts

Posted 22 July 2014 - 06:52 AM



I'm using old pcs I found in dumpsters. Amd 64 3800 @ 2.4, 512mb ram.

Some old atx case with a bunch of bays.

450watt juice box.

Enclosures are $21 each.
Uses 3 5.25 to 4x3.5 bays, 120mm fan

Pci express cards were $35 each, using 2, 4 sata2 ports.

Hdds are 3tb cudas. I scored them @ $99.

$3 USB 4gb pen drive.

I have a 128gb ssd I use for swap space and other fast stuff. Backup when growing arrays, external bitmap, etc. . Also wear leveling murders pen drives, ontop of the fact they're slow.

Running basic Debian, with a few configuring tuning. No gui, no desktop environment. Some of the essential packages, all free and available via apt-get.

I started with 2 hdds in degraded raid5. Then every month added 2 more hdds. Mdadm supports growing of arrays. And ext4 supports growing of the file system, that is on the array. It's slick. After 12tb of raid5, I added one more hdd, then converted to raid6, and called it done. It's almost a week to do array checks.

So Then I started building a 2nd array in the same box with the same method. Filling up with data right away, and slowly growing/resizing as needed. Everything can be done online and resize2fs supports online operations (mounted) with ext4.

Cron job every 1st Sunday of month does parity checks of array, and I used tune2fs to change ext4 options for mounting options. Every other mount, or every 30 days, it gets checked.

I've had a few drunk oops and tripped over power cords, zero errors, no array problems, no fsck errors. Flawless. We lost power doing a rsysnc of a 2tb transfer. No issues. Resumed correctly at the last file where it left off.

I have 512mb bitmaps for the arrays too, helps array checks into seconds instead of days, since it logs where it works at as it goes changing data.

All shared via samba for M$, and NFS for my other Linux clients, and a plex for media content (it's super slick), for the roku and ps3, and running owncloud software for my own cloud encryptioned server. Ultra handy when on vacation and gadget stuff is all full.

Best part is I have a clone of the pen drive, when it pukes, kill power, swap the $3 stick, boot, resume. Basically same setup on a slower 2nd box, but just important stuff on a mirror 2 arrays, 2x2tb. Mirrored mirrors. No intensive parity calculations. I Didn't want to have another machine running, but I ran out of case on the big bitch for 4 more hdd spots. Plus I'd have to buy another card for 4 more ports, and this other box I also found in a dumpster, and it has 4 on board sata2. So I used it. Ask anymore questions, I'll answer as I check my account.

Sent from my rooted HTC Supersonic using Tapatalk 2 Pro


#14 Valleyforge

Valleyforge

    Member

  • Member
  • 134 posts

Posted 23 July 2014 - 05:23 AM

Flexible as in you can't switch anything and everything, Linux doesn't care. No driver issues if swapping mobo, ethernet, chipset.

For example swapping an amd setup for a Intel, it's still called cpu0,cpu1,etc, or swapping nics, Eth0,eth1,...

 

Newsflash.  Windows has done this since Windows 7.  I regularly move Win7 installs between machines, and after one reboot everything works properly.


Laptop: Dell inspiron E7440, 8GB RAM, 480GB Crucial M500 mSATA, Win7 Pro
Workstation: i5-4690K, Z97I-Plus, H100i, Obsidian 250D, 480GB Crucial M500, 1TB WD Black, Win8.1 Pro, Hyper-V

NAS: Asustor AS-604T, 3GB RAM, 180GB Intel Pro 1500 & 2x4TB HGST NAS

HTPC:Intel NUC D54250WYK, 4GB, 64GB Crucial M4 mSATA, Win7 Pro

HTPC 2:Intel NUC DN2820, 4GB, 64GB Crucial M4 mSATA, Win7 Pro


 

#15 anywhere

anywhere

    Member

  • Member
  • 93 posts

Posted 23 July 2014 - 06:55 AM

They finally fixed that thing where it tailored itself to the hardware, and wouldn't let that license be used elsewhere?

Maybe that was rumor? Been out of the m$ money scene for 16 years.

Yay, finally they did something useful. I hated it back in the win98 days.

I still like Linux. 200mb installs let's me use these 100 free pen drives, since modern stuff accepts USB boot.

Sent from my rooted HTC Supersonic using Tapatalk 2 Pro


#16 continuum

continuum

    Mod

  • Mod
  • 3,581 posts

Posted 02 August 2014 - 03:09 AM

If you want to do things as cheap as possible, anywhere has it done about that way. All depends how much you value your time and setup.

 

If you only have 2TB now and need a growth plan to 10TB in five years, I honestly wouldn't sweat it very much-- so long as you're within the capacity of a single harddisk or two (which today tops out at 6TB per disk), it's pretty easy to manage your growth in a single box, be it a desktop computer or a NAS.

 

They finally fixed that thing where it tailored itself to the hardware, and wouldn't let that license be used elsewhere?

That's a license type issue with OEM licenses issue-- retail licenses are handled differently. And yeah, I don't think it's been an issue in quite a while.

 

But yeah, if you want to build your own NAS on software RAID, it's hard to argue with Linux and RAIDZ/RAIDZ2...


#17 anywhere

anywhere

    Member

  • Member
  • 93 posts

Posted 02 August 2014 - 06:45 AM

For the record, I started with 3tb, and I've been growing 2tb a month.

I didn't think I'd aquire so much so fast, but once that network share has a little tiny sliver, 200gb on 3tb, I went berserk.

Right Now for non important data, I have 3 arrays, each is 6x3tb in raid5, 18tb/15 usable

/movies
/tvshows
/pics.and.audio

All mounted in /media, so all 3 share as one giant mount @ 45tb

I'm near 80%.

Box duties are plex/nfs/samba/owncloud/luks and of course mdadm the arrays.

Encryption is done on the md0 device, so filesystem automatically is en/decrypted on the fly @ server so clients don't do brute the math.

I can grow/reshape the array, add hdds, change raid level. Then resize the luks partition. Then resize the ext4 partition.

Bamm just grew Xtb transparently. I could do this with 10 hdds at once. Or one drive per array, anyway you want. Just need a lot of case, or fancy SAS rack with external cables.

Mdadm doesn't care what the device is or where it is.

/dev/sd(abcd) and /dev/usb(01) and maybe another array of /dev/md0, to make /dev/md1?

Or make an array out of network shares? Sounds slow. But it'll work.


I love this setup. All on Debian. Running on a mobo/cpu/case I found in a dumpster @ work.


Food for thought, let us know what wild mad scientist stuff you whip together.

Sent from my rooted HTC Supersonic using Tapatalk 2 Pro





0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users