[Axle Gear]

Hello! I'm planning to set up an external security camera to tape an area, and want to have a PC set up that can save that much data onto a single virtual drive. It needs to be 720 hours long, so we only have to change it once a month, and can back it up to some sort of long-term storage later after sorting it. (Most would just be deleted, probably, unless something was reported.)

It seems to record at a rate of about 328 KB per minute, so we'd need alot of space.
Any suggestions? We'd need it to beable to hold the backup in progress, last month's backup, then (preferably another drive) for compressed copies of the previous 2 months, so we have access to 3 months plus current. Plenty of time for people to make pull requests or file reports.

This post has been edited by Axle Gear: 07 November 2006 - 02:38 AM

[Axle Gear]

Of course, data redundancy, security, and extreme reliability are a MASSIVE concern. =)

[bfg9000]

It's only ~14GB so I'd use drive sleds with 18 or 20GB drives (realistically, using 60GB drives would allow quarterly drive changes rather than monthly). Some type of RAID1 arrangement would protect from drive failure but that is really not backup. If the recordings are ended in discrete chunks, that would simplify backing up. It really depends on how secure the data must be from fire, etc.

And since such drives are so cheap I would not bother with reusing them by compressing to backups. Just keep the originals.

[qasdfdsaq]

328KB/min is only ~44kb/sec. Are you sure you haven't made a mistake there? Even the horrendous quality streaming video clips on my phone use 50% more bitrate than that, and they'd be far too poor quality for surveilance.

[SaxicolousOne]

The SV35 line of drives from Seagate is apparently designed for just this sort of thing (reliability while constantly recording, and so forth):

http://www.seagate.c...ing/po_sv35.pdf

Even the smallest one could record at your 328 KB/min. for almost a year, so you have the option of spending all that capacity on either very long recording time or greater video quality.

[MisterDuck]

Axle Gear, on Nov 7 2006, 01:36 AM, said:

Hello! I'm planning to set up an external security camera to tape an area, and want to have a PC set up that can save that much data onto a single virtual drive. It needs to be 720 hours long, so we only have to change it once a month, and can back it up to some sort of long-term storage later after sorting it. (Most would just be deleted, probably, unless something was reported.)

It seems to record at a rate of about 328 KB per minute, so we'd need alot of space.
Any suggestions? We'd need it to beable to hold the backup in progress, last month's backup, then (preferably another drive) for compressed copies of the previous 2 months, so we have access to 3 months plus current. Plenty of time for people to make pull requests or file reports.

Are you sure about those numbers?

328 KB per minute works out to ~44 Kbit/sec. That's a very low datarate for any video stream. What camera are you using, and what's the compression being used? Are you sure that's the correct datarate? I work in the video surveillance industry, and that doesn't seem right. I'd but 328 Kbit/sec, or maybe 328 KB/sec if it were full-speed motion-JPEG or something, but 328 KB/min is pretty darn small.

Our VGA video (using WMV9) ends up being between 300-800 KBit/sec, or roughly ten to fifteen times the datarate quoted above. WMV9 QVGA video ends up being somewhere between 100 and 400 KBit/sec or so.

[Axle Gear]

Actually, that should be per second. My bad!

It can drop to nearly <1kbit/second when there's no activity (the software is quite good at coding it to omit redundant data) or soar to well over 4 mbit/second if there's a lot of people or motion onscreen.
But what I stated is just what it averages overall during the course of it's time.

[Atamido]

328 * 60 * 60 * 720 = 850,176,000
Or about 850GB. I would add on a little in case there happens to be extra movement (requiring higher bitrate) and make it a solid 1TB for your current month's recording space.

Then you want to store last month's video, which would be another 1TB, bringing the total to 2TB.

Then you say you want to store "compressed copies of the previous 2 months", but you don't give any impression of how much space that would take up. Guessing that "compressed" means about 1/4th the space, then the two months together would be 0.5TB, bringing to total to 2.5TB.

So, your question is really, "How do I store 2.5TB of live data in a way that offers redundancy, security, and extreme reliability?"

Personally, I would use a RAID6 of 6x750GB hard drives with a hot spare and a batter backed up raid controller. That would allow for any two drives to fail, plus a third after the automatic rebuild on the hot spare. The Areca ARC-1220 would do peachy. 750GB drives can be had for ~$350 each, so the controller and drives alone would be $3K.

Of course, RAID is not a backup, so I would also use a tape carousel to regularly back up data to tapes that can be taken to an off-site location (in case the building burns down). And if you don't need to keep data forever, then you can also reuse tapes. Expect to pay $3K-6K for a LTO3 tape drive.

Of course, that assumes that you need to backup stuff regularly, or even at all. How much would you be out if you lost your data? You could always have a couple of 500GB hard drives and backup each half month to the drive in an external enclosure. That would allow you to move only the new data off-site every 2 weeks, though it would require more physical handling.


Off-Topic: I really wish there were a decent open source program for video surveillance. I have seen people hack together some interesting pieces, but nothing all inclusive. It's a shame, because there's such a big need out there.

[bfg9000]

If the files are indeed compressible, then they should be recorded directly into the compressed format in the first place to reduce storage requirements. But it sounds like it's not expected to record at 4mbit/s continuously anyway.

[ehurtley]

I would imagine that the video is nearly incompressible. Most modern video codecs do the best possible job compressing as it is. And since the OP mentions that the data rate fluctuates with activity, it appears that it has an extremely aggressive compression scheme as it is. Don't count on being able to get any extra compression. So figure 4 TB.

That means for a minimum multiple-failure system, you would need 8 750 GB drives (For a 7-drive RAID 5 with one hot spare,) or for a GOOD multiple-failure system, 10 drives (two 4-drive RAID 5s with one hot spare each, then RAID-0ed.) Or you could go all out and go for a RAID 6E, which would require 9 drives (6 'data', 2 'parity', and 1 'spare', although all three functions are spread across all disks.) But a RAID card that supports RAID 6E will be noticeably more expensive than one supporting RAID 50, and might even cost more than the cost of adding the one more drive to reach RAID 50.