Steve Flynn

Looking for advice on a small NAS with quite a specific/odd workload

Recommended Posts

Having never looked at NAS systems before I'm finding that the amount of choice is a little overwhelming so I figured I'd ask some people who undoubtedly know more than I.

I've been tasked with looking into the possibility of buying a relatively small NAS system, but one which is going to have a slightly task.

Firstly, it must support AES-256 encryption - no less is acceptable as this drive will contain sensitive information.

Secondly, it must be capable of being attached to Windows desktop PCs, AIX PowerPC servers, and rackmounted windows virtual machines without any hassle.

Thirdly is it's workload. It's going to contain millions of very small files (as an example, 20 million TIF scans ranging from 1 kilobyte upto to maybe a few meg. Depends on how big the document was really!). Throughput is the key here - we need to be able to read millions of files, manipulate them, maybe breaking multipage tiffs up to single pages and possibly combining single page scans into new multi-part tif documents.

Lastly is the size - somewhere around the 4 - 8 Tb would be good to start with, so I'm thinking of an 8 bay enclosure with a couple of 2Tb drives in there to begin with.

The primary use for this kit is to be sent off to a client who will plug it into (probably) a Windows Server (it depends on what they are running their systems on). They'll fill it with data and courier it back to us... hence it must be encrypted. We'll then send the unit off to our datacenter and have it connected to either an AIX 5.3 server or a rack-mounted Windows box.

Once this kit is attached to our servers, we'll begin manipulating the data, reading these 20 million small files and producing output directly onto our SAN storage we already have. Because of this workload, performance with small files is particularly important to us.

Any suggestions on suitable pieces of equipment?

Steve.

Share this post


Link to post
Share on other sites

What is the budget level for this system? That will be the driving force as to how this platform is built. The need for fast transfer speeds working with tons of small files while also having encryption kind of means that you are outside of the scope of anything SMB and into enterprise gear.

Share this post


Link to post
Share on other sites

What is the budget level for this system? That will be the driving force as to how this platform is built. The need for fast transfer speeds working with tons of small files while also having encryption kind of means that you are outside of the scope of anything SMB and into enterprise gear.

Budget is around the 1000 - 2000 UK pounds mark. Obviously management wants to keep costs down but speed costs, as I keep telling them! :)

I'm hoping that something like a QNAP TS-509 level of kit will be sufficient for our needs but having never used a NAS system before either at work or at home, I've no previous experience to say whether it will be up to the job. Currently, we've been running this kind of exercise using encrypted USB hard-drives, as the volumes have been relatively low (8 million documents delivered over the course of several weeks, with USB hard-drives being couriered to and fro) but now we're moving into the 20 million documents weighing in at 3 or 4 Tb that's becoming unworkable... so we're looking at alternative solutions.

Edited by Steve Flynn

Share this post


Link to post
Share on other sites

I'll contact one of our reps at QNAP and see what they say about the encrypted volume support. I have used models before with it, but it has been done at the software level, which dramatically lowers overall performance. Think 100MB/s down to 15-20MB/s. The solution is to move to a system that supports hardware encryption or allowing the drive to do it itself if the support is there, but I'm not sure what SMB-level models work like that.

Share this post


Link to post
Share on other sites

I'll contact one of our reps at QNAP and see what they say about the encrypted volume support. I have used models before with it, but it has been done at the software level, which dramatically lowers overall performance. Think 100MB/s down to 15-20MB/s. The solution is to move to a system that supports hardware encryption or allowing the drive to do it itself if the support is there, but I'm not sure what SMB-level models work like that.

I should also point out that the encryption is a must. If it can only be done by the processor in the NAS enclosure in software then so be it. We don't have any option about that and if it hits performance to down to the 15 MB/s levels then so be it... still got to be better than a Western Digital SmartBook or whatever they are called! :)

S.

Share this post


Link to post
Share on other sites

Well I suppose in that case I guess you are right. Both the QNAP models and Synology models support AES256 (tested it with the DS411+) but performance does take a massive hit. We have an 8-bay QNAP model in right now that I want to revisit hardware/software encryption on to see how it gets handled with high performance disks.

Share this post


Link to post
Share on other sites

We have an 8-bay QNAP model in right now that I want to revisit hardware/software encryption on to see how it gets handled with high performance disks.

Any idea when the results will be out Kevin? We'd be especially interested in seeing how much of a hit was generated as it'll help with planning the expected timings of the workload if/when we buy something similar.

Share this post


Link to post
Share on other sites

We have most of the benching done with both HDDs and at least one SSD config. Should be able to share data soon.

Share this post


Link to post
Share on other sites

We have most of the benching done with both HDDs and at least one SSD config. Should be able to share data soon.

Super. Enquiring minds here are most interested to know what the impact of hardware encryption is on one of these devices.

Share this post


Link to post
Share on other sites

Got some pre-review benchmarks on the QNAP TS-879 Pro to share with you. These were best case scenario results using eight 50GB OCZ Deneva SSDs setup in RAID0 going over 10G Ethernet.

Numbers are R/W

Non-encryption

2MB Sequential 816/649MB/s

4K Random 11.4/12MB/s

Encrypted

2MB Sequential 146/119MB/s

4K Random 1.58/0.99MB/s

As you can see, encryption cut the SSD performance by a 1/10th or so. Hard drive performance would have an even bigger slump.

Share this post


Link to post
Share on other sites

Got some pre-review benchmarks on the QNAP TS-879 Pro to share with you. These were best case scenario results using eight 50GB OCZ Deneva SSDs setup in RAID0 going over 10G Ethernet.

Numbers are R/W

Non-encryption

2MB Sequential 816/649MB/s

4K Random 11.4/12MB/s

Encrypted

2MB Sequential 146/119MB/s

4K Random 1.58/0.99MB/s

As you can see, encryption cut the SSD performance by a 1/10th or so. Hard drive performance would have an even bigger slump.

Ouch! Although at least I can set some expectations with the management here on what kind of potential performance they're going to (not) see!

Many thanks Kevin - this information is really, really appreciated.

Share this post


Link to post
Share on other sites

I think this link will helpful on your requirement. Small NAS. As i know that you can reduce the duplicate files and data clutter in a single moment. We called it as SIS (single instant storage). Here we can identifies duplicate files, saves one copy and replaces the other version.

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now