jonthomas83

Help building a video server with LOTS of drives

Recommended Posts

Hi, first post, excuse me if this has been asked, I couldn't find my answer.

I'm currently running a video server with 6 hot swappable 4tb drives running the maximum 6 SATA ports on my motherboard. I'm looking to get a bigger case and more trayless backplanes to support up to 16 drives - mainly for parity purposes but will give me a much larger space for more movies.

Could someone help advise on how I can only find motherboards with maximum of 10 SATA ports, but they're expensive. Is there a better more efficient way of connecting 16 drives to a motherboard? Right now, my options, as far as I can tell is an ASUS motherboard with 10 ports and PCI card with an extra 4 ports and another with 2 ports. Is this correct or am I missing something.

I'm going to turn my current server back to a gaming machine (too high spec to be used as an inefficient server), so was wondering what sort of processor and RAM I'd have to buy to run a video server efficiently?

All my videos are 1080p with a few 720p, mostly lossless. I have around 600 movies and TV shows at the moment, but hoping to get more with this expansion. I currently run ubuntu with XBMC and would like to continue to do so until I buy a small HTPC to connect to the TV and will then stream to that (with XBMC on the client) using samba.

I hope I've provided enough info.

I'd really appreciate any help you guys could offer me, although I've got some experience, I still feel like a noob! Thank you and happy new year.

Share this post


Link to post
Share on other sites

Oh, nfs samba shares idle the cpu @ gigabit speeds... .

Plex, luks encryption, and parity math will show what you got.....

Sent from my rooted HTC Supersonic using Tapatalk 2 Pro

Share this post


Link to post
Share on other sites

Oh, nfs samba shares idle the cpu @ gigabit speeds... .

Plex, luks encryption, and parity math will show what you got.....

Sent from my rooted HTC Supersonic using Tapatalk 2 Pro

Sorry, "8 core anything", what does that mean?

Also confused about what you mean by "nfs samba shares idle the cpu..."

Share this post


Link to post
Share on other sites

I7 or AMD fx cpu

Simple samba or nfs "file serving", my AMD k6-2 from the 90s was sufficient.

Parity math and encryption underlying the filesystem, then nfs and samba shares need some serious math.... All depends how complicated you want it to be before data is actually read/written.

I'm running 8 on board sata, 4 more 4 port cards, 24 3tb hdds. 4 separate raid6 arrays. Then a jbod MD spanned from the MD raid6 arrays.

Eats up cpu, but there's underlying redundancy that's split up., and I have one huge file system.

As they say, more then one way to skin a cat. Pick a knife.

Sent from my rooted HTC Supersonic using Tapatalk 2 Pro

Share this post


Link to post
Share on other sites

I7 or AMD fx cpu

Simple samba or nfs "file serving", my AMD k6-2 from the 90s was sufficient.

Parity math and encryption underlying the filesystem, then nfs and samba shares need some serious math.... All depends how complicated you want it to be before data is actually read/written.

I'm running 8 on board sata, 4 more 4 port cards, 24 3tb hdds. 4 separate raid6 arrays. Then a jbod MD spanned from the MD raid6 arrays.

Eats up cpu, but there's underlying redundancy that's split up., and I have one huge file system.

As they say, more then one way to skin a cat. Pick a knife.

Sent from my rooted HTC Supersonic using Tapatalk 2 Pro

Hi there, sorry for the late reply to this, but I just wanted to say thank you for this. It was immensely helpful and I really appreciate you taking the time.

I've bought all the parts for my server now. 10 SATA port motherboard with 2 port PCIe SATA card from StarTech. I'm going to be running a top Pentium with 8GB RAM. Eventually, will be running 16 drives but for now am happy with 12 and will expand when I outgrow them. My disks will be running 7 data disks with 1 parity (x2) eventually (RAID6).

I will be using SnapRaid on an Ubuntu Server VM using KVM on an Ubuntu Server host.

All theoretical until I try it in anger, but it's been well planned so far and I have a little bit of experience with all of the above so I should be fine! :-)

Again, thank you for your advice, it helped greatly.

Edited by jonthomas83

Share this post


Link to post
Share on other sites

Good question. I knew someone would ask.

Basically, I've learned lessons from a previous build where I was building a smaller Ubuntu Server based NAS for my music collection. On that server there are a few different applications which work and some which don't and I've wanted to uninstall and install different things. In the end it became messy and a lot of things are now unused and redundant. For anyone who's followed this route before, you'll know how frustrating this can be (I'm a noob to this stuff, so mistakes are going to happen), a lot of the time, you end up wiping it clean and starting over. I don't want to do that this time!

So the idea is that the host will be a barebones config and the VM will be my media server. I can clone that at any time to work out best software to install for different purposes and features and then simply replace the media server with the new improved version at any time. This makes a lot of sense to me and will allow me to create multiple sandboxes to play with, should I need to mess about with trying alternative servers.

Granted, this approach would be less necessary with a video server than it would with the music server as there a lot more applications around for music - beets, MPD, Groovebasin etc, all worth playing around with. With video, I just want to serve it up. So now I've learned my lesson, I'll be going back and re-installing my music server once this video server is sorted.

Sorry for the long post, but I believe it's a good approach worth sharing.

For further reading: http://www.havetheknowhow.com/Configure-the-server/Configure-KVM.html (this guy is great and his tutorials have helped lots!)

Share this post


Link to post
Share on other sites

A bit more involved then my setup.

Dedicated media storage , and a 2nd box with host debian, and lots of virtual box ideas. Reason it has 32gb of ram. Filrsever only has 512. I'm cheap.

Share this post


Link to post
Share on other sites

Ha, nothing wrong with being cheap, I've looked to make savings where possible, but I've got everything I wanted under budget, which I'm very happy with.

I'm looking forward to getting started now! :-)

Thanks again for your advice!

Share this post


Link to post
Share on other sites

17 drives in my NAS, just threw in a pair of 4 TB drives. The rest are 2TB drives.

It's an AMD Athlon CPU with a board with 8 SATA ports. OS drive is a 16GB IDE SLC SSD.

The archive is all 100% mirrored, no RAID stuff so no overhead. Each drive has a mirror. Pooling accomplished with DriveBender on Win7. Has run 24/7 for years!

The mirror drives are all in SATA multiplier boxes, connected to a RocketRAID controller set for legacy JBOD mode.

post-71169-0-20893700-1424450718_thumb.j

post-71169-0-64949000-1424450726_thumb.j

Edited by FastMHz

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now