Sign in to follow this  
amdoverclocker

Storage for Splunk

Recommended Posts

Anyone care to go into detail on their Splunk storage/server setup? We are looking into the EMC solution, Isilon. Not sure I am sold that its the right solution for me but I'd like to hear from others on how they handle Splunk data. We are looking at 25-35GB/day or so with a retention period of 18 to 24 months. Thanks!

Share this post


Link to post
Share on other sites

I don't know that we have a lot of Splunk users here, though there is a startup in town I've been talking to that uses Splunk. I can look into it to see how they're configured.

Share this post


Link to post
Share on other sites

Anyone care to go into detail on their Splunk storage/server setup? We are looking into the EMC solution, Isilon. Not sure I am sold that its the right solution for me but I'd like to hear from others on how they handle Splunk data. We are looking at 25-35GB/day or so with a retention period of 18 to 24 months. Thanks!

Rough numbers without a lot of considerations.

35*365=12,775 GB. 12.4TB Per year. Call it 25TB after 2 years of retention.

Isilon "starts to make sense" at around 45-50TB of usable capacity. Smaller than that and it's expensive for what it is. If you have other potential uses for Isilon this is a good opportunity to implement them. Ie, hadoop, other shares, etc.

Do you have any requirements for your logs? Does it need to live "out of band" on it's own device? If not, do you have other storage you could add-on to for this project?

I know that Splunk data compresses very well, so an array that does compression/dedupe would be interesting to you.

Let me know!

Share this post


Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this