Jump to content


Photo

LTO tape minimum streaming speeds


  • You cannot start a new topic
  • Please log in to reply
12 replies to this topic

#1 screwtop

screwtop

    Member

  • Member
  • 22 posts

Posted 26 June 2007 - 10:31 PM

I'm looking at acquiring an LTO-3 tape drive (HP Ultrium 960) and have some concerns about the minimum data rate required for streaming in my environment. I understand the data rate matching (DRM) on the HP 960 allows it to slow down to 27 MB/s, but even without compression this might be too fast for collections of small files being archived on the fly. I understand it's possible to use LTO-2 media in an LTO-3 drive, but will this actually result in a lower minimum data rate? Or will I just have to bite the bullet and buy a dedicated hard drive just for staging the archives prior to streaming to tape?

#2 jpiszcz

jpiszcz

    Member

  • Member
  • 578 posts

Posted 27 June 2007 - 03:13 AM

I'm looking at acquiring an LTO-3 tape drive (HP Ultrium 960) and have some concerns about the minimum data rate required for streaming in my environment. I understand the data rate matching (DRM) on the HP 960 allows it to slow down to 27 MB/s, but even without compression this might be too fast for collections of small files being archived on the fly. I understand it's possible to use LTO-2 media in an LTO-3 drive, but will this actually result in a lower minimum data rate? Or will I just have to bite the bullet and buy a dedicated hard drive just for staging the archives prior to streaming to tape?



27MB/s is the minimum and I while LTO-3 can write to LTO-2 I wouldn't recommend doing that, use NetBackup Enterprise and Multiplexing, this will allow you to push from multiple machines onto the same stream onto the same tape and maximize (e.g., a gigabit connection)

#3 tygrus

tygrus

    Member

  • Member
  • 116 posts

Posted 28 June 2007 - 09:33 AM

[27MB/s is the minimum and I while LTO-3 can write to LTO-2 I wouldn't recommend doing that, use NetBackup Enterprise and Multiplexing, this will allow you to push from multiple machines onto the same stream onto the same tape and maximize (e.g., a gigabit connection)


It's hard to get some details from the software and driver manufactures. Our old DLT7000 (35GB native, ARCserve backup) would indicate compression ratio and probably only one write speed so you could work out actual data rate to tape (reduced speed meant slow disk and show horning). Our new server with LTO-2 drive (200GB native, BAB 11.5) doesn't seem to get the feedback from drive to calculate actual compression. It still seems to do spurts and then slows and stop-rewind-continue then tries from max speed again. If slowed down earlier and tried to read ahead further (into system RAM and top up drive cache) then it could do better.

Anyway, 20MB/s of tape data average (some less) and the tape is not getting it fast enough. Local server (with the tape drive) backs up 250GB of files from 6 volumes spread over 8x 72GB Seagate 15K (complicated use of RAID 0,1 or 10 each) plus another 20GB from another server (old&slow), 4 multiplexed streams at a time. 45MB/s average raw, <20MB/s est. to tape (est. >2.3:1 avg compression ratio. MSOffice docs, SAS datasets etc.). The backup software sets aside a several 100MB's of RAM for caching but it's obviously not enough in my situation.

Data rate matching range 10-30 MB/s 27-80 MB/s
LTO4 : ?-160MB/s (?MB cache, ?, 800GB native per tape)
LTO3 : 27-80MB/s (128MB cache, U320 SCSI, 400GB native per tape)
LTO2 : 10-30MB/s (64MB cache, U160 SCSI, 200GB native per tape)
LTO1 : 8-16MB/s (64MB cache, U80 SCSI, 100GB native per tape)

http://www.infoworld...Phandson_1.html
http://h18006.www1.h.../11739_div.html
http://www.quantum.com/

#4 jpiszcz

jpiszcz

    Member

  • Member
  • 578 posts

Posted 28 June 2007 - 11:56 AM

[27MB/s is the minimum and I while LTO-3 can write to LTO-2 I wouldn't recommend doing that, use NetBackup Enterprise and Multiplexing, this will allow you to push from multiple machines onto the same stream onto the same tape and maximize (e.g., a gigabit connection)


It's hard to get some details from the software and driver manufactures. Our old DLT7000 (35GB native, ARCserve backup) would indicate compression ratio and probably only one write speed so you could work out actual data rate to tape (reduced speed meant slow disk and show horning). Our new server with LTO-2 drive (200GB native, BAB 11.5) doesn't seem to get the feedback from drive to calculate actual compression. It still seems to do spurts and then slows and stop-rewind-continue then tries from max speed again. If slowed down earlier and tried to read ahead further (into system RAM and top up drive cache) then it could do better.

Anyway, 20MB/s of tape data average (some less) and the tape is not getting it fast enough. Local server (with the tape drive) backs up 250GB of files from 6 volumes spread over 8x 72GB Seagate 15K (complicated use of RAID 0,1 or 10 each) plus another 20GB from another server (old&slow), 4 multiplexed streams at a time. 45MB/s average raw, <20MB/s est. to tape (est. >2.3:1 avg compression ratio. MSOffice docs, SAS datasets etc.). The backup software sets aside a several 100MB's of RAM for caching but it's obviously not enough in my situation.

Data rate matching range 10-30 MB/s 27-80 MB/s
LTO4 : ?-160MB/s (?MB cache, ?, 800GB native per tape)
LTO3 : 27-80MB/s (128MB cache, U320 SCSI, 400GB native per tape)
LTO2 : 10-30MB/s (64MB cache, U160 SCSI, 200GB native per tape)
LTO1 : 8-16MB/s (64MB cache, U80 SCSI, 100GB native per tape)

http://www.infoworld...Phandson_1.html
http://h18006.www1.h.../11739_div.html
http://www.quantum.com/


I have been using tape drives for more than 10+ years, if you are getting 10-30MB/s, you have problems.

I run LTO-2 at 40-96MB/s regularly and LTO-3 from 60-140MB/s.

#5 Jackal29A

Jackal29A

    Member

  • Member
  • 9 posts

Posted 29 June 2007 - 03:27 AM

I have been using tape drives for more than 10+ years, if you are getting 10-30MB/s, you have problems.

I run LTO-2 at 40-96MB/s regularly and LTO-3 from 60-140MB/s.


I've just got a Tandberg LTO-4 drive and can't even remotely approach the promised native rate, I'm doing a 2 step backup, first D2D then from a 2 Disk RAID 0 (2 x WD5000YS, Adaptec 4800SAS) to tape.
On average I get 30-40MB/s with ocasional 80MB bursts. I tried using NTBackup (with registry tweaks) and Backup Exec 11.1d (comes with the drive), results are nearly the same. The tape is the only device connected to an Adaptec 29320ALP-R card on a 64/133 slot.
The RAID 0 is capable of sustaining well over 40MB/s so I must be doing something wrong, how can you get such high speeds? could you please elaborate a little further?

#6 jpiszcz

jpiszcz

    Member

  • Member
  • 578 posts

Posted 29 June 2007 - 03:29 AM

I have been using tape drives for more than 10+ years, if you are getting 10-30MB/s, you have problems.

I run LTO-2 at 40-96MB/s regularly and LTO-3 from 60-140MB/s.


I've just got a Tandberg LTO-4 drive and can't even remotely approach the promised native rate, I'm doing a 2 step backup, first D2D then from a 2 Disk RAID 0 (2 x WD5000YS, Adaptec 4800SAS) to tape.
On average I get 30-40MB/s with ocasional 80MB bursts. I tried using NTBackup (with registry tweaks) and Backup Exec 11.1d (comes with the drive), results are nearly the same. The tape is the only device connected to an Adaptec 29320ALP-R card on a 64/133 slot.
The RAID 0 is capable of sustaining well over 40MB/s so I must be doing something wrong, how can you get such high speeds? could you please elaborate a little further?


Yes, it is the way tapes work and the number of tracks per each tape, for LTO-3 it has 384 tracks per band. In order to fully utilize the tape you need to 'multiplex' the data onto the tape.

The only way I know how to do this currently is with NetBackup Enterprise.

STREAM1 @ 50MB/s ->
STREAM2 @ 50MB/s -> (MPX=3) STREAM ==> TAPE
STREAM3 @ 50MB/s ->

This is how you achieve 100MB/s+ transfer rates to tape.

Using only a single transfer I have not been able to obtain greater than 60-64MB/s.

Justin.

#7 Jackal29A

Jackal29A

    Member

  • Member
  • 9 posts

Posted 29 June 2007 - 04:46 AM

Yes, it is the way tapes work and the number of tracks per each tape, for LTO-3 it has 384 tracks per band. In order to fully utilize the tape you need to 'multiplex' the data onto the tape.

The only way I know how to do this currently is with NetBackup Enterprise.

STREAM1 @ 50MB/s ->
STREAM2 @ 50MB/s -> (MPX=3) STREAM ==> TAPE
STREAM3 @ 50MB/s ->

This is how you achieve 100MB/s+ transfer rates to tape.

Using only a single transfer I have not been able to obtain greater than 60-64MB/s.

Justin.


Justin,
Multiplexing works only when backing up from different computers or it also works backing up from different drives/volumes on one server?
Sorry if it is a stupid question but I had never heard about multiplexing streams before.
Thx.

#8 jpiszcz

jpiszcz

    Member

  • Member
  • 578 posts

Posted 29 June 2007 - 04:48 AM

Yes, it is the way tapes work and the number of tracks per each tape, for LTO-3 it has 384 tracks per band. In order to fully utilize the tape you need to 'multiplex' the data onto the tape.

The only way I know how to do this currently is with NetBackup Enterprise.

STREAM1 @ 50MB/s ->
STREAM2 @ 50MB/s -> (MPX=3) STREAM ==> TAPE
STREAM3 @ 50MB/s ->

This is how you achieve 100MB/s+ transfer rates to tape.

Using only a single transfer I have not been able to obtain greater than 60-64MB/s.

Justin.


Justin,
Multiplexing works only when backing up from different computers or it also works backing up from different drives/volumes on one server?
Sorry if it is a stupid question but I had never heard about multiplexing streams before.
Thx.


It can be used either way, think of it like a hard drive read benchmark, if you run 3 streams (and also multiplex level of 3) that would kill the HDD. If you had 3 diff HDD, read 1 stream on each and used a multiplex level of 3 then it would give you better speed.

You should Google for:

1. NetBackup Multi Streaming vs. Multiplexing- there is a document/answers that can probably explain the two better than I can here :)

Justin.

#9 Jackal29A

Jackal29A

    Member

  • Member
  • 9 posts

Posted 29 June 2007 - 05:13 AM

It can be used either way, think of it like a hard drive read benchmark, if you run 3 streams (and also multiplex level of 3) that would kill the HDD. If you had 3 diff HDD, read 1 stream on each and used a multiplex level of 3 then it would give you better speed.

You should Google for:

1. NetBackup Multi Streaming vs. Multiplexing- there is a document/answers that can probably explain the two better than I can here :)

Justin.


Thx again, I'll investigate it.

#10 screwtop

screwtop

    Member

  • Member
  • 22 posts

Posted 29 June 2007 - 09:30 PM

Some useful tips here - thanks!

I'm also wondering about the effect of disabling compression on the drive. Even with no compression, this would still leave the (say) 27 MB/s minimum data rate required for streaming, correct?

In your experience, have you found that multiplexing complicates the management of backups and archives on tape, or the restoration process?

Hmm, it sounds like a dedicated disk or striped array is going to be necessary to get reliable streaming to tape in my environment. I'm still intrigued by the suggestion (in a Fuji white paper, IIRC) of using previous-generation media to "lower the bar" a little on the minimum data rate...

#11 jpiszcz

jpiszcz

    Member

  • Member
  • 578 posts

Posted 30 June 2007 - 03:24 AM

Some useful tips here - thanks!

I'm also wondering about the effect of disabling compression on the drive. Even with no compression, this would still leave the (say) 27 MB/s minimum data rate required for streaming, correct?

In your experience, have you found that multiplexing complicates the management of backups and archives on tape, or the restoration process?

Hmm, it sounds like a dedicated disk or striped array is going to be necessary to get reliable streaming to tape in my environment. I'm still intrigued by the suggestion (in a Fuji white paper, IIRC) of using previous-generation media to "lower the bar" a little on the minimum data rate...


Since HW compression is done on a chip on the drive, turning it off is not going to buy you much, as far as I know most people leave it on. Also note with LTO-X technologies each byte written to the tape is then read back for verification before it says its good.

Justin.

#12 Jackal29A

Jackal29A

    Member

  • Member
  • 9 posts

Posted 05 July 2007 - 07:19 AM

I have downloaded a trial version of ARCServe 11.5 and made some test using the staging and multiplexing options.
The configuration is as follows:

Server:
- Supermicro 7043R-8P 2xXeon 2.8GHz, 4GB Ram Windows 2003 Server Enterprise SP2
Tape drive:
- Tandberg LTO-4 7202 on an Adaptec 29320ALP-R (PCI-X 133 slot)
Storage:
- Adaptec 3410S (PCI-X 66 slot) with 5x73GB (15k) RAID5 150GB of data, 2x36GB (15k) RAID1 28GB of data and 2x36GB (15k) RAID1 for system
- Adaptec 4800SAS (PCI-X 133 slot) with 4x500GB (7.2 SATA WD WE2) RAID10 200GB of data and 3x500GB (7.2 SATA WD WE2) RAID0 for backup staging (D2D2T)

What I found is that using multiplexing (4 streams) I can achive a maximum of 3,500-4,000 MB/min while the four streams are active, as soon as the smaller partitions are finished the througput drops to around 2,500-2,900. When only one stream is active it drops even further ro the point where the tape stops streaming and becomes very slow.
Using disk staging, first D2D and then D2T, I can achive and sustain 5,800-6,500 MB/min.

#13 screwtop

screwtop

    Member

  • Member
  • 22 posts

Posted 14 August 2007 - 08:08 PM

Just a quick update. I've just been testing LTO-2 media in my HP Ultrium 960 (LTO-3) drive, and it will continue to stream happily with no interruptions or shoeshining at 10 MB/s. This could well be slow enough for a typical single disk source, and quite possibly even 100 Mb Ethernet. Note that I'm using mbuffer to buffer and throttle the data on its way through.
--
screwtop



0 user(s) are reading this topic

0 members, 0 guests, 0 anonymous users