[Big Buck Hunter]

Looks like HDD sectors are finally going from 512 (and some 528) bytes to 4096. Looks like some additional media fault tolerance has been built in as well.

http://www.sys-con.c...ad/368465_p.htm

Frank

[whiic]

This has been discussed by IDEMA and all HDD manufacturer's since... well, for a very long time. Motivation to increase block size increases, but problems that increasing block size cause remain.

So WHEN is it REALLY going to happen? I have a feeling it has become like one of those features that will be implemented in "next version of Windows" since... for a very long time (and for numerous versions of Windows as well). It's like one of those things that are about to happen N years from now, no matter what year we are currently living. There's lots of those technologies in the storage industry, take holographic storage for example: it's always a few years from wide scale availability but it's never closer than that. The only thing that changes is the promised capacity: 1TB in just a few years, 10TB in just a few years, etc. Soon they'll promise 100TB in just a few years, yet there's not even 1TB holographic storage available to public.

I wish larger blocks will someday become reality. There needs to be BIOS, OS and software support for it first. Old software is not compatible, but of course not all software needs to be compatible (as OS may offer quite a bit transparency), it would merely be some low-access I/O software that would become outdated, right? Need to make BIOS and OS compatible remains. Is Vista compatible? If not, that already means there's going to be a significant delay in implementation.

[whiic]

I read that sys-con.com article again. It claims the standard is now complete... yet IDEMA.org has the following list:
Long Data Block Standards
Approved Standards
There are currently no approved standards for this committee
Proposed Standards
blah blah blah blah blah

Maybe IDEMA.org hasn't been updated? That sys-con.com article is quite new (April 30, year 2007 presumably but not stated).

[whiic]

That sys-con.com article isn't very descriptive about the nature of the "new standard". Is it derived from ATA/ATAPI-7 standard that includes support for bigger than 512 byte physical sectors while retaining the current LBA sector size at 512? This practically means write operations are performed like with flash memory: read whole physical sector, replace overwritten part physical sector of physical sector and write the result back to physical sector (along with new ECC for whole physical sector). This causes severe reduction of write performance on old systems that don't have OS support for 4 KiB physical sectors but would provide full backwards compatibility and even boot support on BIOSes that don't support non-512 byte sectors. It would also make it possible to use of 4 KiB sectors when OS has booted while using compatibility only to get past BIOS limitations, thus suffer no performance loss after OS's own drivers are loaded... IF the driver's support 4 KiB sectors, that is. And only IF the HDD if formatted specifically with 4 KiB sectors in mind (i.e cluster size of 4 KiB, 8 KiB, etc. and with no offset).

Is this about ATA/ATAPI-7?

[Big Buck Hunter]

whiic, on May 2 2007, 10:01 AM, said:

Is this about ATA/ATAPI-7?

Not that I am aware of, since ATA/ATAPI only define the interface, and not the geometry of the device you are using. I have an old 420MB drive in NY that has a LLF of 528bytes/block, and modern ATA adapters do not have a problem with it. Also, t13.org handles ATA/ATAPI.

I think 4K blocks will be a good thing, since most CPU architectures use 4K pages, it seems to make mapping memory to disk easier (sparc64 and a couple of other arches use 8K pages).

Frank

[whiic]

"Not that I am aware of, since ATA/ATAPI only define the interface, and not the geometry of the device you are using."

Don't blame me, I'm just a messenger. I read it from IDEMA.org -> Patents -> Long Data Block Standards -> Proposed Standards -> HGST ATA Standard Proposal. It links to a file named Colegrove4k_sector_paper_r0.doc (author Dan Colegrove from Hitachi GST).

Quoting preamble of that document: "The ATA/ATAPI-7 Disk drive interface standard includes support for drives with 4 kilobyte physical sectors. Before ATA/ATAPI-7 the disk drive interface supported only 512 byte sectors."

[Olaf van der Spek]

Big Buck Hunter, on May 3 2007, 12:14 AM, said:

I think 4K blocks will be a good thing, since most CPU architectures use 4K pages, it seems to make mapping memory to disk easier (sparc64 and a couple of other arches use 8K pages).

I really don't understand the advantages. Why would it be cheaper to do a single request for 1 4 kbyte block than a single request for 8 sequential 1/2 kbyte blocks?

[Big Buck Hunter]

whiic, on May 2 2007, 07:42 PM, said:

"Not that I am aware of, since ATA/ATAPI only define the interface, and not the geometry of the device you are using."

Don't blame me, I'm just a messenger. I read it from IDEMA.org -> Patents -> Long Data Block Standards -> Proposed Standards -> HGST ATA Standard Proposal. It links to a file named Colegrove4k_sector_paper_r0.doc (author Dan Colegrove from Hitachi GST).

Quoting preamble of that document: "The ATA/ATAPI-7 Disk drive interface standard includes support for drives with 4 kilobyte physical sectors. Before ATA/ATAPI-7 the disk drive interface supported only 512 byte sectors."

Neat doc. I'll ping Mr Landis and see if I can get the straight dope on the situation. If you find any other docs, toss em on this thread.

Frank

[Big Buck Hunter]

Additional info.

1: I believe that CDRoms are 2048 bytes per block. This may contradict the "Before ATA/ATAPI-7 the disk drive interface supported only 512 byte sectors." comment depending on what the definition of "disk drive interface" is. Who knows.

2: From a slashdot entry (I could not find the original question though)

Quote

I can speak with some authority on this - I work for one of those aforementioned hard-drive manufacturers, and have been doing a small amount of work on this exact thing.

The easy answer is this: in order to do ECC-like data checking on a larger set of data (say, a group of eight 512-byte sectors), it means that if you want to write sector three of that eight, you end up having to re-read the whole thing before you do anything else - thus basically giving you 4,096-byte "sector" anyway.

The other half of that answer is this: do you know what the "real" storage capacity of a CD is, without all the error checking? It's a bit less than double. Even most of the enterprise folks wouldn't accept a 40% hit in data density in return for what works out to not that big an increase in reliability (data redundancy doesn't buy you that much unless that data is on different spindles). They'd just rather get the whole data space and do a RAID, especially since that's what they're going to do anyway.

3: Do we have a math major here that can show me how to apply Shannon's source coding theorem to see if larger block sizes actually improve ECC (or reduce the space needed by the current level of ECC)?

4: It appears that my old 420MB drive is SCSI

5: A good doc on filesize distribution on a windows PC: http://research.micr...ast07-final.pdf . In the case the wasted space argument comes up. We're probably going to see more filesystems add tail packing and block suballocation to their features pretty soon, so I do not believe this to be a concern. To find out how many files under linux are below a certain size, use... find ~ -type f -size -512c |wc -l (replace 512 with a size and ~ with a directory). Does anyone have a windows equiv for this?


6: Do we now get 4096 bytes for our bootloader (grub/lilo)? Can Dos/Windows now have more than 4 primary partitions? How does this affect GPT or other partition schemes?

I'm really looking forward to this change.

Frank

[imsabbel]

Well, doesnt shannons therorem actually _demand_ an infinite block size?
And thus, the shorter the real world blocks, the worse the applyability (is this a word?)?

As a rule of thump: Just think about a random distributions of errors, and an encoding that can correct one error in X bits. And another that can correct 8 errors in 8X bits.
Its obvious which one will be more effective.
(just take the case of 8 errors. The 2nd one will always be able to correct it, the first one will fail in >50% of all possible error distributions)