Sign in to follow this  

_Open formats vs. Proprietary/closed formats._

Recommended Posts

Originally from From Why Linux can only read but not write on WinNT partitions?

Please see the context there.

The problem, it seems to me, is a legal problem. My data is mine. It belongs to no-one but me. No-one has the right to make it difficult or impossible for you to access your own data.

I definately do not agree with this. It makes no business sense. Who said you don't have access to your own data? You have something on an NTFS partition, you have NT to access your data. Linux CAN READ from an NTFS partition. There, you have access to your data. You have an Excel file? Use Excel to read it. Even GNUmeric (or whatever they call it) can read it.

Because of the immense, staggering complexity of the completely closed MS Office format, nothing but Office has good support for any Office document. The best non-Office support is probably StarOffice, and it frequently makes mistakes even importing documents, let alone exporting.

The problem with what you are doing is open standards, and I don't believe that's very competitive.  Take for example HTTP, an open standard.  You know how long it's been since a revision?  What, after the huge explosion of WWW traffic in what, 10 years, we are still on HTTP 1.1?  

Name something wrong with HTTP.

HTTP hasn't been revised much because it was designed well in the first place, unlike many proprietary protocols. SMB (Server Message Block), the Windows filesharing protocol, for example, is horrible. It spends quite a bit of time just figuring out what version of SMB the "other guy" is using just to be compatible with its bugs. Look at the SAMBA mailing lists.

Now, let's take another open protocol, HTTP. Last revision: 1998 (version 4.0)

We now, in addition, have XHTML and XML, highly extensible and powerful (and open) formats. Which are revised frequently, if that is what you are after.

Conforming to a proprietary protocol is asking for your investment to become obsolete. Look what happened to Netware's IPX--now used only for ancient DOS games that don't understand anything else. What was it replaced with? A better, open protocol. TCP. Speaking of which, TCP's layered design makes it general enough to handle pretty much any communication needs, ever. Again, I'll spare the details.

Now, considering that, do you think that the internet would be even close to as widespread as it is today if HTTP and HTML were proprietary formats, owned by a company?

If I missed something in that previous example, let me give you something I know a little better.  USMARC format.  The way the Library of Congress stores bibliographic information (book information).  It's an open file format (published on LC's site).  Anyone can write a USMARC-format file.  Yet, it's pretty cryptic but works.  It was written in 1991.  Will it ever change, maybe, but not in the near future.  In my opinion, USMARC will never be improved upon for another 20 years.

A format used only by the government is, for all intents and purposes of this discussion, not open. Yes, it is available to the public, but that does not matter if nobody cares. The fact that the U.S. Government, not famous for brilliant technical design or efficiency, uses an odd format that may have been designed nonoptimally doesn't say anything against open formats themselves. Perhaps against the government, but that is another issue entirely. (besides,as you said, it works)

I can't believe that I am reading a message from somebody that is criticizing open formats in favor of closed ones with a straight face.

Here's another example: Compuserve GIF. An extremely simple (you can do the math in your head) format that was patented. It was never improved upon from its relatively basic and limited design because there was no incentive to change something that was already making money on the absurd licensing fees anyway. Then came along a new, better, open format: PNG.

PNG supports any compression method, though only zip is used at this time, it supports transparency and translucency (alpha channel) on both 8 and 24-bit images, compared to GIF's "off or on" transparency only, and it supports 24-bit color images, whereas GIF supports only 8-bit and lower.

It would seem that the only good that proprietary formats to is to encourage the making of open formats, no?

What I'm trying to get at is (hopefully I made sense), with open standards, there would be very little incentive to make a new format.

With well designed formats/protocols, there is little incentive to make a new one. For one thing, if a protocol/format replaces another, everyone needs to change over, which can be very time consuming and expensive.

You are probably right that there is more incentive to make new proprietary formats--they rarely gain widespread use so are replaced often, thus reinventing the wheel. Or they are replace by open formats. (see above)

What will happen is once you have this bright new idea on how to make a word processing document include graphics, tables, font-embedment, etc., that made Word 2005 XP Pro worth upgrading to, some open-source copycat will make something similar, and for free.  Then why should I even try if no one will buy my product?

That is a horribly flawed argument.

First, it is making the assumption that the subject's owner, Microsoft, frequently comes up with new innovations. Microsoft is an implimentation company--their claims about innovation aren't worth the packets they're downloaded on.

What they actually love to do is take open formats and do what is called "embrace and extend", in which they change (not necessarily improve, but change) a format or protocol just enough that it is incompatible with everyone else's open implimentation. Examples: Kerberos, Java, C#(=Java), and on and on. They do, I am sure, occasionally come up with a new idea, but every time I have almost thought they did, it was actually developed by some little known company (which mysteriously vanishes after MS impliments the idea) or by another individual or nonprofit group.

Second, it is implying that in the presence of other, OPEN Office formats, MS Office doesn't make money. It accounts for about half of Microsoft's profits. (Why should I even try if no one will buy my product?)

Third, you are implying that it is a good thing to pay another few hundred dollars to upgrade a product that altruistic programmers have developed for free. Since you are speaking of business sense: Welcome to free market economics. I guess MS will just have to lower their prices to compete (oh, darn.)--but they do not and have not. (see "monopoly abuse" on Google)

Fourth The person that has the bright idea about how to do something new is overwhelmingly likely to be an independant researcher, such as a CD professor, that has written up a paper details. Look up the history of most modern fundimental computing ideas and you will find this is frequently the source. With that new (usually open) standard, competition will be increased because several different companies will be able to impliment that idea--and the differentiating factor will be how well it is implimented, which allows them to show off their employee's skill. Contrast this with an idea's exclusive ownership by a company.

Fifth If such an idea does pop up, open source developers can impliment that into a number of products for the good of all. Altruism. Utilitarianism. Principals that improve the quality of life for people in general.

Businesses in general prefer buying (sorry, licensing [sorry, leasing]) software from other companies, feeling safe in the illusion that if it screws up, somebody else takes the fall for it-thus the software still sells. Whether or not it should sell is a debate for another time.

If a company can't keep up with the competition, damn, so long and thanks for all the fish. Leave room for someone that can.

...And if no one buys a product because the existing one is good enough, well, the business should have a more diverse portfolio of products than that. (Or find a way to force upgrades)

 People right now can include graphics, have tables, and embed fonts.  No need to change, right?

I don't see how this applies. In response, though, I am sure that someone will find an even more interesting feature that no one will use to add to an Office suite to sell it.

Yes, if this world was more conerned about moving technology and not the mighty $, your idea would make perfect sense.  But right now, it's not reality.  Look at all the Linux startups and services.  For example,  I was just reading up on them.  They are barely hanging on a threat, supported by the original founders already US$3k in the hole.  Will they survive?  Hopefully...

What does that have to do with open/closed formats? Many Linux companies, and even more .com companies, have died because they started with a business model that could not possible make money.

Well, tough.

In Linux's case, it doesn't need a company for anything more than political purposes. Frankly, most Linux developers could not care less if company XYZ used their program. There's no competition, just adding a feature or improving an existing one for the good of ones own computer use and the good of the community. If someone tries to make money off of their work, great, the developers were altruistic enough to allow that to happen (without being paid royalties, no less!). If that company fails, well, oh well.

Proprietary formats are almost never a better solution.

Share this post

Link to post
Share on other sites

Open protocols are often far superior to closed ones, but that doesn't mean I think closed ones should be against the law.

That the market chose IP over IPX just strengthens the case I made here.

If it were up to Tannin, I would probably end up having to write my code in Ada. It is a safer language than C, and safer is better, right? Yes, let's outlaw C.

BTW, I was reading the thread in order. I didn't see that you had created a new thread, until I had finished my response to Tannin. Sorry.

Share this post

Link to post
Share on other sites
Open protocols are often far superior to closed ones, but that doesn't mean I think closed ones should be against the law.

Neither do I. That would be a rather extreme measure, and it would probably lead to some interesting results internationally.

If it were up to Tannin, I would probably end up having to write my code in Ada.  It is a safer language than C, and safer is better, right?  Yes, let's outlaw C.

Whoa, now. I wasn't defending the illegalization of anything, in fact, I am frequently bewildered by a great many people who seem to think that if they were only in charge, or could at least pass an "obvious," but usually flawed and poorly thought out law, that everything would be all better.

Now if *I* were just in charge for a day, I could make everything better, but the democratic voting system has the habit of eliminating people who would be incompetant lawmakers like me.

Oh, wait...

BTW, I was reading the thread in order.  I didn't see that you had created a new thread, until I had finished my response to Tannin.  Sorry.

I'll never forgive you. :-)

Share this post

Link to post
Share on other sites

Mate, it's very, very simple.

Let's say you are a property developer. You buy some land and your in-house architect designs some houses, and you build them. You sell one of them to me.

Now, up until the point where we sign the contract, the house belongs to you. But from that moment on, the house belongs to me; I am the only person who is entitled to put things in that house. You are required by law to hand over any and all means of access you have. Not only must you give me the key, you are not allowed to keep a spare key for yourself.

Now, although the house is mine, and I am free to do anything I like with it, even knock it down if I want to, the design of the house is still yours. You may build another 50 houses to that exact same design if you want to, but if I buy the vacant block over the road and put up another house the same, you have every right to sue me.

All this is as it should be.

All I am saying is that we should apply the exact same logic to virtual possessions as we do to concrete possessions. When you sell me a house, you have to give me the key so that I have sole control over my possessions. Or, to put it another way, when you sell me a program that stores my virtual posessions (my accounts, let's say), you should be required to give me the keys to my own data

Want me to make this a bit more concrete? Let's try an example:

You are a software developer. I run a plumbing business. You write an accounting program and sell it to me. (And to as many other people as you can, of course.) At first, all goes well. But then one of several things happens:

(a) You go out of business. Next time your program breaks (for most programs break now and again) I can't access my data. I can't tell who owes me money, I can't pay my staff. So I call a software guy and he says he can provide me with a program which will do everything I need. "Oh, fine", I say, "and it will read my existing records?"

"Oh, no, they are in a proprietary format. Sorry. Don't know how."

This actually happens in the real world, Cas. I've seen it myself, quite a few times now.

(B) I decide that your $1000 a year licencing fee is too high and decide to take my business to your competitor instead. (This is called "free enterprise", and it's quite a common way of organising things. As a matter of fact, it usually works rather well.) And, although I have carried out my side of our contract to the letter, paid you the $2000 start up and the $1000 PA support and licence fee these last five years, now my new program can't read my own financial records. I can't send out invoices, I can't pay my bills, I can't pay my staff.

I've seen this one happen too.

The solution is very simple. If you are offering a product to the public that stores their data, then you must make the storage format public. It's no big deal: you're not disclosing your programming methods, only allowing me to access my own data in whichever way I see fit.

No doubt there are a million more examples I could cite, but I think the point is made.

PS: Where the hell did I say anything about mandating programming languages? Please, if you are going to engage with the points I raised, try to stick to the points I actually raised. If you want to make up new ones, then by all means do so, but please give the credit for them to the right person - i.e., your good self - and not me.

Share this post

Link to post
Share on other sites

Indeed, if you were to buy the software in question, you would be free to peruse the source code. You would have all rights to the software (the seller would not retain a 'key'), and you wouldn't have to worry if the seller went out of business.

You raised a number of legitimate concerns in your usage scenarios. These scenarios should be considered by users, when evaluating the overall value proposition offered by a software sale or license. The availability of source code or format specifications are perfectly valid selection criteria, and are common in the embedded space.

Share this post

Link to post
Share on other sites
PS: Where the hell did I say anything about mandating programming languages?

You seemed perfectly happy intruding upon my development process. You clearly have no moral reservations about forcing me to do additional work, and forcing my customers to pay for it, whether they want to or not.

By introducing legislation of the type you are describing, you are saying that this type of government intrusion is acceptable. You will also have to create a bureaucracy of the type I described. How else can you be sure that the file format specifications published are accurate, and up to date?

The government has shown (by mandating Ada for example), that they do not belong in the software business. As if $500 hammers weren’t enough, do we really need $100,000 text editors?

Share this post

Link to post
Share on other sites

Huh? Additional work? What additional work?

You have already designed the file format.

You have already documented the design.*

All you have to do is make that existing documentation public, in whichever way is most convenient to you. Posting it as a TXT file on your web site would be sufficient to comply with the law. That would take, what? Five minutes? Cost how much? 25c for the dial-up call to your ISP?

Let's take a worst case scenario. You have a complex file format and your internal documentation for it is hopelessly mixed up with the documentation of your data access routines (which, of course, you don't want to make public, and are not requied to make public). That suggests that your documentation skills could use some work, but no matter. Now you would need to take an afternoon to strip the confidential material out of your documentation, or else just type up a from-scratch table. And you only ever need to do it once, no matter how many thousands or millions of copies you sell.

As for administration, what administration? The existing legal mechanisms are perfectly capable of dealing with any issues arising. If you sell me software that fails to perform the function for which it was intended, I can sue you. That's the law as it already stands. All that changes is that now you need to provide me with documentation of the file format. If you get that wrong, so wrong that I can't access my own data, and you are so uncaring about your customers that you can't be bothered correcting your mistake, and it's so important to me that I'm prepared to spend good money on a lawyer - then you ain't doing your job anywhere right and you deserve to be sued.

In reality, how often would this happen? Well, when was the last time you heard of an OEM sueing a CPU manufacturer because there was a mistake in the data sheet? Have you ever heard of it?



* Unless you are some little tin-pot backyard crazy man from the days of make-it-up-as-you-go-along programming in GW-BASIC with no structure, no documentation, no plan, and lots of GOTOs. (And much as I think you've got some seriously wrong end of the stick on this matter, Cas, I know damn well that ain't you.)

Share this post

Link to post
Share on other sites

I understand that it may not seem like it, but preparing this type of document is more work than you realize. Developing a first class file format or network protocol is hard work, and often takes multiple iterations to get right. It is very common for release 1 to incorporate a file format that works, but is less extensible or elegant than one would like. Since this proprietary format is under your exclusive control, it is relatively easy to upgrade as better ideas come along.

If you publish your format however, the situation changes dramatically. As anyone who has worked on a formal standards committee can tell you, the difference between an internal design document, and a formal specification is enormous. An internal document that describes the format from the perspective of a single implementation, often leaves vast areas of ambiguity for developers implementing it in a different way. Suddenly your program is no longer able to read files of type .xyz, even though you designed the .xyz file format(and your customers blame you). After consulting the 20 page design document, it is clear that it was ambiguous on some point, and must be clarified. This is how standards develop, and they result in documents that are hundreds and thousands of pages long. This type of document requires a tremendous investment, and diverts coders away from software development.

Worse still, you have probably lost the ability to advance the format. If a new version of your software obsoletes the old format, many people will cry foul. They will suggest that the format was changed, explicitly to break interoperability with some competitor. Even if you regularly publish the new formats, many will complain that you are forcing your competitor on to a development treadmill in order to maintain compatibility.

Aside from all of these practical considerations, I am also a laissez-faire capitalist, and clearly have different ideas about the role of government.

I am afraid that we will simply have to agree to disagree on this point.

Share this post

Link to post
Share on other sites

Indeed, I doubt that we shall ever agree, and I note that the process is not as simple as it seems at first. Nevertheless, it's a badly-needed step in the direction of progress and a vital plank in the defence of free enterprise. I doubt that format descriptions would usually be exact and complete, but for business held to ransom by software vendors, a half-loaf would be a great deal better than none at all.

After all, how many other great legislative advances toward social responsibility in business have been supported by the businesses concerned? A short list of human-friendly advances that were bitterly opposed by the relevant industries follows:

Seat belts in cars

The eight-hour day

Air bags

Building worker safety rules

Tobacco advertising

Anti-racial discriminaton employment laws

Building guarantee rules (if you build me a house and it falls down inside ten years you have to fix it)

Exploitative retail store credit schemes

Portable mobile phone contracts

Share this post

Link to post
Share on other sites

Convicted out of his own mouth:

... Gates email, from 1998, which dealt with stopping non-Microsoft browsers being able to work well with Office: "Allowing Office documents to be rendered very well by other people's browsers is one of the most destructive things we could do to the company."

Share this post

Link to post
Share on other sites

As you well know, I see this as no kind of conviction.

The competitor is free to develop their own office suite, and render its documents perfectly.

The inability of the prospective user to render office documents outside of office is a perfectly good reason to pass it over, in favor of some other suite.

Share this post

Link to post
Share on other sites

Create an account or sign in to comment

You need to be a member in order to leave a comment

Create an account

Sign up for a new account in our community. It's easy!

Register a new account

Sign in

Already have an account? Sign in here.

Sign In Now
Sign in to follow this