Say what you want from us but not what we don't want to hear... Or how much did we regress ?

Blog post by mmu_man on Tue, 2008-11-04 04:42

Last week I received a mail telling me MSI wanted to hear from me what I wanted on their next products on their forum. Frankly, I didn't even remember having an account on that forum, where I registered to help someone on a BeOS SMP question. So I jumped in and started telling what I, as an Haiku developer, really expected from a hardware manufacturer, trying to explain, with humour but determination, why open hardware is so important to me.

Then after some other's post I replied a second time, first to someone mentioning the need for HDMI support on why I didn't agree because of DRM, then a maybe-rantful-but-oh-so-true digression about the availability of specifications. Sadly, neither post are available for your reading pleasure, a moderator found them to be "rubbish" and removed them altogether, who obviously doesn't use alternative Operating Systems, never wrote a driver with only uncommented Linux code as reference, doesn't live in France, and never saw a TV set manual from before 1980.

Since my views weren't welcome there, I'll try to at least make them clear here, and either MSI (and other vendors) read them or not, but I'm used to the latter anyway. Please note that was the act of a moderator, which might or might not be part of MSI, nor reflect their own policy (I wish).

On open drivers

Oddly it seems I'm not the only one asking for opensource on that forum... But it seems to be more "politically correct" when it's for GNU/Linux... which usually means the result would be GPL, so unsuitable licence-wise for porting to Haiku, and unportable anyway because no stable driver API in Linux means no way to understand it easily. It's the very reason I directly ported OpenSound System 4 instead of trying at ALSA, as I knew I had no single chance, because it's so tied to Linux.
Those who tried to write a driver using only the uncommented obscure contradictory source code for 5 different linux drivers for the same hardware will know what I mean. Oh yeah, I actually also found a "datasheet"... well, it's quite incomplete and even wrong according to drivers I've seen and tests I made... plus that doesn't cover the CMOS sensor chip connected to it. I recall asking for a datasheet for this one, and actually getting one, though it wasn't of much help either, really.

Of course, people will just say "Why do you ask for specs or opensource drivers ? It's all fully supported by Linux!"
Sure... But:

  • Not everyone uses GNU/Linux. Yes, you've had to learn the hard way that there wasn't only Windows, but also GNU/Linux. Now it might be time to learn there is not only Linux in the FOSS world. There are actually many, many "Operating System" (you'll understand that term one day...): AmigaOS, AROS, FreeBSD, NetBSD, OpenBSD, Minix, Syllable, GNU/Hurd, V2OS, MiNT, ReactOS, FreeDOS, Haiku..., each with their own pros and cons, but each with their own reason of existing in the software ecosystem. Thinking a single OS for everyone is enough is foolish. It's because there are so many different projects with different goals that innovation can really happen, because people exchange ideas and code. That's called technodiversity, and was the conclusion of my talk(fr) at the last RMLL in july...
  • Not everyone uses Linux on x86. Yeah you know there are other architectures out there... And since half of the "supported by Linux" hardware is actually supported by the means of a binary blob with an "open" kernel glue, this scam only works on Linux on x86. No problem for x86 motherboards, sure, but well video cards, NICs, wifi cards and others... Just try to use them even in Linux on ppc.
  • Sometimes the claimed "support" for Linux just doesn't exist, or is just the bare minimum, and often isn't the fact of the vendor itself but actually the act of developers writing drivers against the very will of the same vendor.

Of course FreeSoftware developers are smart, and devise many strategies to work around this.

  • virtualization. Yes of course, you can run Linux on anything that can run vmware, but well if that's not cheating, I don't know what cheating is.
  • Linux emulation: FreeBSD has a Linux emulation layer allowing it to run Linux binaries, but that doesn't work for kernel drivers. At least they can run flash player (but that sux anyway).
  • Windows emulation (ndiswrapper, mplayer's DLL loader, wine)... but of course, that means it restricts using the driver or application to Linux on x86, and you've just learnt that there is not only x86 around. Also, it is often criticized by FreeSoftware proponents as giving vendors one more excuse for avoiding real open drivers, as "hey you see, it works in Linux now!", which is wrong, depending on the platform you run it.
  • Other OS emulation. Haiku for example, has a FreeBSD compatibility layer that allows it to port BSD network drivers quite easily (BSD drivers are often regarded as cleaner, simpler to understand, using a stable driver API unlike Linux, and have a much more permissive licence than Linux' GPL), even though there are fewer than Linux ones.
  • Reverse Engineering: That's the most widely used method of providing drivers to Linux and other alternative OSes (not only FOSS ones!). While it sounds great because developers don't rely on vendors, it's really a problem because:
    • It once again gives an excuse to the vendor to say "Oh it already works in Linux, you don't need open drivers", even sometimes claiming their "support" for Linux. And usually they just forget about other OSes along the way.
    • It wastes precious developer time and resource and money (they must buy many different models to make sure they work), instead of having them work on other useful stuff that could actually bring advantages over competition ("kill apps", or just fixing bugs).
    • While it "seems" to work, this time not only on x86, it usually only supporting the minimum features of the hardware, those which the developer really needed, and only that, because it just takes time and money even if (because!) they do that in their spare time.
    • Also, because they come from source that cannot be ascertained (reverse engineering is about as much as russian roulette), and even though FOSS devs know their field, there is no single way one can make a guarantee about the correctness of the driver. It might be missing safety checks, making the hardware do weird things, or even destroy it. It stems from this that most vendors "certifying" their hardware on Linux are just certifying the fact they tried to boot it and it works (sometimes not even this), but not the fact that it acts according to the specifications, because those same specifications were not made available. Therefore, many (not all hopefully) hardware vendors certifying Linux are actually making false claims, and might (and should) even be potentially considered responsible for misbehaviour from such drivers.

On schematics of 30 years old TV sets

I suppose some people won't trust me when I say I actually have the full schematics of my parent's TV set, but it's true. It came along with the blue covered user manual, glued to the last page. At that time it wasn't a problem because hardware was meant to be repared, not thrown away on the first issue. It was actually much more ecological, and was even of interest to hobbyists to try to understand how it worked. At that time, there were many publications about RF and TV design, with explained schematics, an actually quite long tradition as even the vacuum tube radio era also had all those schematics published, most of which are still available on the Web. Of course, now TV sets are much simpler, all is crammed into a few ASICs and CPUs... But then again, the datasheets for the 30 years old chips used in the designs were also available, why can't it be true of current dies, whatever size they are ? We can't have regressed that much, can we ? I just hope we won't end up "buying" something with just the usufruct right of using it, not owning the real hardware to be able to adapt it to our real need.
All this is actually also true about computers. Yes, I do have the full schematics for my ORIC Atmos, which were even published in an issue of the french importer's magazine, "Micr'ORIC". And those magazines weren't afraid either of showing source code, often assembler at the time.

On fake trade secrets

If you actually develop drivers you probably know the common reason invoked when asking a company for specifications.
We are sorry, but this is secret trade secret and cannot be disclosed to maintain our competitive advantages.
The most obvious examples are graphics card makers, because they cost more and more, and use more silicium surface each year.
This simply doesn't hold, because:

  • That's what patents are for. They provide you a monopoly on the exploitation of an invention for a (quite long by comparison with current technology lifespans) time in exchange for public disclosure of the inner workings of the invention. (Though I should write about that plague that is called software patents one day, but that'd deserve a whole article.)
  • Just as with security, the "by obscurity" way didn't, doesn't, and will never work.
  • Each competitor in the market (certainly the GPU one) have largely the resources to employ an army of people to reverse engineer each others chip, and certainly does, making this "advantage" moot. Of course FOSS developers don't have this manpower available, so once again, they are the only ones hurt by these unfair practices.
  • Besides, mostly those NDA-covered specs are actually an excuse to hide hardware bugs they are just ashamed of anyway, so-called erratas...

A "special" case are WiFi chips. Makers hide behind an FCC ruling, that forbids chip makers from allowing users to use the chip beyond the allowed power for each frequency band. But of course chips now are based on generic circuits that are controlled by the driver which sets the frequency and requested power as it sees fit. Of course, opening the driver or the specs would mean the vendor "deliberately" gives the customer the possibility to break the FCC requirements.
But well, we all know this just doesn't work because in the end someone will rightfully reverse engineer the driver and write an opensource one which will allow it.
Btw, RE is absolutely legal in France, when done for interoperability reasons, which is exactly the case for drivers, and probably in other countries also... And of course the FCC has no jurisdiction in France (but probably there are similar legislation, I didn't check and it's 4am now).
In the end, it's the rights of the purchaser to actually use what they cashed for vs. a rule that will be broken anyway...

On tied sale

When you go to your local store, and want to buy a PC, usually you don't even get asked what you want with it, nor do most people wonder what they want with it.
You usually just buy a working "solution". And that "solution" happens to be a PC, with Windows installed on the disk. Or a Macintosh, with MacOSX installed as well, even though many other OSes run on the Macintosh as well. Most non-tech-savvy people actually don't even know what Windows is, let alone an Operating System. They just want something that they can use. But sometimes, what they need is not actually what they get. Sometimes they would need Linux, or BSD or something else, because it would suit their usage pattern better. But there is no way they can even know about this, because shops are contractually linked to OEMs and Microsoft, and are forbidden to ship with anything else than Windows. The contracts themselves are just secret, so you don't even know what it tells, and what threats are made to OEMs.

All could be well, if Windows was the only OS around, and if it was only available preloaded on PCs, then it would be just logical. But here is the trick: It is also available in boxed version, separately from the PCs. Therefore, it is obviously a distinct product than the PC. The french law, and many others alike, define the act of selling two separate products as a bundle without proposing the option to only purchase one as Tied Sale and clearly forbids it.

Yet all goes unnoticed, because vendors just don't care about "Linux freaks" who rant instead of just using what they are sold. For years, this fight has been disregarded as such by Windows users who just didn't care about their own freedom. Back then, BeOS also suffered a lot from it, and it was probably the major cause of its doom. And Linux didn't receive all the light it deserved. Until recently, where the Vista release actually changed the game. Indeed, long time windows users are starting to see the fight against vendor lock-in with a different angle, as they are getting forced into buying Vista when they still have their XP licence that they paid for and would like to use instead. Ain't it funny ?

The recent appearance of so-called "netbooks" (this term was actually a PSION trademark long ago btw...) seems to have changed the game a bit, but still, returns seem to be higher on Linux devices than XP ones, because people are still so impregnated by Windows they can't imagine something working just a little differently (eh, they even copied the Luna grey Windows decor on the eeePC Linux distribution, isn't that ridiculous ???). And of course, with drivers coming either from binary blobs or RE-ed sources vendors cannot really certify their stuff works with Linux anyway...

On availability of specifications

Now that I explained why open drivers are important, how it already used to be the way we are just asking it to be, why the vendor excuses simply don't hold, and finally why the computer and the OS are two separate products, it should become obvious that providing specification is not only needed, but also a moral (and maybe legal in some countries) right of the purchaser.
I don't know how to make myself understood... I mean, when you buy a tv, you get a manual telling how to use it. When I, as an OS developper, buy a computer, not an OS, I need a manual telling me how to use it. That manual is called the specification. It's the clearest I can be.

Yes, I know mainboard makers don't always have much choice over getting the specs or not. But they have the choice of buying from a chip maker that provides specs, and one that doesn't.

I recall of a sound and video card maker called Guillemot... which when asked to provide specs so others could write Linux drivers for them, replied "We will not provide Linux drivers for now", either not understanding they didn't have to, or just not getting it at all. I actually learnt later that they didn't even have the source code for the drivers of their own video cards, but instead got them readymade from nvidia... How could they just provide support for that ? That's desperating.

On DRM

Along with SATA, which I had to purchase an add-in card to buy a new harddrive because I couldn't find a 500GB IDE one and I didn't like being imposed that, I complained in my forum post about people asking for HDMI. Not because HDMI might be technically better than DVI (which I'm not even sure it is), but because this standard (I'm not even sure it can be called a "standard", is the specification publically available ?) imposes using DRM over it, which is both technically and ethically defective by design.

I've still yet to see any formal (read: mathematical) proof that DRM is actually possible.

Besides, it locks people into using their data on selected hardware, and, of course, selected Operating System(s), which is always welcome by our friends in Redmond and Cupertino. It denies you the very right of actually playing the data. As a trivia, when the French parliament transposed the EUCD (the european worse-than-original DMCA) into French law, then dubbed DADVSI, which of course started 2 days before christmas and was supposed to be done in one night, but ended up being a little longer, the parliament legalized and protected DRMs, forbidding writing software that broke them, therefore getting around my constitutionnally given freedom of speech (software is speech), and even funnier, they wrote in the law, that for each medium, there could be a limited number of copies allowed, which, in the case of DVDs would actually be 0. Yes, you read it, you have the right to make copies of your DVDs in France, as long as it's 0. Much much funnier than Ford's black or black cars, isn't it ?

Conclusion

I really wonder how much we regressed over the years... sure, the complexity of TV sets grew... but so did their design opacity. Same for other hardware.

As for MSI, well, it's probably not their fault, but I'll use another vendor for the time being.

I've just came up with this thought, and wonder why I didn't get it earlier in that formulation... It's so obvious now.
Hardware vendors who do not provide open drivers and specs are accomplices of the OEM imposing tied sale and Microsoft monopoly.
And here is why:

  • Not providing open drivers and/or specs implies not allowing FOSS/alternative OSes to get support on par with Windows
  • Other OSes not being able to grow proper hardware support prevents them from effectively competing with Windows
  • Not being able to compete on par with Windows due to lack of hardware support forbids OEM to eventually propose a viable alternative

As such, all vendors not providing the base requirement for OS developers are as guilty as the one maintaining its monopoly on 90% of the desktop OS market.

Hopefully I'll sleep better tonight, well the remainder of it, at least I feel relieved from something.

Oh, by the way...

;

Your brain has been tainted by 1 line of SCO code... You can't write Linux code anymore ;-)

Comments

Re: Say what you want from us but not what we don't want to hear

Very good article. I agree with all of it.

Re: Say what you want from us but not what we don't want to hear

I like Your articles, mmu_man. You have right!
Thank You!

Re: Say what you want from us but not what we don't want to hear

Sad but true
Merci François

Re: Say what you want from us but not what we don't want to hear

+1

Re: Say what you want from us but not what we don't want to hear

Well, I don't want to make any advertisement, but there is this Chinese product
http://www.lemote.com/english/yeeloong.html
I guess you already have heard about the microprocessor, the Chinese one, the longson.
In this page as you see they claim to be completely "open". I quote:
"he world's first fully open hardware/software. All system source(BIOS, kernel, drivers etc.) are open source, no close firmware needed."
However, you know...they are Chinese...
That laptop will compete with the eepc (though they will discontinue models smaller than 10" ) and the price is supposed to be released soon and lower than the eepc surf 4G (around 200 euros).
If anyone is interested I can try to contact them as I am currently living in China and can speak some Chinese. I have read in the Beijing Linux User Group that they were willing to give away some computers (maybe not this laptop). Nevertheless as François said, they might only know Linux.
Qu'est-ce que tu en penses François? (Me French 2 ;) )

Re: Say what you want from us but not what we don't want to hear

Oh yes, I recall reading about the cpu, but not the whole machine...
Though the first all open is probably the OpenMoko Neo... hmm actually I recall seeing one of the chip still needed NDAs to get specs about extended features... so maybe you're right then ;)
Yeah I know "some" chinese know how to fake openness sometimes, to hide other stuff... but eh, let's just hope the chinese people will someday be really free anyway. They are a great civilization.

I'd be happy to get one to make sur Haiku supports it :D
At least it's a good opportunity to show them Haiku and let them know Linux is not the only player in town.
Très bonne idée ;)

When I say DRM cannot work...

Well, I forgot to quote about the wonderful Blu-Ray DRM, BD+, that was supposed to last 10 years... That's just never going to happen :D

Re: Say what you want from us but not what we don't want to hear

mmu_man président !
mmu_man président !
mmu_man président !

:-)

Re: Say what you want from us but not what we don't want to hear

You're my new hero! Very excellent article. +1

There are people actively working to develop open hardware

I feel your pain. I started the Open Graphics Project in 2004 because I was frustrated with the state of open documentation on graphics hardware. Today, with the help of many people participating in the project and the Open Hardware Foundation, we're making some progress. Our ultimate dream is a sizable market of fully-documented hardware. Not only are we developing hardware that is fully documented, but the hardware itself is "open source", so if you want to learn from it, expand on it, or find something we neglected to document, all of the information you need is there. I think after we have achieved some more milestones with OGD1 (our FPGA-based graphics development platform), I'm going to look into my next pet peeve: WiFi.

Check us out:
www.opengraphics.org
www.openhardwarefoundation.org
http://en.wikipedia.org/wiki/Open_graphics_project

Oh, and let me not be self-centered here. Check out the Arduino, an open hardware project more successful than the OGP:
http://en.wikipedia.org/wiki/Arduino

Project like this are taking baby-steps right now, so they need your support.

Re: There are people actively working to develop open hardware

Yes, I know about OGD1. Would love to get it supported but I don't have the budget at the moment, but feel free to download Haiku and try writing a driver, it shouldn't be too hard. Don't hesitate to come and ask on IRC :)
About open hardware, there are other interesting initiatives like http://www.opencores.org/

Re: Say what you want from us but not what we don't want to hear

Agree, +1

There is an additional issue. This behaviour cannot be broken by letting people know which hardware to buy from manufactorers that can and will support FOSS development when then build a new machine, because many others will deduct from such a list that their hardware isn't capable of running Haiku, and stick to their well known
[][]
[][].
A little chicken and egg problem (allthough I logically solved that one).

Regards,

Arnold

Re: Say what you want from us but not what we don't want to hear

Of course not, but it's possible to "recommend" the open ones, maybe with a label (users like "Designed for Foo" labels).
What about "Any OS", much like the Any browser campaign ?

Re: Say what you want from us but not what we don't want to hear

+1 That's a pretty fine idea.
Look's like the logo designers lurking here have another challenge!

Follow up

I already mentionned the recent Blu-Ray DRM breakage...

Now it's even funnier:
Microsoft Begs Hardware Makers To Take Support Seriously.
Is the world going upside-down?
They are asking vendors to stop making Windows NT only drivers, because those don't work in Vista and Windows7, and would be the cause of Vista's failure! (The later is still to be proven though...)

It's nothing but fair that Microsoft gets to taste a bit of the very issue they created in the first place. They were the one inducing makers in releasing Windows only drivers by maintaining their monopoly, right? It's actually funny it comes back to bite them...

Releasing specs instead of wasting time making Vista/Windows7 drivers will ensure fair support for every OS out there, and maybe even provide jobs for many devs to write those drivers. It's their chance of ending their dependency toward Microsoft.

Besides, there is no way they will release Windows7 drivers for their old hardware anyway... Releasing specs would allow other people to write drivers for them at no cost that they wouldn't ever have done.

Re: Say what you want from us but not what we don't want to hear

Hey,

I totally understand your frustration, this is also why I very selectively buy my hardware these days. As a tip, consider the motherboards that are supported by coreboot, generally there is ample information available (reverse engineered or otherwise) about the chipsets and other components.

Couple that with a well-understood videocard like R200 based ATI, or a motherboard with integrated intel video hardware (although, these days most of the ATI chips are well documented by AMD) and you have pretty much the most open system you can buy these days. It should at least come with specs for pretty much everything.

Good luck!

Re: Say what you want from us but not what we don't want to hear

Francois Revol wrote: "Besides, there is no way they will release Windows7 drivers for their old hardware anyway... Releasing specs would allow other people to write drivers for them at no cost that they wouldn't ever have done."

But they do not want us to reuse their old cards with the new operating system WinNext using any newmade driver developed inhouse or freely developed by enthousiasts... they want to sell us their new hardware.

Re: Say what you want from us but not what we don't want to ...

Schematics have been on a slippery downhill slope.

My Amiga actually came with schematics, and this is as recent as early 90s.
10 years later, it was still perf