Saturday, January 14, 2006

My own piece of Linux "evangelism"

It's no surprise that I like to sing the praises of Linux. I've been using it, in one form or another, since the day I discovered it's existence.

Increasingly, however, when I read articles about Linux I am constantly annoyed at people's frustration that it's not how they want it. They read the part that said that Linux is Open Source (I don't usually capitalise those last two words but I feel I should start doing it) and can be customised and therefore they expect it to automatically do whatever they want.

I don't know when my annoyance at the lack of understanding of this particular "mantra" started but it has been recently exacerbated by articles on binary kernel drivers among others. I regularly contribute to several forums on Linux and School IT in general and a lot of people just don't seem to "get" Linux at all.

The binary kernel drivers argument was one of the first arguments I've had online where I've been so annoyed at the lack of understanding that I've pursued the question but only after going away for a while to make sure that my reply was calm enough. The discussion centred on the fact that Linux does not allow a stable kernel interface for binary drivers.

Now, the entire legality of binary drivers within Linux is one which hasn't surfaced properly yet and I can see that one day someone is going to get some nasty jaws appear out of the water and take them by surprise because they've misread the GPL. Leaving that aside for one moment, binary drivers are the bane of any Linux supporter.

Binary drivers are those without source code, like almost every hardware driver that exists for Windows. Companies like nVidia release drivers for their hardware in this form to avoid losing their precious patents etc. to a bunch of people who have bought their hardware and just want to use it. Fair enough, they have to make a living and if part of that living involves never releasing source code, that's up to them.

As a case in point, there do exist Open Source drivers for nVidia cards but they just don't feature the 3D acceleration that the binary drivers do (so you can use nVidia cards in a Linux system, but they won't have the same speed when playing games. The average desktop, however, would run just the same). Therefore, in this case, the drivers for the nVidia cards are only used by people who have spare 3D cards lying around in Linux machines that they are using at least some of the time for gaming. Professional users of 3D would probably not be using the nVidia binaries or even Linux.

nVidia achieved this marvel of modern technology (running 3D applications on a 3D compatible system with an nVidia card, where someone has already wrote 99% of the other necessary supporting code for them) by using binary drivers, which plug into the kernel at certain points. They evaded most of the technical and (possibly) legal limitations of interfacing with various versions of the linux kernel by releasing an Open Source kernel "wrapper" which lets the binary driver load itself without caring about what kernel it's actually running under and without the driver having to be rewritten for every kernel change.

The argument I had on Slashdot centred on people releasing binary drivers for Linux. The general overview was that Linux people were hostile and unhelpful to people writing binary drivers, that the manufacturers constantly had to keep updating the drivers for various kernels and that a stable binary driver API would help matters.

Needless to say, my reply was less than assistive in getting support for binary drivers in an Open Source kernel ("First, I think you're missing the fact that, overall, Linux doesn't care that you can't put your binary-only drivers on it").

The argument centred on the fact that companies will usually only release drivers as binary modules because they spend so long developing and testing their drivers that they are wasting an extraordinary amount of money if they then throw all that work out for anyone to copy it.

They don't seem to take account that Linux is entirely built on years of work that people, including many large corporations such as IBM itself, routinely give away for "nothing". Do you really think that the trade secrets in your hardware design are so fantastic that a) nobody has thought of them or that b) nobody who has your binary drivers and hardware couldn't reverse engineer them, legally, anyway?

Many of the network card drivers in Linux, for example, are reverse-engineered or written from open specifications and then placed under the GPL or other Open Source license. I can't see where this could pose a problem for the hardware manufacturer as they are effectively getting "free" drivers written for them, at little or no expense, as well as having those drivers supported and maintained for the forseeable future (at least until the hardware is considered obsolete and possibly even after that).

To quote Donald Becker's page (although this quote was written at an indeterminable date): "Linux now supports almost every current-production PCI Fast and Gigabit Ethernet chip!".

So every manufacturer of Ethernet cards has effectively had Open Source drivers written for them and distributed worldwide for free. I don't see any network card companies complaining about the fact that pretty much any network card inserted into a Linux machine is detected and used without having to download and install any driver, binary or otherwise.

It may be that the patents and trade secrets covering a network card are far fewer or of less importance than those covering a 3D graphics acceleration card. However, given the number of patents on items like TCP offload engines and the like, it seems unlikely.

If you are writing drivers for hardware, surely you'd be glad that someone is willing to ask you for the specifications of the hardware so that they can write and maintain such a massive, difficult piece of code as a hardware driver for another platform that you probably will never be able to support as well as your main platform?

The main misunderstanding comes from the binary driver "API" idea, the vision of a single standard interface to ANY piece of hardware within a computer and all of the associated kernel functions that will NEVER change. That sounds almost easy, no?

The Linux kernel is never easy. It changes every single day of its life. Unlike the major desktop operating system of Microsoft Windows, Linux is updated almost every minute by someone, somewhere. When the Linux IDE code was considered obsolete, unmaintainable and unsustainable, it was rewritten from the ground up. When that effort failed to stabilise quickly enough, it was restarted again on a smaller scale.

When the scheduler started experiencing problems on systems with hundreds of CPU's, it was ripped out, modularised and put back in. When USB and Firewire were introduced, they were bolted on to the existing frameworks and then rewritten time and time again to obtain a set of code that was maintainable and extendible for new standards like USB2.

At each point, everything was redesigned, not just re-engineered. People went back to the drawing board and said "Why are we still doing things for hardware we no longer support?", "Why are we bodging CD-writing by making an IDE-SCSI hybrid module?", "Why can't we use the SCSI code we already have to support these new fangled USB mass storage devices as well?"

Each time, any stable ABI would have broken. Each time, a new version of the stable ABI would have had to been released. It's not a stable ABI if it keeps breaking. On the other hand, if someone had said "SCSI works this way and you must not change it" then many things would not have been possible or would have meant reinventing the wheel for them to be supported.

When you consider the number of hardware interfaces that Linux supports (PCMCIA, PATA, SATA, PCI, ISA, MCA, PCI-E, PCI-X, AGP, USB, Firewire, I2C, the list goes on and that's just for the x86 platform) and take into account the amount of drivers for each style of interface (some of which share something like 99% of the code, most of which are completely individual) the fixing of a stable interface gets harder and harder without bringing the source code to an unmanageable level.

Yes, Windows does it to an extent. However, try to run a Windows 98 scanner driver in Windows XP. Most of them won't let you do it. The interfaces changed and are no longer compatible. Try and install an ISA card in a machine running XP, it won't recognise it. Microsoft obsoleted ISA cards because they felt like it. That's another issue, but what if Linux were to obsolete a major subsystem? First, there would be outcry, secondly, the code would be around so that people who WANTED it could still use it.

Try and get USB Mass Storage devices working on 98. You usually cannot without a specialised driver and even then, only on 98 Second Edition because the USB in 95/98 did not support it properly. The standard interface they chose in 95/98 WAS NOT COMPREHENSIVE ENOUGH to support Mass Storage Devices and had to be changed. 98SE made it possible by introducing new access methods but the drivers are usually totally different to those used for the same hardware under Windows 2000 or above.

Try and get most older games working in Windows 2000 or above. It's possible for the vast majority but far from easy and it's usually easier to run an emulator like DOSBOX or QEmu to do it because the interfaces and standards used in the DOS, Windows 3.1, Windows 95, 98 etc. era were obsoleted and changed and updated and even removed because they were incompatible with the "new" ideas going into later versions of Windows (e.g. early Windows versions had no real concept of multiple users on a single machine, early DOS games expected complete control of a processor in order to run and exact timings which aren't practical in a modern multi-threaded operating system).

Throughout the history of any operating system, and sometimes even applications, the set standards that seemed so perfect 10 years ago are never used properly or have to be worked around to make them work properly (consider things like 48-bit LBA drive access) and usually that means having to change the interface or corrupting it to your purposes.

Linux does not want to spend the next twenty years supporting it's own dreadful mistakes and misjudgements. Being Open Source, it does not need to. If something's wrong, they can change anything they like because no part of the system has to stay as it is. Linux is a liquid concept. However, should such changes occur many of the current binary drivers for Linux are likely to need substantial support in order to continue working properly, if at all. That support can only come from people with the driver's source code, i.e. the manufacturer.

When a kernel interface change stops PCI-Express cards from being accessed the same way they used to be, nVidia may be able to bodge something in their kernel wrappers or they may have to recompile their binary drivers to take account. Either way, they would have to spend a lot of time and money to support a change that is completely outside of their core business. People would be moaning at them for not doing their job. They would have to keep up, as they do today, with every change. And ten years from now, when they go bust, none of us will be able to use nVidia 3D acceleration on anything but the last kernel they supported.

Then again, they may just decide that it's too much bother and stop producing drivers for Linux. Were they Open Source or, ideally, in the kernel, the updating would *probably* be done for them automatically and without charge. They would be tested, without charge, by far more people than any beta test could summon up, with far more exotic configurations. Every time the kernel changed, they would be kept working until the day that there wasn't a single competent person in the world who wanted to keep supporting their hardware.

When IPv6 came along, Windows and Linux supported it by redesigning every piece of their networking code to take account of it. When IPv7 or whatever is next planned comes along, you're going to have to redesign everything all over again or put in some horrible backwards compatibility kludge to help older programs use it. That means that you will forever have to carry your older systems with you and all their backwards-compatibility layers, or you could just redesign the networking code to take account of it all for you so that old programs don't need to change and new programs can use the new features. They may be entirely seperate systems but why introduce a whole new layer if you could just slightly redefine one that's been working for years?

Binary drivers die a death as soon as the manufacturer stops updating them and are wounded by every kernel upgrade. Open Source drivers live for as long as there is a single person in the world willing to support them, barely feel a bump on a kernel upgrade and will stay in hibernation for as long as their source code exists, ready to be resurrected by anyone who wants to try them out (say, a computer museum curator who wants to run some ancient hardware card that has to be soldered onto a modern connector and have it's drivers tweaked to support the new, bodgeful interface).

A stable binary interface is impossible and totally against the idea of having a system that anyone can submit an idea for improvement to. When someone invents a better way of running hardware, all the internals HAVE to change or you end up with a mess of code that nobody in their right mind wants to touch and certainly not one which you would want people to be learning *from*.

Linux is bigger than an operating system. It's bigger than the companies that use it for commercial gain. It's bigger than the millions of people around the globe that use it every single day whether they know it or not. Linux (and Open Source in general) is about making things work, making them work well, making them work for the forseeable future, letting anyone see how they work, letting people come up with ideas for how to make them work better and keeping them working. None of those goals can be reliably met by using junk like source-less binary drivers or "stable" binary interfaces.

The annoying part is not that people disagree with the above, it's that they demand that Linux should change and not themselves. If Linux does not do what want, Linux is in the wrong. How often do they also swear at how stupid the design of some internal Windows API is? Linux is an emotional creature. It does not care about people who don't care about it.

If you want to rip Linux off and sell it with a thousand binary components and you can find a way to do it legally, even if you sell a unit to every single person on the planet, then you are on your own and Linux won't care that they break your system in their next upgrade. If you decide to Open your drivers and do the same, the chances are that someone will come along and help you to keep your stuff working, especially if it means that they get to play about with your system, ask questions, try new things, fix problems that only they have and can go off and learn from your code.

Open Source encourages software evolution. The better-written and better-performing man wins and their source code gets incorporated into more projects, their code gets learned by more people, who spread it to more code. Before long, every project needs this code to work properly, ensuring its own long life. However, if something EVEN BETTER comes along after that, it will be usurped for the greater good. If it's really better, it will take you over and smother you. If it's not, it will just linger and die and you will remain in your throne.

Binary drivers are one thing that I'd like to see smothered quite quickly. They are not necessarily better written or better performing but are kept there by corporations that are trying to gain money and recognition from Open Source without giving anything back. They legally, ethically and practically hinder alternatives from cropping up to usurp them as they know that they would be quickly smothered and left for dead. They need to maintain their little monopolies over their precious property. There is no analogy in nature for such a beast.

The fact that you can't manage to create a driver for the OS of my choice just means I won't buy your gear, or recommend it, or maybe even consider it. Whining about lack of co-operation from Linux people when I am happily running the product of years of their freely-given hard graft is not getting to get you any sympathy from me. If the drivers for your device came from your company along with the full co-operation of your company, I'd sing your praises and buy your gear. If you just want to cling onto the back of this Linux thing that people seem to be installing more of nowadays but not give anything back, don't expect Linux or its users to do you any favours either. I will only pile my money into something that I know will last me a long time and give me good value for money.

And now an admission... I own an nVidia graphics card - a weird one. It's a PCI Geforce4 440MX. Yes PCI. Not AGP or PCI-E. Bog-standard, old-fashioned PCI. I want to use it (it's my most expensive graphics card purchase ever at £50) and I don't want to replace it. That's not much money but the card is CAPABLE of doing everything I need. If I needed PCI-E levels of performance, I would have a PCI-E card.

I have it in my Linux desktop machine, primarily because that's what the machine used in it's previous Windows incarnation. I used to play Counterstrike and the motherboard does not have an AGP slot. The GeForce fitted the bill nicely. In Linux I have little or no use for it's 3D features (besides possibly the occasional game of TuxRacing) but it runs faster than the motherboard's onboard graphics.

I voluntarily use the nVidia binary drivers. The reason is that they provide better performance playing video, 3D etc. They prove that the card is capable of doing what I want. The binary drivers are not too much of a bind for me to recompile every time I change the kernel. They don't cause any crashes at all and the card works perfectly. If I need to diagnose a problem, rebooting without the nVidia driver but with the Open Source nv driver is not a big deal.

However, if nVidia updates their drivers to a version that doesn't support my card, introduces bugs, etc. I will not be upgrading to those drivers. If the Linux kernel people manage the technical/legal/ethical feat of making sure that nVidia cannot distribute any drivers but GPL ones, I will instaneously revert to the Open Source nv driver unless nVidia DO release a GPL one.

I won't be petitioning the Linux kernel people, I won't be rushing out to buy a new card that has got OS drivers, I won't buy nVidia's newest card that does run on Open Source drivers. I WILL be complaining to nVidia for not releasing the type of driver they should have released in the first place. I will use whatever works best for my current hardware to legally and technically interoperate with the rest of my machine's software.

If that means that, ultimately, my performance is reduced to poor levels because of having to use an inferior OS driver, I will be blaming nVidia for not bothering to contribute to that driver, to enable features that their hardware is perfectly capable of, and will adjust my next purchase according, to a company that does support OS and does not artificially limit the capabilities of a piece of hardware by refusing to openly publish code or specifications for it.

The nv driver already has what I need to run the card. Anything that isn't in the nv driver is due to nVidia not being co-operative.

I will not stop updating my kernel to the latest stable version, even if that means I break the nVidia card... an up-to-date kernel is worth much more than a single, replaceable driver.

I will not allow the kernel maintainers to be blamed for nVidia's lack of assistance. They do not care and never have cared about binary drivers and have stuck to their word on that.

I will not allow the nv maintainers to be blamed for nVidia's lack of assistance. They tried their best to get SOMETHING out of a company that wanted to give NOTHING.

I will accept it as inevitable that I knew I would eventually run into this problem, because I chose to use binary drivers for that component.

I will not be surprised if those same binary drivers stop working or fall into decay one day.

At least if I had open-source drivers which were capable of driving the card to it's full capabilities, I could keep running them through whatever legal or ethical turmoil Linux or nVidia goes through - that's the point of the GPL. If all else fails, I can still change the code myself (and I am capable of doing that) to make it work again. I don't have to rely on company X to keep my card working for me, breaking god-knows-what-else in the process. And I know that my hardware isn't part of some secret cover-up of something that I really don't care about when all I want to do is play TuxRacer.

No comments: