STPPC2x Beta 2- GP2X port of Simon Tatham's Portable Puzzle Collection

I've just released Beta 2 of my port of Simon Tatham's Portable Puzzle Collection to SDL/GP2X. It is much improved and the only remaining "problem" is that mines doesn't work. The other 26 games run fine from start to end.

I think the problem with minesis due to endian/signedness differences between x86 and ARM within the puzzle code itself (which I don't change in order to port any of the games). The game compiles and works under x86 Linux and runs fine but the GP2X crashes with an assert on mines.

The beta 2 version of STPPC2x (and the associated source code) can be found here:

http://archive.gp2x.de/cgi-bin/cfiles.cgi?0,0,0,0,25,2543

Or on the website where I keep all the released versions:

http://www.ledow.org.uk/gp2x/

GP2X port of Simon Tatham's Portable Puzzle Collection

Damn it's a long time since I programmed in C.

I've just spent the last few weeks porting the GTK/Windows/MacOS/Palm collection of games that are in Simon Tatham's Portable Puzzle Collection to work on the GP2X handheld game console.

That involved the creation of an SDL "frontend" for the existing puzzle infrastructure (which, I have to say was beautifully abstracted and documented from a programmer's point of view). It worked beautifully well on the whole. The puzzle framework is designed in such a way that it can be easily ported by just filling out the definitions of a few dozen functions. Basing my work off the existing GTK port, most of the simple functions were only a line or two to convert to SDL (using the SDL_gfx library helped a lot too). Even the font functions were quite simple, when using SDL_ttf. And because the GP2X is Linux-based, I was able to prototype all the functions using the GTK functions running simultaneously with their SDL equivalents on the same development PC, as you can see from the screenshot. Each game is running an SDL and GTK copy of the game and synching display and input between the two, while running on a normal Linux PC.

After they all functioned enough to give me a playable game in SDL, I started stripping out the GTK functions, libraries and dependencies and ended up with an executable that only needed SDL, SDL_ttf and SDL_gfx to run. That ran on any SDL-based architecture so I could compile the same code for either the GP2X or the Linux PC. For the GP2X it was the "simple" matter of cross-compiling them to the GP2X's ARM architecture, linking them against some ARM SDL libraries and then testing them on the machine itself.

But it has to be said that the hardest part of all was actually getting things to compile and link properly! I now officially hate Makefiles, gcc and linkers. I wasted more time trying to get code which I *knew* was working to compile and link in various ways without errors than I ever did coding! That really was the most horrible experience and half the time I had no clue as to what the problem was, even after analysing and googling the various weird and nonsensical errors I was getting. Programmers should really not have to play about with such convoluted and finnicky command lines just to get a simple C program with pre-compiled libraries to compile and link. Granted, wanting to compile a program for both x86 and ARM, with dynamic or static libraries should mean a little tweaking but it should never be as difficult as it was. I was actually *scared* to touch the Makefile each time in case I broke it and I kept more backups of that than I did the actual SDL interface code.

Anyway, I eventually got the puzzles to the point that they were mostly playable, mostly fun, mostly working and mostly pretty-looking. I've tested them all on GP2X and they are, on the whole, working (there is one that errors out in the actual game code, which I haven't touched, but only on the GP2X - a Linux machine with exactly the same code doesn't error. The other problematic one is just a bit too slow because it tries to do a lot in a short time on a slow processor). I was working my way through them one at a time to release them, because some of them exhibited subtle bugs, some of them required more work on input to allow them to "enter" numbers into the puzzles and at least one of them was (and still is) just being a pain in the bum. But now the vast majority of them work and I've built "beta 1" of what I have called (in a rare fit of originality) STPPC2x, or Simon Tatham's Portable Puzzle Collection for the GP2X. :-)

I'm surprised at how well and how fast they work, considering that the development 600MHz machine doesn't run them any faster than the GP2X at 200MHz and I'm not even using the hardware acceleration on the GP2X properly.

The first beta, with 25 out of 27 games playable, most of them near-perfectly, has been released on both the GP2X archive and at my website: http://www.ledow.org.uk/gp2x/ You can get the source code and toolchain that I used from my website too, because I find the most annoying thing about picking up other people's code is when you don't have the same compiler/libraries/filesystem setup as they do, so it's a problem to compile even before you start coding.

I consider it a good piece of work, considering that my last C excursion ended about an hour after starting it and I ended up writing the program I needed in something else instead. That was several years ago but I was able to pick this up and run with it. I estimate the total *actual* development time on STPPC2x to be approximately 48 hours for approximately 1000 lines of actual code (not including the Makefile), spread over a period of two and a bit months (it's hard to get in a mood where you want to fight with compilers, pointers and libraries). At least six of those hours was getting the Makefile to work properly. At least two were creating scripts to compile and link each file individually because I was sick of the Makefile not working properly and needed something that worked *here and now* so I could work on either the Makefile or the actual code at my leisure. At least ten hours on top of those 48 were things like "waiting for stuff to compile", "waiting for stuff to copy", "waiting for stuff to upload to my website/the GP2X archive", "preparing screenshots, instructions, proper licensing info, directory structure etc. for releases" and "waiting for the GP2X to finish writing to the SD card". A 600MHz development machine trying to copy megabytes of files over a USB 1.1 connection to an SD card isn't ideal, especially when you trying to write instructions, take screenshots, upload via FTP, etc. simultaneously. :-)



Anyway, it's been my first big programming project for a while. I'm always writing *something* whether it's a shell script or batch, a VB program to do a quick job etc. or something for my own entertainment but they all involve different standards of work - either I'm the only person to ever use the program at all, or it's for a limited technical audience, and it's very rare that the source code is ever looked at in either case.

In the first 24 hours, it saw 100 downloads from the GP2X archive alone and I haven't looked at my website's logs yet. At least I know that it was worth my while.

Darren Stone's WinXScreensaver mirror

Ever since I posted my article about the substrate screensaver, I've been getting emails of thanks (even though it wasn't my program!). Now, it seems, the author of the substrate screensaver port to Windows has let his website lapse and it's no longer available from there.

Well, that's not a big problem for substrate fans because I mirrored that particular screensaver along with the article I wrote and someone else has also ported it to Windows, as mentioned in the comments in the article above. But it now looks like people want the full program of WinXScreensaver with all the other screensavers and some curious screen-saver management abilities. Having scoured my extensive software archives of "stuff that'll come in useful one day", I have managed to locate a copy of this file (I never delete anything, but that doesn't always mean I can find where I put it. The Linux slocate command is worth its weight in gold as far as I'm concerned).

Given that it's based on XScreensaver which uses the X11/MIT open source license, I assume that the license of the Windows port allows me to mirror it and post it. I also assume that the original porter, Darren Stone of http://tron.lir.dk, doesn't mind me posting a copy here. Darren, shout if I'm wrong.

So, seeing as mirroring old files is now becoming a bit of a hobby for this blog, there is now a copy of the file up at my website.

It's WinXScreensaver1.1-Install.msi that I have, I have no idea if that was the latest version. Archive.org's history of the site is sparse at best. It's definitely an unmodified working version, though, because it's the one that I used to trial the program for something fancy I was going to do on my network. Hopefully people will find this useful. Enjoy! And give credit for the port to both the XScreensaver authors and Darren Stone.

My own piece of Linux "evangelism"

It's no surprise that I like to sing the praises of Linux. I've been using it, in one form or another, since the day I discovered it's existence.

Increasingly, however, when I read articles about Linux I am constantly annoyed at people's frustration that it's not how they want it. They read the part that said that Linux is Open Source (I don't usually capitalise those last two words but I feel I should start doing it) and can be customised and therefore they expect it to automatically do whatever they want.

I don't know when my annoyance at the lack of understanding of this particular "mantra" started but it has been recently exacerbated by articles on binary kernel drivers among others. I regularly contribute to several forums on Linux and School IT in general and a lot of people just don't seem to "get" Linux at all.

The binary kernel drivers argument was one of the first arguments I've had online where I've been so annoyed at the lack of understanding that I've pursued the question but only after going away for a while to make sure that my reply was calm enough. The discussion centred on the fact that Linux does not allow a stable kernel interface for binary drivers.

Now, the entire legality of binary drivers within Linux is one which hasn't surfaced properly yet and I can see that one day someone is going to get some nasty jaws appear out of the water and take them by surprise because they've misread the GPL. Leaving that aside for one moment, binary drivers are the bane of any Linux supporter.

Binary drivers are those without source code, like almost every hardware driver that exists for Windows. Companies like nVidia release drivers for their hardware in this form to avoid losing their precious patents etc. to a bunch of people who have bought their hardware and just want to use it. Fair enough, they have to make a living and if part of that living involves never releasing source code, that's up to them.

As a case in point, there do exist Open Source drivers for nVidia cards but they just don't feature the 3D acceleration that the binary drivers do (so you can use nVidia cards in a Linux system, but they won't have the same speed when playing games. The average desktop, however, would run just the same). Therefore, in this case, the drivers for the nVidia cards are only used by people who have spare 3D cards lying around in Linux machines that they are using at least some of the time for gaming. Professional users of 3D would probably not be using the nVidia binaries or even Linux.

nVidia achieved this marvel of modern technology (running 3D applications on a 3D compatible system with an nVidia card, where someone has already wrote 99% of the other necessary supporting code for them) by using binary drivers, which plug into the kernel at certain points. They evaded most of the technical and (possibly) legal limitations of interfacing with various versions of the linux kernel by releasing an Open Source kernel "wrapper" which lets the binary driver load itself without caring about what kernel it's actually running under and without the driver having to be rewritten for every kernel change.

The argument I had on Slashdot centred on people releasing binary drivers for Linux. The general overview was that Linux people were hostile and unhelpful to people writing binary drivers, that the manufacturers constantly had to keep updating the drivers for various kernels and that a stable binary driver API would help matters.

Needless to say, my reply was less than assistive in getting support for binary drivers in an Open Source kernel ("First, I think you're missing the fact that, overall, Linux doesn't care that you can't put your binary-only drivers on it").

The argument centred on the fact that companies will usually only release drivers as binary modules because they spend so long developing and testing their drivers that they are wasting an extraordinary amount of money if they then throw all that work out for anyone to copy it.

They don't seem to take account that Linux is entirely built on years of work that people, including many large corporations such as IBM itself, routinely give away for "nothing". Do you really think that the trade secrets in your hardware design are so fantastic that a) nobody has thought of them or that b) nobody who has your binary drivers and hardware couldn't reverse engineer them, legally, anyway?

Many of the network card drivers in Linux, for example, are reverse-engineered or written from open specifications and then placed under the GPL or other Open Source license. I can't see where this could pose a problem for the hardware manufacturer as they are effectively getting "free" drivers written for them, at little or no expense, as well as having those drivers supported and maintained for the forseeable future (at least until the hardware is considered obsolete and possibly even after that).

To quote Donald Becker's page (although this quote was written at an indeterminable date): "Linux now supports almost every current-production PCI Fast and Gigabit Ethernet chip!".

So every manufacturer of Ethernet cards has effectively had Open Source drivers written for them and distributed worldwide for free. I don't see any network card companies complaining about the fact that pretty much any network card inserted into a Linux machine is detected and used without having to download and install any driver, binary or otherwise.

It may be that the patents and trade secrets covering a network card are far fewer or of less importance than those covering a 3D graphics acceleration card. However, given the number of patents on items like TCP offload engines and the like, it seems unlikely.

If you are writing drivers for hardware, surely you'd be glad that someone is willing to ask you for the specifications of the hardware so that they can write and maintain such a massive, difficult piece of code as a hardware driver for another platform that you probably will never be able to support as well as your main platform?

The main misunderstanding comes from the binary driver "API" idea, the vision of a single standard interface to ANY piece of hardware within a computer and all of the associated kernel functions that will NEVER change. That sounds almost easy, no?

The Linux kernel is never easy. It changes every single day of its life. Unlike the major desktop operating system of Microsoft Windows, Linux is updated almost every minute by someone, somewhere. When the Linux IDE code was considered obsolete, unmaintainable and unsustainable, it was rewritten from the ground up. When that effort failed to stabilise quickly enough, it was restarted again on a smaller scale.

When the scheduler started experiencing problems on systems with hundreds of CPU's, it was ripped out, modularised and put back in. When USB and Firewire were introduced, they were bolted on to the existing frameworks and then rewritten time and time again to obtain a set of code that was maintainable and extendible for new standards like USB2.

At each point, everything was redesigned, not just re-engineered. People went back to the drawing board and said "Why are we still doing things for hardware we no longer support?", "Why are we bodging CD-writing by making an IDE-SCSI hybrid module?", "Why can't we use the SCSI code we already have to support these new fangled USB mass storage devices as well?"

Each time, any stable ABI would have broken. Each time, a new version of the stable ABI would have had to been released. It's not a stable ABI if it keeps breaking. On the other hand, if someone had said "SCSI works this way and you must not change it" then many things would not have been possible or would have meant reinventing the wheel for them to be supported.

When you consider the number of hardware interfaces that Linux supports (PCMCIA, PATA, SATA, PCI, ISA, MCA, PCI-E, PCI-X, AGP, USB, Firewire, I2C, the list goes on and that's just for the x86 platform) and take into account the amount of drivers for each style of interface (some of which share something like 99% of the code, most of which are completely individual) the fixing of a stable interface gets harder and harder without bringing the source code to an unmanageable level.

Yes, Windows does it to an extent. However, try to run a Windows 98 scanner driver in Windows XP. Most of them won't let you do it. The interfaces changed and are no longer compatible. Try and install an ISA card in a machine running XP, it won't recognise it. Microsoft obsoleted ISA cards because they felt like it. That's another issue, but what if Linux were to obsolete a major subsystem? First, there would be outcry, secondly, the code would be around so that people who WANTED it could still use it.

Try and get USB Mass Storage devices working on 98. You usually cannot without a specialised driver and even then, only on 98 Second Edition because the USB in 95/98 did not support it properly. The standard interface they chose in 95/98 WAS NOT COMPREHENSIVE ENOUGH to support Mass Storage Devices and had to be changed. 98SE made it possible by introducing new access methods but the drivers are usually totally different to those used for the same hardware under Windows 2000 or above.

Try and get most older games working in Windows 2000 or above. It's possible for the vast majority but far from easy and it's usually easier to run an emulator like DOSBOX or QEmu to do it because the interfaces and standards used in the DOS, Windows 3.1, Windows 95, 98 etc. era were obsoleted and changed and updated and even removed because they were incompatible with the "new" ideas going into later versions of Windows (e.g. early Windows versions had no real concept of multiple users on a single machine, early DOS games expected complete control of a processor in order to run and exact timings which aren't practical in a modern multi-threaded operating system).

Throughout the history of any operating system, and sometimes even applications, the set standards that seemed so perfect 10 years ago are never used properly or have to be worked around to make them work properly (consider things like 48-bit LBA drive access) and usually that means having to change the interface or corrupting it to your purposes.

Linux does not want to spend the next twenty years supporting it's own dreadful mistakes and misjudgements. Being Open Source, it does not need to. If something's wrong, they can change anything they like because no part of the system has to stay as it is. Linux is a liquid concept. However, should such changes occur many of the current binary drivers for Linux are likely to need substantial support in order to continue working properly, if at all. That support can only come from people with the driver's source code, i.e. the manufacturer.

When a kernel interface change stops PCI-Express cards from being accessed the same way they used to be, nVidia may be able to bodge something in their kernel wrappers or they may have to recompile their binary drivers to take account. Either way, they would have to spend a lot of time and money to support a change that is completely outside of their core business. People would be moaning at them for not doing their job. They would have to keep up, as they do today, with every change. And ten years from now, when they go bust, none of us will be able to use nVidia 3D acceleration on anything but the last kernel they supported.

Then again, they may just decide that it's too much bother and stop producing drivers for Linux. Were they Open Source or, ideally, in the kernel, the updating would *probably* be done for them automatically and without charge. They would be tested, without charge, by far more people than any beta test could summon up, with far more exotic configurations. Every time the kernel changed, they would be kept working until the day that there wasn't a single competent person in the world who wanted to keep supporting their hardware.

When IPv6 came along, Windows and Linux supported it by redesigning every piece of their networking code to take account of it. When IPv7 or whatever is next planned comes along, you're going to have to redesign everything all over again or put in some horrible backwards compatibility kludge to help older programs use it. That means that you will forever have to carry your older systems with you and all their backwards-compatibility layers, or you could just redesign the networking code to take account of it all for you so that old programs don't need to change and new programs can use the new features. They may be entirely seperate systems but why introduce a whole new layer if you could just slightly redefine one that's been working for years?

Binary drivers die a death as soon as the manufacturer stops updating them and are wounded by every kernel upgrade. Open Source drivers live for as long as there is a single person in the world willing to support them, barely feel a bump on a kernel upgrade and will stay in hibernation for as long as their source code exists, ready to be resurrected by anyone who wants to try them out (say, a computer museum curator who wants to run some ancient hardware card that has to be soldered onto a modern connector and have it's drivers tweaked to support the new, bodgeful interface).

A stable binary interface is impossible and totally against the idea of having a system that anyone can submit an idea for improvement to. When someone invents a better way of running hardware, all the internals HAVE to change or you end up with a mess of code that nobody in their right mind wants to touch and certainly not one which you would want people to be learning *from*.

Linux is bigger than an operating system. It's bigger than the companies that use it for commercial gain. It's bigger than the millions of people around the globe that use it every single day whether they know it or not. Linux (and Open Source in general) is about making things work, making them work well, making them work for the forseeable future, letting anyone see how they work, letting people come up with ideas for how to make them work better and keeping them working. None of those goals can be reliably met by using junk like source-less binary drivers or "stable" binary interfaces.

The annoying part is not that people disagree with the above, it's that they demand that Linux should change and not themselves. If Linux does not do what want, Linux is in the wrong. How often do they also swear at how stupid the design of some internal Windows API is? Linux is an emotional creature. It does not care about people who don't care about it.

If you want to rip Linux off and sell it with a thousand binary components and you can find a way to do it legally, even if you sell a unit to every single person on the planet, then you are on your own and Linux won't care that they break your system in their next upgrade. If you decide to Open your drivers and do the same, the chances are that someone will come along and help you to keep your stuff working, especially if it means that they get to play about with your system, ask questions, try new things, fix problems that only they have and can go off and learn from your code.

Open Source encourages software evolution. The better-written and better-performing man wins and their source code gets incorporated into more projects, their code gets learned by more people, who spread it to more code. Before long, every project needs this code to work properly, ensuring its own long life. However, if something EVEN BETTER comes along after that, it will be usurped for the greater good. If it's really better, it will take you over and smother you. If it's not, it will just linger and die and you will remain in your throne.

Binary drivers are one thing that I'd like to see smothered quite quickly. They are not necessarily better written or better performing but are kept there by corporations that are trying to gain money and recognition from Open Source without giving anything back. They legally, ethically and practically hinder alternatives from cropping up to usurp them as they know that they would be quickly smothered and left for dead. They need to maintain their little monopolies over their precious property. There is no analogy in nature for such a beast.

The fact that you can't manage to create a driver for the OS of my choice just means I won't buy your gear, or recommend it, or maybe even consider it. Whining about lack of co-operation from Linux people when I am happily running the product of years of their freely-given hard graft is not getting to get you any sympathy from me. If the drivers for your device came from your company along with the full co-operation of your company, I'd sing your praises and buy your gear. If you just want to cling onto the back of this Linux thing that people seem to be installing more of nowadays but not give anything back, don't expect Linux or its users to do you any favours either. I will only pile my money into something that I know will last me a long time and give me good value for money.

And now an admission... I own an nVidia graphics card - a weird one. It's a PCI Geforce4 440MX. Yes PCI. Not AGP or PCI-E. Bog-standard, old-fashioned PCI. I want to use it (it's my most expensive graphics card purchase ever at £50) and I don't want to replace it. That's not much money but the card is CAPABLE of doing everything I need. If I needed PCI-E levels of performance, I would have a PCI-E card.

I have it in my Linux desktop machine, primarily because that's what the machine used in it's previous Windows incarnation. I used to play Counterstrike and the motherboard does not have an AGP slot. The GeForce fitted the bill nicely. In Linux I have little or no use for it's 3D features (besides possibly the occasional game of TuxRacing) but it runs faster than the motherboard's onboard graphics.

I voluntarily use the nVidia binary drivers. The reason is that they provide better performance playing video, 3D etc. They prove that the card is capable of doing what I want. The binary drivers are not too much of a bind for me to recompile every time I change the kernel. They don't cause any crashes at all and the card works perfectly. If I need to diagnose a problem, rebooting without the nVidia driver but with the Open Source nv driver is not a big deal.

However, if nVidia updates their drivers to a version that doesn't support my card, introduces bugs, etc. I will not be upgrading to those drivers. If the Linux kernel people manage the technical/legal/ethical feat of making sure that nVidia cannot distribute any drivers but GPL ones, I will instaneously revert to the Open Source nv driver unless nVidia DO release a GPL one.

I won't be petitioning the Linux kernel people, I won't be rushing out to buy a new card that has got OS drivers, I won't buy nVidia's newest card that does run on Open Source drivers. I WILL be complaining to nVidia for not releasing the type of driver they should have released in the first place. I will use whatever works best for my current hardware to legally and technically interoperate with the rest of my machine's software.

If that means that, ultimately, my performance is reduced to poor levels because of having to use an inferior OS driver, I will be blaming nVidia for not bothering to contribute to that driver, to enable features that their hardware is perfectly capable of, and will adjust my next purchase according, to a company that does support OS and does not artificially limit the capabilities of a piece of hardware by refusing to openly publish code or specifications for it.

The nv driver already has what I need to run the card. Anything that isn't in the nv driver is due to nVidia not being co-operative.

I will not stop updating my kernel to the latest stable version, even if that means I break the nVidia card... an up-to-date kernel is worth much more than a single, replaceable driver.

I will not allow the kernel maintainers to be blamed for nVidia's lack of assistance. They do not care and never have cared about binary drivers and have stuck to their word on that.

I will not allow the nv maintainers to be blamed for nVidia's lack of assistance. They tried their best to get SOMETHING out of a company that wanted to give NOTHING.

I will accept it as inevitable that I knew I would eventually run into this problem, because I chose to use binary drivers for that component.

I will not be surprised if those same binary drivers stop working or fall into decay one day.

At least if I had open-source drivers which were capable of driving the card to it's full capabilities, I could keep running them through whatever legal or ethical turmoil Linux or nVidia goes through - that's the point of the GPL. If all else fails, I can still change the code myself (and I am capable of doing that) to make it work again. I don't have to rely on company X to keep my card working for me, breaking god-knows-what-else in the process. And I know that my hardware isn't part of some secret cover-up of something that I really don't care about when all I want to do is play TuxRacer.