LedowLand

Steam

2010-07-29T12:14:00.000+01:00

I've been singing the virtues of Steam for a while now. Lots of people find it very intimidating as a service but I've just been doing some quick mathematics and have confirmed to myself *why* I use it, even with it's known "disadvantages" (Most people's complaint is the "DRM" - Steam is the only DRM that I actually tolerate, it's so reasonable, and why people get out of their pram over other people knowingly and willingly buying into Steam, I have no idea).

My Steam account started in 2003 from a Half Life "Generations" pack (actually 2, one each for myself and my brother) that we'd been playing for years before Steam even existed. When WON died, we were forced to move onto Steam, and it generated a lot of bad feeling back then. Over time I slowly added games until my current situation where I have 141 games on my Steam account.

I went through the history of the account recently and worked out that I'd spent almost exactly £400 (not including the initial cost of the Generations Pack). Doing some quick mathematics on the exact figures, I worked out that my Steam account had cost me approximately £1.11 a week since I'd started it. That's £1.11 of my money, when I choose, spent how I like, when it's convenient to me. There is no game-rental service that can even come CLOSE to that, and I "own" these games forever (as much as anyone "owns" a copyrighted work).

For that price, I got nearly 150 games, of which, say, half a dozen were completely free and another half a dozen were freebie crap thrown in with other purchases that I wouldn't have bought but ended up with because they were free. That's £2.83 per game, and I apparently bought a game every 2-3 weeks on average over those years.

There were several times when I just did not have the money to buy the game I really wanted, or the one that I'd been waiting to be on a "weekend special" for years, so I actually paid more than I needed to in order to own what's on my Steam account now. I also ended up buying some games a couple of times over - sadly they weren't ones that would credit my Steam Gifts, so I couldn't give them to other people and they were "lost" purchases. I can remember one game that I bought and the next week bought again because it was in a ridiculously cheap bundle with lots of other good games that wasn't available the previous week.

So, how many hours entertainment have I got out of this Steam account? I have no idea. I can only go by recorded statistics which were added to Steam quite recently. That shows that I got 519 hours of recorded gameplay since they started recording. This, of course, is not spread evenly over all the year but also doesn't contain several thousand hours from the half-a-dozen years that Steam did not record such statistics. If I had to estimate *actual* game time on Steam alone, it would be at least 4-5 times that figure. Hell, my Counterstrike:Condition Zero stat alone would probably show more than 519 hours of gameplay.

So, what's that in terms of value - Well, I can easily say that with that huge choice of games available at the click of a button, I waste no time with crap games and only play those that I find entertaining. Some games have literally never been installed on my computer, others have never left it even when I was trying to find a few hundred extra MB's of space. Altitude has already knocked up over 100 hours of gameplay and that's only been released relatively recently. According to Steam, I've played 16 hours of games in the last two weeks - an hour per day, or thereabouts. Assume that's the same since I had the account (and that sounds a very reasonable estimate) - that's 2500 hours at least since I started - which is marvellously in line with my 4-5 times estimate. So let's say I got about 2500 hours of playtime out of 141 games for £400 since I started my account in 2003.

So how much did this entertainment cost me per hour? £0.16p. Let's compare that to some other leisure activities:

World of Warcraft: £8.99 for a month's subscription. Assuming you only ever wanted to play that game, with my same usage (about 7 hours a week) that would be £0.32p per hour. Close, but that's still double the price of my Steam account, and only on a single game (albeit an MMORPG).

A cinema ticket: £9.80 for a basic seat in my local cinema, during their quietest time to see a just-released, non-3D, 140 minute movie - £4.20 per hour. One-off and can't watch again without paying.

A DVD: £9.99 for a just-released DVD of a 102 minute movie from Amazon - £5.87 per hour (but arguably the same price as the cinema on average). I can keep watching that over and over as much as I want, though.

A Blu-ray: £15.93 for the same movie - £9.37 per hour (assuming I had the equipment to play it). Again, I can keep watching that over and over as much as I want.

Paintballing: £9.99 per person for a full day (7 hours) - £1.42 per hour assuming I could find enough people also willing to pay that. A one-off event.

Abseiling or caving: £18 per person for a three-hour session - £6 per hour. A one-off event.

Pottery course: £143 for a 15-week course (two hours per week) - £4.76 per hour. A one-off event.

Now, you can argue that similar activities are probably more exciting or more worthwhile, that I gain knowledge / skills, that I interact with people more, etc. And, great, that's good and I obviously do some or most of the above anyway. But if you're a gamer, it's undeniable that you're getting lots of good value for money there compared to some other similar "time-wasting" activities that you could do to fill up a slow Sunday afternoon. 16p / hour is a fabulous price for myself to pay - I can't find anything else that I really get that much entertainment out of for such a cheap price, and I *have* had to be dragged away from the computer by partners in the past.

The reasons for the value gained, of course, are many.

1) Steam give some good bundles and discounts. Even if you don't include their regular "Free Weekends" of various games, they often have astounding weekend deals like a bundle of modern, big-name games for £10 that were selling for £50 in the shops.

2) You can choose what you buy and when - which obviously biases it to your exact tastes. Don't have a game to fit that hour of spare time, or for your little brat of a cousin to play when he comes round? You can probably find something in Steam's Under £4 or Under £7 categories. Strangely, they're the only pricing categories available at a single click and yet £7 wouldn't even get you a budget game in most video game stores. The range of games is huge, too, and virtually every genre is covered.

3) The pricing is immediate, sometimes short-lived, the purchase goes through instantly, the download starts immediately after, you don't have to configure or install things - 99.9% of stuff will "just work" after you click download and double-click the game - and so impulse-buying is a big factor. Steam is probably the only program that I allow to "keep me informed" of new releases. I abhor advertising and have learned to ignore but, actually, the Steam deals are worth looking at quite a large amount of the time. Hell, some of the games I already owned, I bought the steam version so that I could install them conveniently.

4) Freebies. If you have an ATI graphics card, you get a freebie. If you have an NVidia graphics card, you get a freebie. If you just hunt down some forum posts you can get free games entirely legitimately (http://forums.steampowered.com/forums/showthread.php?t=851573) - Alien Swarm is the latest one, and Valve made a big fuss of advertising that *everyone* can get it for free, but there are lots of other good games there. And that's before you even look at the demo's of existing games that are available, or the numerous free Steam Mods if you have the Orange Box. Let's not forget - Steam is free to own and there's a dozen or so free games on those lists. They may not be brilliant (but Peggle Extreme made me buy Peggle, the HL-based mods available are out of this world, and TrackMania Nations will make you lose time like there's no tomorrow - and made me buy TrackMania United) but they are good enough, and they are free.

5) The extras - you can move your Steam account to any PC you own (or don't!) and download the games and play them. Most games will even carry over your settings and savegames too. You can even shift quite a lot of stuff onto a Mac. So long as you don't try to run your account on two computers at the same time, you can install it and play it wherever you like and get all the games you've purchased (or been given as freebies, or as gifts from Steam friends). Most games have achievements, rankings, stats so you can find a reason for even that brief five-minute game just before dinner ("Hold on, darling, I just need to kill two more people with a grenade and I've got all my achievements for this game").

Steam is undeniably good value. Maybe not for every single individual gamer on the planet but I can't deny that I've more than had my money's worth out of the games on it. The only other thing that comes close was GOG.com (Good Old Games) which sell DOSBox-wrapped and other older games for about $4.99 each. So do Steam sell those sorts of games, though. And they also sell top-of-the-line, pre-release, etc. games that need a huge stonking PC to run them.

To me, Steam is one of the best game-purchasing methods out there. It makes me want to add games to my account, it makes me want to play the vast majority of the games that I've put on there, it makes it easy and simple to play them and it costs me less than 16p for every hour that I choose to play. If I had more time, had no life, were interested in more games, I could easily spend a lifetime on there and, even with every Steam game added to my account, still get the value down to about 10-15p per hour. There are other costs, of course, but they are incidental to my usage of Steam anyway - Internet bandwidth, PC cost, hard disk space, etc.

It's truly ridiculous to then look at such "new" competitors as OnLive who are demanding the same game prices (and it looks like they just photocopied a portion of the list of Steam games), plus a monthly rental, plus you supplying roughly the same hardware but 10 times the Internet bandwidth. Or a video game rental store. Or purchases from a video game store. Or most of the download-software stores.

Steam truly is remarkable value. I gain nothing by saying so, but hell - the best £400 I've ever spent. I know some people who could burn through that amount in a month's, let alone seven year's, worth of gaming.

Copyright, and the UK's Digital Bills to clamp down on my viewing habits

2010-04-12T12:20:00.003+01:00

Just been reading The Register's article on the Digital Economy Act. Thought it was finally time for a bit of a rant...

The way forward is, was and always has been encryption. It just wasn't needed at any point in the past because the chances of being caught, the implications of being caught and the mass of possible defences made it a non-issue to be caught. Tor exists and does a pretty damn good job at stopping people knowing who you are. It's just that nobody has bothered to integrate such facility into a popular file-sharing protocol that's widely used because, to be honest, it hasn't been needed. When the necessity arises (and such a Bill being implemented in such a way *in real life* and not just in a politician's head would be that point), encryption destroys all the chances of monitoring such activity.

But "Piracy" as a concept isn't something that will die easily because it's mainstream and solves many mainstream problems much more effectively than anything else available.

The piracy problem, as thus stated, won't be solved by "looking for" people doing it. The record industries did that with a group of individuals and most of them either ignored their demands, provided a reasonable defence, cost too much to convict or when the record companies took them to court they made enough of an argument to stop things happening. Literally only a handful of cases ever made court, maybe a lot more were settled out of court, and still it's a tiny, tiny, drop in the piracy ocean (with maybe a handful less grannies and 10-year-olds downloading songs because they "got caught").

When you look for people, you only find those that *want* to be found or that aren't taking care not to be found. The people who are the real problem (the warez groups, the industry insiders providing pre-release movies, etc.) very, very rarely ever get traced back and hardly ever face any sort of sanction. Who gets stung? The people trying to view the content they would like to see. And those same people 99.9999% of the time have a reasonable defence, whether the government likes that or not (and, no, I don't consider "I'm pirating out of protest" to be a defence - but something like "I own it on DVD and wanted to watch it on my foreign DVD player" I consider pretty reasonable).

Of course, there is an obligation to uphold the laws but the fact that piracy is *so* prevalent, so tricky to define, can be made *so* hard to track / prove by the simple step of running, say, TOR, can be understood and performed by uneducated users, and provides material that just isn't available any more means that's it's not going to be pushed underground quietly.

It's almost a service now. And the problem is that the whole piracy thing goes ten million times deeper than "kids posting MP3's online". Let me give you some real-world examples that have occurred since only the beginning of the year (so only 3-months or so).

I was on a trip around Europe with some friends. We wanted music to play in the car. I don't buy, download or otherwise listen to music (the only CD I have ever personally owned was Jive Bunny and that was because someone bought it for me to go with a CD player they'd bought me, which never got used) but my passengers insisted on having something. They copied the tracks from their iPod to an SD card and we played it in the car (the fact that I have only a tape-player and that we had to use an SD-to-tape adaptor should tell you how often I listen to music).

Technically, I'm not even sure if that's legal or not. Is that "broadcasting"? Is that distribution? Should I be required to report my friend? Did my friend pay for the track originally? My friend is Australian - does that mean she might have contravened international laws and/or license restrictions on her purchased music? I have no idea whatsoever, and the SD card got wiped because I needed it for my digital camera. To the layman, we were just playing some music in the car, the same music we hear on the radio for free but without the annoying guys talking over it or stupid "customisations" of songs (we heard Capital's version of Kesha's Tik Tok which had a ten-second intro "sung" especially for Capital... it just made us laugh).

I was talking to my girlfriend about an advert that used to be on TV. She's foreign and had never seen it. I found it on YouTube, in what looked like a recording from an old videotape. Is that copyright infringement? Should I have reported it? Does the company that made the squirrel-assault-course ad for Carling Black Label care, or are they grateful for the publicity? How long would it take to find all the rights-holders for that 30-second clip and ask their permission? Was Youtube broadcasting it in contravention of international / UK law and/or was I doing so in showing my girlfriend?

I wanted to show her Button Moon, Count Duckula, Trap Door and Dangermouse. I have a load of episodes of each on DVD (not to mention the complete set of Batfink and Dungeons & Dragons cartoons) and the DVD's were in the room next to me. Time how long it takes to hunt a disc, remove from box, insert it into laptop, set up the DVD software, wait for the disk to spin up, etc. It was ten times quicker to go on Youtube and show her them. Was that wrong? Does the average person understand that stuff on Youtube may not be there legally? How does that mesh with content-holders who post *all* of their stuff legitimately on Youtube? Does the rights-holder for some of that material even exist any more? I have no idea. Technically, Happy Birthday is only licensed legitimately if you pay money to a corporation... how many average people even *KNOW* this let alone care, let alone would bother to report themselves, let alone pay, let alone want to sing it if they had to pay for it? It's no excuse, the law is the law, but the law sometimes make an ass of itself by painting itself into a logical corner filled with nonsense and then realising its mistake only decades later.

The same girlfriend had never seen The Good Life, and I wanted to show her. I have the complete set of DVD's at home with every episode ever made, but they were about 3000 miles away. I couldn't find anything but clips online except on torrent sites, and we were having a quiet evening in and wanted something to watch. I downloaded the torrent. It took three clicks, 20 minutes and less than a Gig of disk space and I had every episode. We watched them. Is that illegal? Is it immoral? Did the BBC suffer through my actions? The program is over 30 years old now, are the BBC still making money from it, or relying on it to generate revenue for them, or expecting it to do so indefinitely?

The same girlfriend also loves QI. We have all the books and DVD's. We watch one or two every night, along with things like Have I Got News for You or Whose Line is it Anyway. We were abroad, though, and didn't want to miss the latest one, but I couldn't access it. I used a UK proxy and get_iplayer and after about 5 minutes had a copy sitting on the hard drive. I was very, very careful not to download it while it was being broadcast because that would be illegal because I don't have a TV licence at home (no TV's!) and I knew I was bypassing the BBC's protections already. I knew my actions would be traceable but I didn't really care about the temporary storage and later viewing of that episode, though, only the TV license. I have quite a good knowledge of copyright law, but even I know that's still a grey-area and I still "take the risk". How does the same situation translate to a layman?

I've religiously bought every one of the QI DVD's and ripped them to my laptop (our primary viewing apparatus) for convenience - the DVD's occupy a coveted and well-guarded section of my DVD-shelf and would just add to my baggage allowance. Have I broken DMCA-type-laws by downloading them illicitly, or by decrypting the DVD, or transporting them to regions they aren't licensed for? Am I hurting content producers? If I let my girlfriend's family watch the episode with us (they don't speak English anyway, but hypothetically), is that illegal? I have *absolutely* no idea.

We both love Whose Line Is It Anyway and love the fact that 4od has them all online. We're slowly working our way backwards (the early series were crap) through them all. I used a proxy to do this from a foreign country after reading the following snippet from their FAQ:

"Can I watch 4oD in another country?
Rights agreements mean that our 4oD service is only available in the UK and the Republic of Ireland, (although C4 does not always have rights for programmes in ROI). Even if you are a citizen of the UK or ROI you cannot access the service from abroad"

It was easier to leave a program running on a computer left at home because I *know* I'm English and that if I was at home, I could watch them. I have lots of the DVD's but they don't really have the complete set available in a simple fashion (it's lots of old VCR compilation-tapes and things to get *every* episode, I believe). We even looked for a way to buy/download the episode quickly online, hoping that a British-registered card would allow us to download them even abroad. No chance, according to Wikipedia:

"A Download to Own (DTO) or "Buy" feature was once available on selected content, allowing users to purchase a programme and keep it for as long as they wish. This service has now been suspended..."

But now all their programmes are available for free on 4od or (some) on Youtube. So we went via a proxy (read: my home broadband connection) and we suffer the 20-seconds of ad's quite gladly because they are reasonable (like ad's on TV used to be). They get an extra viewer of their ads and content (which is what their entire business model is based on, I assume, being a TV channel?) and we got the episodes we wanted to watch without too much hassle. Did I break the law? Have I harmed Channel 4 by watching these programs that haven't been filmed for about 20 years with some adverts in them, bypassing their "region protection"?

My ex is a big, big fan of Just Good Friends (remember that?). I bought her Series 1 and 2 from Amazon and we were waiting for Series 3 to come out. Every time the date given approached (even with industry blogs saying dates, and Amazon pre-order dates, etc.) it would slip. Eventually they gave up giving dates and still now, five years later, you can't buy it. It's assumed that it's because of music-licensing rights to some of the music playing on the jukeboxes in the background in the pub and various minor things like that. Have I destroyed Ronnie Hazlehurst's life by downloading that last series so we can finally watch it after about 30 years since it's original broadcast, and five years of waiting for a DVD to come out? Do you really think that a person who's waited that long *wouldn't* buy the DVD if it were to come out tomorrow, just to complete their collection? You can't gift DVD-R's with some torrented AVI's on them - it's not the same. Incidentally - anyone ever seen "The Two Of Us" with Nicholas Lyndhurst available for sale *anywhere* on any format? Me neither. But if I found a torrent, I'd want to see it again. But EVEN THEN, if a DVD was released, I can name at least five people it would make a good present for (myself included!).

My girlfriend's father is a big WW1 fanatic. He has a huge collection of memorabilia and regularly goes metal-detecting in the mountains to get more. I wanted to show him Blackadder, series 4, which is set in the trenches. He doesn't speak English so he can't "watch" it but I had a quick look for it online in case there were Italian subtitles available. I didn't find any (if anyone knows of some, please do tell) but I did find the entire Blackadder, all series, available for streaming from some random website. Almost certainly illegal. Did it really hurt anyone to show him a brief few seconds and explain the jokes and let him look at the set, uniforms, etc. to see if they were accurate? If I could have found some Italian subtitled DVD's, I'd have bought them but would it be a big deal to show him an episode with Internet-posted subtitles so he could see if he'd enjoy it? Are the sites that host foreign subtitles to things like that breaking the law?

Since New Year, I've paid about £200 and downloaded dozens of games for that money from GOG.com and Steam. It's said that Steam uses a Bittorrent derivative to download games quickly to my hard drive (they employed Bram Cohen, author of Bittorrent). All I know is that I click a button, enter my card details and in about ten minutes I have a game on my hard drive. The protocol, then, isn't illegal so it's an enormous job to distinguish legitimate downloading from piracy. Ever Linux distro I've ever used has a Bittorrent download too. I'd hate to go back to the days of HTTP / FTP downloading a 4Gb ISO.

I've been in schools that like to show DVDs to kids at the end of the year (on their "fun days"). I have absolutely no idea about the legality of that at all (I'm pretty sure that's broadcasting). I know that the music teachers have to tick a sheet saying which hymns they've sung that year in assembly so they can be charged for them (ask your child's school - it's pretty universal and quite expensive apparently). I don't think Disney do the same but they would certainly do it if the school would pay. I've also seen people copying tapes of educational programmes (that they have bought legitimately several years ago) that are no longer broadcast onto a DVD so they can play them on the modern projectors through their laptops. Is that illegal? I have no clue at all. What about kids going on Google Images and copying/pasting an image into a Word doc? Stupid, from a filtering point of view, yes, but almost all schools do this. Is that infringement?

So, with all the DVD's I've bought and can't use in the quite reasonable way that I want without (potentially, theoretically and only "possibly") breaking the law, with all the money I've spent online for various games, books, movies, etc., with all the stuff I have on my Amazon wishlist (if it even exists online) and want to buy, with all the content I want to absorb each night, I've somehow played my part in destroying the market and should be monitored, penalised and even taken to court?

The average person just can't see sense in that at all. And so the average person just doesn't *care* about piracy, because they've probably done it inadvertantly dozens of times in the past year. Did you publically perform Happy Birthday this year? Have you photocopied pages from a book? Did you put your music from your CD's onto your iPod? Have you watched YouTube clips?

When a law become ridiculous, it gets ignored. Eventually it gets removed from the statute books because it was worthless, stupid, unenforced and ignored. That's the way such "laws" were heading and these Bills only put a blip in that death-curve. Taxi drivers don't need to keep straw in their boot any longer, because it's irrelevant, outdated, nonsensical and pointless. But for many years, that law persisted even into the era of the motor vehicle. Copyright laws are in the same position, but this time we're not moving from horses to cars, we're moving from traditional media to the Internet.

I don't really care about the TV/movie/music industries. As far as I'm concerned they should make money where they can and if they can't, they should pull out of that market, the same as every other business. The fact is, though, that the only legitimate sites I've ever used are BBC iPlayer, 4od, ITV Player (since the stupid Silverlight requirement was scrapped) and similar. I get the content I can on DVD (because it's the least evil digital method, not because it's perfect) and even go to the extent of buying things on video from bootsales so that my digital content is "legitimate". And even my uses of those sites / media are potentially infringing under certain interpretations. I try hard to stay within the law, but the fact is that staying within the law while performing some quite harmless actions is hard - thus it's a bad law.

There's a point where a reasonable person has to make a choice - (potentially) break a (badly-written, outdated, irrelevant and relatively untested) law or stop consuming certain media. The problem is that piracy measures *don't* add a third choice, they just force you to make a choice between those two even more quickly and decisively. I gave up my TV licence because it wasn't value for money. When "piracy" becomes "criminal", then I have to make another choice - continuing consuming that media (potentially against the law) or take up mountaineering, learn Italian, play board games, etc. Neither of those options help the content makers, but they are precisely the ones forcing me to make that decision.

You know what? I'd rather buy second-hand tapes and DVD's at a boot sale of 30-year-old programs that they don't show any more and keep an old VCR running (and risk potential legal action) than use the other legitimate options available to me. That has to say *something* to the media industries surely?

Please, please get real. Come into the land of the sane. It's nice there, honestly. A £1 / episode download link on iPlayer / 4od / etc., a "Buy the DVD" option for just about everything in your archives. Stop pissing all the money I'm giving you away on trying to criminalise my quite reasonable actions to watch YOUR content that I enjoy. It's a nonsense, and the sooner you force me to make that decision the better.

Hobbyist programming advice

2008-06-05T10:09:00.021+01:00

I've programmed quite a bit, in a variety of languages, for work, for fun and for learning. Quite often, I've chosen how I do it and the other times it's been dictated but I've found that I program to a set of rules unconsciously. These mostly apply to "hobby" coding (games, utilities, etc. that you are mostly making for yourself) but can also apply to school or work if you have flexible working regimes:

1) Program in whatever language you like.

Honestly. It really doesn't matter for "casual" use. Don't program in C just because everyone else is, or because you've heard that it's better. Program in a language that you're comfortable with, even if that's BASIC and people tell you it's rubbish. You're the programmer, you should work to what you know. If you want to learn new languages, do so, but don't write every project in the current-fad language just because.

This also means that if you are confident in several languages, you should program in the most appropriate or your favourite language. It doesn't matter which! The most appropriate will make things easier (e.g. using something like Perl for handling a lot of text, or using Java for an object-oriented idea) but your favourite is likely to be the one you know best and you can make it do whatever handstands you need it to despite its shortcomings.

If someone tells you that you should start a program in C because you might need the performance later, test their theory first. Knock up a worst-case in your favourite language and see if it's worth the effort of learning a new language or coding in an unfamiliar one before you start. Chances are, for most things, just about any language will do.

There's nothing worse than being told that you "should" be programming in C when you're knocking up a twenty-line batch file (yes, batch files are programming, of a sort!). However, knowing whether you want an interpreted vs compiled language can be a help when you start - the chances are that it won't matter though.

2) Ignore coding style.

If you're writing for yourself, you can ignore supposed "best" coding styles and work your own way. Even if you're working as part of a small team what matters is clarity. Whether you space or double-space or crunch your arguments together with their commas doesn't matter so long as it can be read and understood. Stylising the code can come later if it's necessary. And if you're the only person to ever read it, it doesn't matter that people aren't used to your coding style.

I would argue that programming teams should have a system where they all use their own coding styles and have "convertors" so that when they put code back into the global pool, the style gets modified to the "team" style and when they check it out, it gets customised to that particular programmer's style. This way, everyone works comfortably but the central repository stays consistent.

If you're a single developer, spending too much time on worrying about the style wastes the "programming mood". You could write a thousand lines of new code with new features in the time it takes to style your code.

3) Document the code.

Documenting your code is important, even for yourself. You will not get to the end of any significant project and remember why every line is in there or how they all work. But you don't need flow-charts, invariant lists, hundred-pages of specification and all the other rubbish that programmers are taught to do if you're just working for yourself.

A simple comment above each block of code is more than sufficient but what's ABSOLUTELY necessary are warnings to yourself:


REM Don't remove this, because the program crashes without it.

// Don't pass different flags here because they are ignored.

;This line isn't redundant, it's deliberate.  Without it X happens.

/* I know it's already supposed to be initialised but
without this, it can pass uninitialiased through
the compiler unnoticed and cause problems.*/

For big projects, you might find a brief Changelog and a Todo come in handy, but everything else should be inside the actual code. It takes long enough to write user documentation without having to document every single thing you do for yourself.

4) Backup

Yes, I know, but you really need to, especially if you're programming. Programs can crash, file accesses can run amok, source code will be heavily edited to fix a basic problem that you'll track down to something else, etc. Ideally make a copy of files you intend to change just before you start a session, and after any major breakthough (which may be as simple as finding the 1-character cause of a serious bug... you will kick yourself if you forget which of the hundred lines that mention "variableX" it was on and have to go through hours of debugging to find it again).

Regularly copy those files to other computers, or a website somewhere. SVN and other source code management programs are useful for this if you do it often but even just a simple ZIP and upload is better than nothing. THIS INCLUDES MAKEFILES AND COMPILE SCRIPTS!

I work on the following basis: At the start of a session, I backup the files I'm likely to change the most (e.g. graphics.c if I'm planning on revamping the graphics), to a dated copy in the same folder (e.g. graphics-2008-01-01.c). If I make significant advances (add a new feature, fix an old bug, etc.), I will copy that file again to the same folder (e.g. graphics-2008-01-01-added-double-buffering.c). At the end of a particular session, I copy all changed files to an SVN folder, which I may or may not decide to "commit" to a remote website. Every time I think I've made enough advances to see that the program obviously works differently, or which would be a real pain to replicate, I copy them to a couple of personal websites/SVN repositories.

Even few months or so, backup all that stuff to a CD-R or something. You'll be glad of it in a few years time. "I used to have code that did something similar, let me dig it out for you", etc.

5) Develop on a slow PC.

Seriously. Find an old junker, a laptop if possible because programming is one of those "I'm in the mood" hobbies that can take you any time and it's a shame to waste it. Don't make it so ridiculously bad that you can't use it comfortably, but a machine with less CPU and less RAM will show you the second you start to implement code that is detrimental to performance.

And it will also encourage good practice. Any program that's slow on your development PC will be unnecessarily slow if you ever do intend to scale it up somewhere else. So when that script to rename your MP3's starts struggling with 1000 files on your old laptop, you know that you should fix it before you convert it to rename 10,000 user accounts on the server in work. And when that game starts running out of memory with only a 10x10 board on your laptop, you know exactly how much it'll take on a modern PC before it starts exhibiting problems, when normally you wouldn't have given it a second thought.

This is especially important if you are developing for embedded systems, handheld games consoles, homebrew circuits, anything that's underpowered. You don't need to program on something of the same MHz or less of your target machine, but you need to be able to get an idea of where your program is slowing down while still using your development machine.

Using an old machine will also deter you from using your main desktop to program on. This can be important for a number of reasons - First, the source code (and associated backups) of any prolific programmer will get out of hand very quickly and you don't want it spread across your system. Secondly, you can backup to a different machine easily as part of the "routine" (e.g. a good system is to develop on a PC which is tricky to connect to the Internet - my old laptop has wireless but for simplicity and security, I set it up so that it proxies through my desktop machine for web access. This means that anything like FTP, SVN, etc. I have to deliberately enact or do from another machine, and it's slower to do so, which also provides the incentive to backup the same files to other machines while I'm waiting). Third, it removes the risk of damage to your everyday machine. This is especially important if you are using pointer-based or low-level languages that can potentially crash or corrupt a machine.

6) Test

If you intend to distribute the end-binaries to anyone, test it thoroughly. Good feedback is significantly hindered if your early versions have lots of basic problems or dangerous bugs. And get lots of people to test it for you. Announcing version 1.0 suggests that you've had lots of people testing it, believe it or not, so make it clear it's a test/0.9/beta/prelease version and then improve on it to make the magic 1.0 something special that works first time for everyone.

Testing is multi-stage. You need to test it thoroughly yourself. This usually means running the end program an awful lot of times, with different inputs. You don't need to generate a program to automate a test suite, you just need to use the end program a lot. This is especially good for games - just play the game a lot - until you realise that actually it IS possible for both players in the fighting game to hit 0 health on the same move, and you should add a "Round Draw" sound/routine/graphic.

Another stage of testing is theoretical. As you plough through the code, look for potential problems. "Oh, bugger, I don't test if someone asks for a -1 sized game!". Users are dumb - don't trust users, they are "rm -f *"'ing all the time, so they will find a way to break your program/their computer if it exists. Such analysis is hard to do if you set out specifically to do it but you'll catch a lot of bugs this way. This is best done when you spot a particular bug or as part of your general looking through the code for functions to tweak.

The best stage of testing is undoubtedly giving the program to other people and asking them to test. They won't have the same file structure, hardware, software, way of using the keyboard/mouse/joystick. They won't be aware of or use the little foibles and habits you've got into ("Oh, I only ever run the game from a script within Windows, I thought it would just work without it", "I don't touch the mouse when it's loading up", etc.). And the least experienced the person is, the better. You'll get hundreds of people test a game but only one or two would accidentally click outside the main menu, or tell you that the program says it can't find a common library.

And people who don't know how the program works will be clicking around in a non-obvious way so they will find all sorts of bugs as they try to figure out how it works, whereas an experienced person will just be dragging/dropping and completely ignoring the menus or buttons, for instance. My wife is the best bug-finder in the world. I guarantee that I can put a program thats worked successfully for months for a thousand people onto her Palm and within an hour she'll have crashed it somehow.

7) Have a nice development environment.

This doesn't mean a flashy GUI, it doesn't mean having a multitude of debuggers for multiple architectures. What is does mean is that there area as little distractions as possible when you're coding. You can compile with one command or click. You can place files into a clean test area with similar ease. You have all the tools you need most of the time. You don't need perfect Makefiles, or fantastically complicated scripts to allow it to compile on every architecture in the world, but you do need to have an easy way to do things for yourself.

For my projects, I often have Makefiles but use them sparingly or in a special way. And a lot of the time, if I'm working on a particular part of the code, I'll have a simple script that compiles, links and runs just the part that I need. I don't intend anybody else to ever use that script, I don't even distribute it, but it's a convenience when I'm programming.

If you're developing for another machine, you should have a way to simulate or access that machine quickly. For instance, I develop for the GP2X, so when programming for it, I usually have it close to hand. That doesn't mean I have to plug it in every session but it's easily accessible. I can plug it in and test stuff in it. As there are two main types of GP2X, I have a way to "emulate" the other type so that I can spot potential problems etc. I also have telnet and VNC access to the machine so that I can leave it running and test multiple programs in many ways in one session without having to keep re-connecting and running the machine.

For some people, this might also mean artificially limiting the memory/CPU available to their programs when they want to see how it would run on their target machine. Or running a software emulator. Or connecting the machine itself, transferring the program over and seeing how it works. Especially in the last case, you want to make it as simple as possible.

Debuggers are useful but you should only be using them relatively rarely. Simple problems are much easier to solve with code inspection. You'll usually know exactly where it crashed and a quick glance will give you the why. Tricky problems can be isolated with a couple of printf's or their equivalent. And difficult problems don't NEED the facilities of a debugger, they need your brain the most, but a debugger can then be a big help.

Have easy access to manuals and function references. Visual Basic used to come with a massive function reference manual that was incredibly useful. Sometimes just flicking through it would give you ideas about how to solve problems because you found functions you never knew existed that could do things in a different way for you. The TI-85 calculator was the same, and even the ZX Spectrum came with a BASIC manual. Even a "For Dummies" book can be an incredible reference for a seasoned programmer, especially if you switch between languages often ("Is it Select Case of switch() in this language?").

If you're programming with a particular library, say SDL, have access to a complete reference to it, with examples. This is immensely helpful, even if it's just a bookmark of a good site. I use one for SDL that lists the C prototype of functions and structures (which I copy/paste straight into a program and then edit to turn it into an actual function call), a brief description with lots of hints on what is and isn't possible (e.g. don't run SDL functions inside SDL threads etc.), and an example or two of using each function. It's amazing how much more helpful a simple example can be over a documented manual. Even a Google for a function name will turn up uses and tricks that you'd never have thought of, plus a variety of sites publishing "gotcha's" for using it.

8) Don't worry about the "proper way" of doing things.

You can't use GOTO. You should write performance-critical code in C. You shouldn't declare global variables. You should get structures down to their minimum required size. You should optimise all your functions. You should consolidate duplicated code.

Rubbish. Program for operation FIRST. You're not a NASA shuttle engineer, you're not dealing with 1000's of financial transactions. There are not going to be people sniffing over your code with disgust because you saved yourself having to restructure a hundred lines of code loops by using a jump instead.

If you intend to teach a programming course, or you're entering code into a critical programming project, then you should follow quite a lot of this advice. But if it's for a quick game that only you or a handful of people will ever want to see the code for, don't worry about it. A few of those items will help make you a more structured programmer, a few will reduce the risk of you leaving an accidental bug in your code, but the majority of them are pendantic ramblings of people who learned to program on punchcard and consider every cycle sacred. Don't get me wrong, I hate unoptimised programs with shoddy algorithms that bloat memory and slow the CPU, I can describe a bucketful of pathfinding, tree, sorting algorithms and their O(N) scales and there was a time when I would audit Z80 assembly and literally save hundreds or thousands of cycles in even the smallest listing, but for casual programming all that sort of stuff can come later.

9) Think.

This is the best bit of programming. Getting your head around why the 100% logical machine in front of you isn't doing what you ask. Invariably, it's because you've done something wrong but finding it is the best part. Programming requires you to think not only logically but creatively ("I haven't got a way to use floating-point numbers on this machine, how can I get around that to draw a circle?", "I can't use this function from inside a thread, so how do I get this thread to do what I want?").

The best part, although it's shocking, is that you do NOT need a computer to program. Argh! What is he suggesting? No, I don't meant that you should write a hundreds lines of C on paper perfectly first time, but that you can program in your head or on paper just as easily. You should be able to craft the basic structure of a program in your head before you start to write it, you'll be able to spot pitfalls, estimate which way is better or takes less RAM or less CPU.

You can program on a busy train. You can program in a quiet room. You can program in the garden. And you don't need a wireless laptop to do it. When I was learning to program, the best part was to get so struck with an idea that you dug out some paper to get your idea down before you'd even sat at a computer to try it.

Sometimes this even involved primitive flow charts but most of the time it meant just sketching out some pseudocode and working out some primitive structures.

I would use other people's programs of operating systems and work out how certain things functioned ("This must be a double-linked list", "They must be spinning in a loop waiting for me to press a key", "The AI in that is watching to see when you move more than X units into a small space before it considers it an attack", etc.). And a lot of the time I found myself stumped and sat down to work out how things worked. Or a way to improve them. Most of my ideas never made it onto an actual computer but it's the best part of programming - to stimulate the brain.

10) Enjoy it, and make it fun

Programming is like a puzzle. The aim of the puzzle is to finish whatever program you started (and sometimes just getting the answer hidden in the yellow squares isn't enough and you have to go and finish off the rest of the crossword). The intellectual challenge is to do that and to debug it when it doesn't work (and some people subject themselves to such intellectual challenges all the time, even when the answer is only a calculator tap away). The point of doing it all is to have fun (once you do the one-millionth wordsearch, or you've spend eight months on the same clue, the puzzles lose their appeal).

Why do you want to subject yourself to decrypting the Enigma code, when you could just as happily do a tabloid crossword? Do things that are within your capabilities, are fun, aren't over-stretching your bain, and that you enjoy doing. That's why people program. That's why there's billions of lines of code just floating out that the people are offering for free... they don't care that it was hard to do, it was fun for them to do it. Fun to prove it could be done, or that it could be done better, or that they could do it, or to get recognition from someone else for their hard work.

This is especially true if your programming games. If you can't enjoy the game at the end, there was little point in writing it you would think, but actually even the worst of games can be fun to program.

STPPC2x Beta 2- GP2X port of Simon Tatham's Portable Puzzle Collection

2008-04-09T17:03:00.003+01:00

I've just released Beta 2 of my port of Simon Tatham's Portable Puzzle Collection to SDL/GP2X. It is much improved and the only remaining "problem" is that mines doesn't work. The other 26 games run fine from start to end.

I think the problem with minesis due to endian/signedness differences between x86 and ARM within the puzzle code itself (which I don't change in order to port any of the games). The game compiles and works under x86 Linux and runs fine but the GP2X crashes with an assert on mines.

The beta 2 version of STPPC2x (and the associated source code) can be found here:

http://archive.gp2x.de/cgi-bin/cfiles.cgi?0,0,0,0,25,2543

Or on the website where I keep all the released versions:

http://www.ledow.org.uk/gp2x/

GP2X port of Simon Tatham's Portable Puzzle Collection

2008-03-29T15:59:00.009Z

Damn it's a long time since I programmed in C.

I've just spent the last few weeks porting the GTK/Windows/MacOS/Palm collection of games that are in Simon Tatham's Portable Puzzle Collection to work on the GP2X handheld game console.

That involved the creation of an SDL "frontend" for the existing puzzle infrastructure (which, I have to say was beautifully abstracted and documented from a programmer's point of view). It worked beautifully well on the whole. The puzzle framework is designed in such a way that it can be easily ported by just filling out the definitions of a few dozen functions. Basing my work off the existing GTK port, most of the simple functions were only a line or two to convert to SDL (using the SDL_gfx library helped a lot too). Even the font functions were quite simple, when using SDL_ttf. And because the GP2X is Linux-based, I was able to prototype all the functions using the GTK functions running simultaneously with their SDL equivalents on the same development PC, as you can see from the screenshot. Each game is running an SDL and GTK copy of the game and synching display and input between the two, while running on a normal Linux PC.

After they all functioned enough to give me a playable game in SDL, I started stripping out the GTK functions, libraries and dependencies and ended up with an executable that only needed SDL, SDL_ttf and SDL_gfx to run. That ran on any SDL-based architecture so I could compile the same code for either the GP2X or the Linux PC. For the GP2X it was the "simple" matter of cross-compiling them to the GP2X's ARM architecture, linking them against some ARM SDL libraries and then testing them on the machine itself.

But it has to be said that the hardest part of all was actually getting things to compile and link properly! I now officially hate Makefiles, gcc and linkers. I wasted more time trying to get code which I *knew* was working to compile and link in various ways without errors than I ever did coding! That really was the most horrible experience and half the time I had no clue as to what the problem was, even after analysing and googling the various weird and nonsensical errors I was getting. Programmers should really not have to play about with such convoluted and finnicky command lines just to get a simple C program with pre-compiled libraries to compile and link. Granted, wanting to compile a program for both x86 and ARM, with dynamic or static libraries should mean a little tweaking but it should never be as difficult as it was. I was actually *scared* to touch the Makefile each time in case I broke it and I kept more backups of that than I did the actual SDL interface code.

Anyway, I eventually got the puzzles to the point that they were mostly playable, mostly fun, mostly working and mostly pretty-looking. I've tested them all on GP2X and they are, on the whole, working (there is one that errors out in the actual game code, which I haven't touched, but only on the GP2X - a Linux machine with exactly the same code doesn't error. The other problematic one is just a bit too slow because it tries to do a lot in a short time on a slow processor). I was working my way through them one at a time to release them, because some of them exhibited subtle bugs, some of them required more work on input to allow them to "enter" numbers into the puzzles and at least one of them was (and still is) just being a pain in the bum. But now the vast majority of them work and I've built "beta 1" of what I have called (in a rare fit of originality) STPPC2x, or Simon Tatham's Portable Puzzle Collection for the GP2X. :-)

I'm surprised at how well and how fast they work, considering that the development 600MHz machine doesn't run them any faster than the GP2X at 200MHz and I'm not even using the hardware acceleration on the GP2X properly.

The first beta, with 25 out of 27 games playable, most of them near-perfectly, has been released on both the GP2X archive and at my website: http://www.ledow.org.uk/gp2x/ You can get the source code and toolchain that I used from my website too, because I find the most annoying thing about picking up other people's code is when you don't have the same compiler/libraries/filesystem setup as they do, so it's a problem to compile even before you start coding.

I consider it a good piece of work, considering that my last C excursion ended about an hour after starting it and I ended up writing the program I needed in something else instead. That was several years ago but I was able to pick this up and run with it. I estimate the total *actual* development time on STPPC2x to be approximately 48 hours for approximately 1000 lines of actual code (not including the Makefile), spread over a period of two and a bit months (it's hard to get in a mood where you want to fight with compilers, pointers and libraries). At least six of those hours was getting the Makefile to work properly. At least two were creating scripts to compile and link each file individually because I was sick of the Makefile not working properly and needed something that worked *here and now* so I could work on either the Makefile or the actual code at my leisure. At least ten hours on top of those 48 were things like "waiting for stuff to compile", "waiting for stuff to copy", "waiting for stuff to upload to my website/the GP2X archive", "preparing screenshots, instructions, proper licensing info, directory structure etc. for releases" and "waiting for the GP2X to finish writing to the SD card". A 600MHz development machine trying to copy megabytes of files over a USB 1.1 connection to an SD card isn't ideal, especially when you trying to write instructions, take screenshots, upload via FTP, etc. simultaneously. :-)

Anyway, it's been my first big programming project for a while. I'm always writing *something* whether it's a shell script or batch, a VB program to do a quick job etc. or something for my own entertainment but they all involve different standards of work - either I'm the only person to ever use the program at all, or it's for a limited technical audience, and it's very rare that the source code is ever looked at in either case.

In the first 24 hours, it saw 100 downloads from the GP2X archive alone and I haven't looked at my website's logs yet. At least I know that it was worth my while.

Darren Stone's WinXScreensaver mirror

2008-03-06T08:09:00.005Z

Ever since I posted my article about the substrate screensaver, I've been getting emails of thanks (even though it wasn't my program!). Now, it seems, the author of the substrate screensaver port to Windows has let his website lapse and it's no longer available from there.

Well, that's not a big problem for substrate fans because I mirrored that particular screensaver along with the article I wrote and someone else has also ported it to Windows, as mentioned in the comments in the article above. But it now looks like people want the full program of WinXScreensaver with all the other screensavers and some curious screen-saver management abilities. Having scoured my extensive software archives of "stuff that'll come in useful one day", I have managed to locate a copy of this file (I never delete anything, but that doesn't always mean I can find where I put it. The Linux slocate command is worth its weight in gold as far as I'm concerned).

Given that it's based on XScreensaver which uses the X11/MIT open source license, I assume that the license of the Windows port allows me to mirror it and post it. I also assume that the original porter, Darren Stone of http://tron.lir.dk, doesn't mind me posting a copy here. Darren, shout if I'm wrong.

So, seeing as mirroring old files is now becoming a bit of a hobby for this blog, there is now a copy of the file up at my website.

It's WinXScreensaver1.1-Install.msi that I have, I have no idea if that was the latest version. Archive.org's history of the site is sparse at best. It's definitely an unmodified working version, though, because it's the one that I used to trial the program for something fancy I was going to do on my network. Hopefully people will find this useful. Enjoy! And give credit for the port to both the XScreensaver authors and Darren Stone.

Useful Scripts, Part 2: Linux "file sorter" script

2008-02-29T11:45:00.004Z

The following Linux-based script (which you can download here) is also one I find myself using on occasion. I've made versions of it that run in just about every OS I've used at one time or another. It's a script that takes a large directory full of files, creates 37 subdirectories (A-Z, 0-9 and "other") and moves all the files in the folder it is run into their lettered folder according to their first character.

This is great when you have a large folder full of files that takes forever to "ls" or access over a network or a folder that is unmanageable for quickly finding particular files in a GUI. It's case-insensitive (I'm sorry, but DOS got this bit right, case-sensitivity can be a real hindrance at times, no matter how careful and experienced you are) so "Afile" and "anotherfile" both get put into the a/ folder.

I've used this on emulator rom folders when they got too many files in them, I've used it (or a Windows-"port" of this script) in user-profile folders on servers to help narrow the users down a bit and even move portions of them to a different storage medium in a logical manner (it's better to move A-L to another drive than it is to move the first 1000 users, from a "I'm only human" point of view). I've used it in archives of zip downloads, photographs, instruction manuals, all sorts. I hope someone else finds it useful.


#!/bin/sh
mkdir other
for LETTER in a b c d e f g h i j k l m n o p q r s t u v w x y z
do
        mkdir $LETTER
        FILESPEC=$LETTER\*
        UCASEFILESPEC=`echo $LETTER | tr a-z A-Z`*
        mv ./$FILESPEC ./$LETTER/
        mv ./$UCASEFILESPEC ./$LETTER/
done
for NUMBER in 0 1 2 3 4 5 6 7 8 9
do
        mkdir $NUMBER
        FILESPEC=$NUMBER\*
        mv ./$FILESPEC ./$NUMBER/
done
mv ./* ./other/

Useful Scripts, Part 1: Linux "screen" wrapper script

2008-02-29T11:02:00.005Z

A few people in the past have asked me for a few of the hundreds of scripts that I have made for myself. My life is a one long session of script/batchfile writing (I use the terms interchangably because a list of commands to run is a list of commands to run no matter what language, OS, etc. it is written for). There are school networks that are running on some of my scripts!

I'm not so keen on letting the larger, more tedious scripts out without me doing a proper licensing thing on them (open-source of course) but the little ones that are just convenient wrappers etc. I don't mind giving away. They are so trivial you can hardly claim copyright on them, even if the only intention of that copyright is to give them away but ensure you get recognition. Anyway, here's the first one, for Linux. It's a wrapper for the "screen" command. You can download the script from here.

Screen is a powerful command. It's ALMOST as good as when you first use a Unix-based machine and are taught to use Virtual Consoles (pressing Alt-F1, Alt-F2 etc. lets you have a variety of commandlines open at the same time even if you aren't in a GUI). Screen lets you run tasks in the background, with their output hidden, in such a way that you can "resume" or connect back to the screen that the program is displaying, from anywhere. It's a kind of VNC for the console. So, for example, if you want to run a program like IPTraf or HTop, but want to be able to see its "screen" from an SSH session later when you're at work, you can do this.

The script itself is quite simple, and I have a habit of calling it my "do offline" script, so I normally name it do_offline.sh:


#!/bin/sh
if [ -z $1 ]
then
        echo "First arg is screen name, rest is command to run"
fi
if [ ! -z $1 ]
then
        SCREEN_NAME=$1
        shift
        echo "Running commands ($*) under screen name $SCREEN_NAME"
        screen -A -m -d -S $SCREEN_NAME nice $*
fi

Basically, you run it as:

do_offline.sh "screenname" "program and arguments..."

e.g.

do_offline.sh htop htop

do_offline.sh lare_file_copy cp -R *.* /mnt/backup/

do_offline.sh kernel_compile make

And it will happily run those commands in the background while putting the output into an invisible "screen". This is useful if you are logged in via SSH... rather than having to fudge access to the F1-F12 consoles you can start jobs in their own screens and then disconnect and reconnect from their screens as necessary without interrupting the job. You can start an 50Gb download remotely and not have to interrupt it when you disconnect your SSH session (the program will just carry on running in its screen and you can "resume" the screen from the local console when you get home to see if it finished).

To resume the screens, you just use:

 screen -r

e.g.

screen -r kernel_compile

And the one annoyance... to "disconnect" from a screen without interrupting the program its running, just press Ctrl-A and then D. Horrible keyboard shortcut.

For example,


# do_offline.sh htop htop
# do_offline.sh kernel_download wget 
http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.24.3.tar.bz2
# screen -r kernel

Is it done yet? Nope?


Ctrl-A  D
# cd /

Go off and do something else, disconnect from the computer, go to work and log in to this computer remotely.


# screen -r kernel_download
There is no screen to be resumed matching kernel_download.

So it's finally finished and we can compile it. But I want the computer to have compiled it by the time I get home


# do_offline.sh kernel_compile make

Go home, log back into the machine at the local console


# screen -r kernel_compile

It shows up as still compiling, so I just watch the progress for a few minutes and then..


Ctrl-A  D
# screen -r htop

Have a look at the CPU usage, wait for it to dip


Ctrl-A D
# screen -r kernel_compile
There is no screen to be resumed matching kernel_compile.

My kernel is downloaded and compiled without me having to watch its progress, limit it's output to a single virtual console on a single physical machine, or keep the remote session connected constantly while it does it.

I hope it's useful to someone. It's already been useful to me and to at least a couple of people on the Linux Questions forum.

GP2X handheld Linux games console - A review

2008-01-21T10:57:00.001Z

For Christmas this year, I managed to persuade my other half that a GP2X would be an ideal present to keep me quiet. For those who don't know, the GP2X is a handheld games console whose main selling point is that it runs Linux behind-the-scenes and has buckets of "homebrew" software, including ports of popular Open-Source games and emulators. As I've stated in a previous article, to me that makes it more valuable than any other games console I've ever owned - I can load it up with "fun" older games and play for thousands of hours rather than spend a fortune on a single modern game which I would play for about a day before I get bored or completed it. (Incidentally, thank you Nintendo for the Wii and bringing the fun back into gaming!)

Anyway, I am now the very proud owner of a "Mark 1" GP2X, called the F-100. This is the black version with the original "joystick" rather than the touchscreen and digital joypad. To be honest, I didn't specify which version and knowing what I know now, I'm glad that all the wife could afford was a second-hand F-100. It's personal choice but I can sacrifice the features of the sucessor F-200 for the features that the earlier model has. Other people would disagree depending on their usage.

Most of the people who are interested in the GP2X, or indeed it's predecessor the GP32, will know about the handheld's features but for a quick rundown...

It runs on two "off-the-shelf" ARM chips (940T and 920T), with variable (and software-controlled) clocking between 60MHz and 260MHz each, the default being 200MHz and the overclocking being quite "safe" overclocking that doesn't cause permanent damage unless you do it for very long periods of time and overheat something. In fact, more powerful games can only run at the higher speeds and will "overclock" the device themselves - all that happens is that it works or it crashes. All that is required to "recover" from a crash is a switch-on, switch-off to get it back to normal.

It comes with 64Mb RAM (32Mb is directly accesible, the rest can be used with some trickery and often is) and a 64Mb internal NAND permanent storage. This contains the bootloader, kernel, some built-in applications like the menu and maybe even some games depending on your particular purchase. This can all be replaced and customised but you won't gain much. The NAND allows people who forgot to get an SD card to use the console straight away, and also provides an avenue for the vendors to pre-load certain games if you buy their bundles. Also, because it has to be a deliberate act to affect the NAND bootloaders or kernel, you aren't going to be bricking your GP2X accidentally. NAND firmware updates tend to come in the form of a bootable SD card.

For main storage, you can use an SD card up to 4Gb (32Gb in the later F-200 model) formatted in either ext2 or FAT32 - most cards come with FAT anyway and if you want to use the connectivity features, you're better off with the more-prevelant FAT. This is where most of your games etc. will go and it's quite easy to have hundreds or thousands of games on a single large SD card. And, let's be honest, SD cards are so tiny that you could easily carry a handful with you and fulfill every gaming need.

The firmware runs uBoot to boot pure Linux 2.4 as the core OS (source and alternative firmwares are available but you don't gain much because GamePark Holdings, the manufacturer, did a good job in the first place) in around ten seconds, with a nice splashscreen and bingely-bingely-beep startup sound. All of the "menus", games, emulators are just ordinary Linux programs compiled on GCC against an ARM target. So everything is either open-source from the start or has many open-source equivalents, even the main menu, boot-up screens, built-in applications etc.

Because it relies on BusyBox internally, it's not even unusual to see wrapper-bash-scripts around games in the Games menu. It has a rather perfect and simplistic method of program execution - when the GP2X starts up, it boots and then runs the menu program (standard Linux binary). That lets you select a game/application to run. On termination, each individual application is responsible for making sure it exec()'s the main menu before it clears up. It's beautifully simple but works and prevents the menu hogging RAM while you're playing a game. And if a program ever crashes really hard, you just switch-off, switch-on and it boots the menu back up again. Programs have full access to NAND and SD storage for savegames etc. but they tend to only place things in their own folders. This does, however, allow you to have a collective "roms" folder and use several different emulators with the same roms.

It operates off two AA batteries, although they HAVE to be high-power rechargeables (preferably 2800mAh) - what do you expect for a dual-200MHz portable machine?! You can get a good couple of hours out of a set of two depending on what you're doing. There is also a mains-adaptor port for static use and you can easily carry enough AA batteries to last you all day if need be. The only minor point here is that the mains adaptor doesn't charge the batteries, but you can't have everything.

The screen is full-colour, 320x240 and is very good in virtually any lighting. It is covered by a plastic protective screen about 2mm above the surface, which protects the expensive bits against that pen you keep in your pocket. There are two speaker grilles on the front (I believe only one is an actual speaker(?) but stereo sound is present in the headphones) and the SD slot sits in the very middle at the top.

It also has a headphone socket, power socket (3.3v regulated), mini-USB socket and EXT socket (we'll get to those last two in a minute). The two AA's sit comfortably in a rear "bump". The game controls are (on the F-100) a "mini-joystick" on the left of the screen which works surprisingly well, A, B, X, Y, Start, Select buttons in their usual places, Vol+ and Vol- just underneath the joystick and L and R shoulder buttons. On the F-100, the joystick also "clicks" down to provide another button. Sadly this was removed from the F-200 model because the joystick was replaced with a 4-way D-pad and a touchscreen was introduced over the LCD.

The GP2X is comfortable to hold for long periods and the design is "flat" on the front, except for the joystick which can be controlled by a wiggly thumb or between two fingers for precision control. The batteries are tucked away from your fingers so it feels quite thin. You can't accidentally eject the SD card or knock the battery cover off while playing and all the other ports have rubber covers to stop you poking things in them accidentally. Headphones plug into the top, keeping the lead out of your way.

The mini-USB socket allows you to connect the supplied USB cable to access the GP2x from a PC - of any kind. No driver software is required and it appears as a standard mass storage device so Linux, Windows and Mac can all "manage" the devices files. Installing a game can literally be a drag-and-drop. You select what content you would like the PC to access each time - either the SD card or (F-100 only) the internal NAND - and it just pops up as a removeable disk.

You can copy your games to your GP2x without switching off by using this feature or you can just eject the SD card and use an SD card reader in your PC (not supplied). There are also a plethora of USB options on the earlier model - the F-100 runs what is known as a USB gadget interface so that it can appear as a "device" to normal PC's. This allows it to be seen as a USB network card, USB HID device (so you can control windows games with its joypad, for example), and it has built-in web server, telnet server and samba server for access over the USB-net. Sadly, these features are lacking from the later model F-200, which I see as a great loss, and were instead replaced with a touchscreen interface in addition to the normal control methods.

Because it's all just Linux, you get some fanatics do things like plug a wireless or Bluetooth USB adaptor into the socket and port a driver for the device. Strangely, they often work, although the practicalities of a handheld limit its usefulness. There are ports of games designed especially for accessing a Nintendo Wiimote over a USB-Bluetooth device, for example.

The EXT socket allows for a whole new range of options. First, TV-out. Yes, this little device can display on your TV! Some games can appear blocky in this mode but some make use of higher resolutions when they detect the TV-out cable. Either way it makes for much better "static" multi-player fun.

Additionally, the F-100 has a peripheral available called the Cradle - essentially a "break-out box" which connects to the EXT port to give you access to 4 USB ports, for connecting devices such as joypads, keyboards, mice, USB keys etc. Games have to support extra controllers but most popular ones do. The GP2x also directly recognises USB mass storage devices connected to it. The breakout box also features TV-out itself too, plus JTAG programming ports (for hard-core tinkering and "un-bricking"), additional audio-out, a power-supply connector and RS232. It's safe to say that the break-out box isn't really that portable because it is intended as a home-device for development, or for using the TV-out feature to turn it into a home console.

But the important thing is, how well does it play games? Well, the absolute best examples for "showing off" don't run to much if you're looking for 3D-power in your handheld, but considering the devices specifications they are very impressive. Payback is a GTA 1/2 clone (some might say a bit TOO close to the original) with 3D, dynamic lighting etc. and plays really well. It has to be said that this is the showpiece of the GP2x and little beats it in terms of hardware use, speed and visuals. On the homebrew side, a complete port (yes, port, not remake) of Quake is the best, in visual terms, that you will see - and it's compatible with virtually every Quake mod, including the official ones. However the little beast should not be underestimated - Quake running at full-speed on a device such as this is no mean feat when there is no dedicated 3D hardware.

The GP2X, it has to be understood, is not going to out-perform much at 3D. It's based firmly on 2D, from design to manufacture to software, and that's where it excels. There are ports of almost every 2D GPL Linux game available - SuperTux, Crimson Fields, LBreakout, Liquid War, GNU Chess, Quake, Hexen, Clonk Planet, etc. there are dozens. But that's NOT what the GP2X is for - I'm sorry but it's not! Neither is it to be used for it's built-in MP3 player, eBook reader or Video player (DivX compatible). Nope. This thing is an emulation machine, pure and simple. The "official" archive is full of games but emulators top the download charts every month.

ZX Spectrum, Amiga, Atari, Commodore 64, Gameboy, NES, SNES, Master System/Game Gear, Genesis/Megadrive, Arcade games, they all have emulators for them that run on the GP2X. Only the most demanding tax the little workhorse but for myself, that was more than enough. I can play all my old favourites, full speed, on a little portable device that I can put in my inside pocket comfortably. This is also the ultimate test of gameplay on the joystick - pulling off Ryu's special moves on a SNES emulator running Street Fighter 2 is flawless (I've heard the F-200 has more trouble because of its D-Pad?). The button layout is very well thought-out and lets you emulate SNES controllers virtually perfectly, and every other comfortably. You never feel that there's a button mapped into an impossible place.

The speed, graphics and sound for the above-mentioned emulators are perfect for the vast majority of games in default settings - it's always those ones with the special chips that give you performance problems. Gameboy games feel perfect, SNES games work perfectly if you have the "basic" chips, so Super Mario World and Mario All-Stars are flawless but things like Street Fighter Alpha, Starfox and Yoshi's Island will suffer. There is a port of MAME available with over a thousand games playable. Most 80's arcade games are fully playable and later ones are hit-and-miss depending on the specifications. I love Final Fight, Wonder Boy, Pang, Ikari Warriors etc. and was very glad to see that they all ran perfectly.

You can stretch the machine to higher-level games (there's even a PSX emulator for the very optimistic) using the built-in overclocking options in most emulators but you rarely go from "Aw, it's unplayable" to "Yay, it's perfect" by doing so. Also, as with all overclocking (of which I am a massive opponent when it is used on PC's), it varies considerably based on the particular manufacturing that went into your particular device. Some people can overclock their GP2X to 270MHz and beyond without problems, others can't get much past the 200MHz defaults. Oh, and it can kill your batteries much more quickly, so in fact what you find yourself doing is finding "sweet-spot" under-clocking limits for every game so that you can save battery power without sacrificing gameplay. Most emulators allow you to do this on a per-game basis, which helps you save as much as possible.

Emulators are definitely leading the software development on the GP2X - RAM timing and MMU hacks to vastly improve performance originated from a want to get every ounce of power out of the GP2X and are present in every emulator and in most homebrew games. MAME lets you access 4 USB joypads connected to the handheld for multiplayer action - a rare feature in other GP2X games, but has been copied into most emulators for the platform. There are a smattering of commercial games, none really priced higher than about £10, and their quality does show through but there are not many able to compete with "free" games coming out of the porting community. I thought that Payback was well-worth the money but I'm not sure I'd fork out for some of the puzzle games. I'd much rather give a homebrew-author the money for a particular favourite.

And with a 2Gb SD card, you can fit almost everything you'd want (including a couple of hundred MP3's and a video or two) onto a single card.

On top of emulation, the homebrew would be the next software "feature". If you can compile against ARM targets (easy with GCC and the various devkits available for the GP2X), base the game off Allegro or SDL or be prepared to write a little hardware-code, you can get games up and running in minutes. The hardware was designed to be accessible - it's all just Linux. You can get the joypad showing up in /dev/joy, you get sound out of /dev/dsp, you can do some memory mapping tricks on /dev/mem and /dev/fb to create double-buffered video in the slightly-trickier top 32Mb of RAM. Everything is just Linux 2.4 with some extra features here and there to let you tweak the LCD backlight, control the battery-low light, speed up either CPU etc.

There are versions of BASIC available which target the GP2x and are designed to create the sort of mini-games that can be more fun than commercial games - sites running nothing but Flash games are proof of this on the Internet, and on the GP2X you can knock up similar games in minutes using one of dozens of development packages - Fenix, Python, BASIC, all sorts of languages are available. There are hundreds of games available, some diabolical, some fantastic. There are even ports of SCUMMVM, Albion, Descent, Doom, Duke Nukem 3D, Ultima 7, Heretic, Hexen, Rise of the Triad, various DOSBox-based games, Frozen Bubble, Super Mario War, OpenTyrian and even the Graphical version of Nethack! (I'm sorry, you can't leave that game out of the list!). Every year there is a competition run for the best GP2X homebrew or ported game and the winners can be very impressive.

At the moment, when I'm not playing the "oldies" on an emulator, I'm playing Ghostpix (a very polished Picross/Nonogram/whatever you want to call it puzzle-game), SuperTux, Liquid Wars, Kuoles, FreeDroid, Frontier2x (an Elite-2-port), Quake, and a million other "five-minutes" games that are just fantastic for a handheld console.

And the most important thing - the GP2X makes gaming fun again. You plug it into your PC, download some stuff from the web or the official archive, throw it onto the SD card over the USB cable, then go to Games, GameName, GameExecutable and play. You can even use the eBook reader to read the instructions on the device itself (emulators tend to have a lot of instructions because, for instance, SNES emulation demands quite a lot of buttons to be mapped so you need to know how to get back out to the menu - usually this is some combination like Vol+ and Vol- simultaneously or R+L+Start).

A lot of thought obviously went into its design. It's sleek, small, comfy, durable and practical. It has plenty of connectivity and hacking potential (even the F-200). It works well and is sturdy. Controls are obvious (Vol+ and Vol- work in EVERYTHING, just about, even though they are software-controlled) and well thought-out. The built in applications are more than good enough and substitutes are easy to come by - there's even one that makes it look and work like the PSP interface.

It could benefit from an add-on battery pack like the PSP has, especially if you can get several hours out of such a thing, even on the highest demands. And it really needs an in-built charging circuit. But apart from that it's very, very good and it should really get more attention from hardware designers.

For the next model, a "hybrid" of the first and second models would sell much better - sacrificing old functionality for new functionality isn't a good choice to make. If you could upgrade its 3D capabilities without destroying backwards compatibility (the homebrew/porting scene is far too important to just discard and try to build another), that could only be a good thing. I don't see it being impossible, even if it's only in the form of a 3D accelerator chip and a custom OpenGL library to manage it. But on many fronts it's already perfect.

The capabilities are there for networked games (at least in the F-100), but it doesn't appear that many people have used them. Maybe a small wireless or Bluetooth chip could solve that problem in a backward-compatible way (especially with the Wiimote being Bluetooth-based, it looks set to be a standard for wireless games controllers) - it's not like the drivers for such things would be impossible to port. You could easily upgrade to a 2.6 kernel (some people already have!) for the next version and open up a whole new world of new drivers you could take advantage of. You would have to tweak it, though, to ensure power-use stayed as low as possible - embedded kernels aren't exactly rare, though.

But the best thing about the GP2X is the reputation that goes with it. Nobody knows what it is, so you get some strange looks when you produce it from a pocket on the train. Some even stranger ones when someone recognises the Mario "ting" from an obviously non-Nintendo device. It is absolutely fantastic for wiling away long journeys, it has to be said, purely because of its design for a short attention span... listen to some music, read an ebook, play a SNES game, listen some more, play some Megadrive, listen some more, carry on that campaign in Crimson Fields, etc.

All in all, the GP2X has managed to do what a lot of the larger handheld console developers haven't. It turns a profit, based purely on hardware. Software is especially prevelant even though there are few "launch" titles. It's great fun and well designed. And it has a community effect that's unmatched.

Here's to a GP2X sequel that's even better!

Why can't my computer...

2007-10-28T01:51:00.000Z

There are a lot of things that annoy me about computers, or more usually, Windows in particular.

Why can't my computer:

1) Accurately gauge how long it will take to do something? (Prime Culprit: Windows)

File copies, program installations, downloads, no matter what it is, if it's got a progress bar it's not going to be accurate. Not only will it not be accurate, it won't even be close most of the time. Sometimes, granted, it'll be spot-on but that's just got statistical averaging written all over it. Why, in the middle of a file copy, does it suddenly decide that it's going to take 3 days, no, 10 seconds, no, wait, 18 hours, no, hold on, ... ?

Yes, not all things are predictable, but the computer could at least estimate (within a reasonable margin of error) and not blindly pick up random and wildly varying estimates without even flinching. It'd be much better if, when it gets confused or held up, it just gave up until it knew again! (KDE does do this, for instance, when copying files... it'll just say "Stalled" for a second).

2) Know where the drivers for a bit of hardware will be? (Prime Culprit: Windows)

Why do I have to point it to a driver that I've either had to insert myself or, a lot of the time, had to go to www.randommanufacturerswebsite.com/techsupport/drivers/windows/xp/driver/thingamajig/v8/revision2/setup.exe and downloaded myself before it will recognise the hardware I bought?

Why can't there be a standard for hardware so that everything USB or PCI-based contains some information that tells the computer roughly WHERE a set of drivers will be. Or, failing that, some website where it can automatically look up what drivers exist and where when given a PCI/USB id, even if it gets "Not Supported" or "Unknown" some of the time?

3) Know what I mean? (Prime Culprit: All OS)

When I type www.fredc.om into my browser, why can't it just correct it for me (maybe with a Google-style "did you mean?")? Some mistakes are hard to catch but it doesn't even catch the little ones. wwww.fred.com or www,fred,com , for instance. And when I have a file copy command, rather than just error at me because I mis-spelled or, in Linux, mis-capitalised a filename, have a small go at working out what I meant and ask me if it's right. (Yes, there are certain plugins etc. but I want it to be a standard feature. I can't be the only person who's done these silly typos!)

4) Protect me from others and myself?(Prime Culprit: Windows and Linux)

Now, I'd like to point out that this is almost exclusively a Windows problem for one part of the question (protecting me from others) and almost exclusively a Linux problem for the other (protecting me from myself). Windows just doesn't go far enough to stop other people getting into/onto/through my computer's defences. Not even after the 50th version which has promised to do so. But it has some fantastic features to protect me from myself, if they are enabled. File deletes are, on the whole, confirmed first and undoable later. Plus, it's quite hard to completely shoot yourself in the foot by, say, deleting your Windows directory accidentally. On the other hard, it's extremely easy if you are not careful to totally balls up your Windows installation just by clicking on the wrong website, the wrong email etc. or even disabling the wrong service.

Linux, though, protects you from outside elements a lot more. And even if they do get through, it is quite easy to recover from them and, additionally, their impact will be limited to the user accounts that are affected. However, even as a normal user, you can wipe out your home directory in one command without any confirmations and with little chance of getting it back unless you have specifically put into place procedures to recover it (such as replacing commands with safer versions, configuring user accounts so that they can't do that sort of thing, or just having easy-to-restore backups in place).

So it seems that Linux could benefit from a bit of Shadow Copy, a bit of System Restore or some kind of filesystem rollback and Windows could benefit from a bit more privilege seperation, a bit better programming and a focus on non-virus software rather than anti-virus software (i.e. before-the-event practices that stop the viruses getting on there so easily in the first place).

And this doesn't just apply to desktop environments. Humans make mistakes. Operating systems should be designed to take account of this fact and help where possible.

5) At least give me a clue? (Prime Culprit: All OS)

"mplayer: error while loading shared libraries: liba52.so.0: cannot open shared object file: No such file or directory"
(Note that this is an example only - when you compile mplayer from source, it does in fact warn you or leave out support when pre-requisites are missing).

Well. Lovely. Fantastic. So you know that you NEED liba52. You won't run without it. You were obviously written with it in mind. So why can't I instead get:

"Mplayer: Error: You haven't installed liba52. You can download this from http://liba52.sourceforge.net/"

Now, with modern dependency checking this sort of thing is getting rarer but even so, where's the error message that a human can parse easily? Windows does just the same with missing DLL's. And compile-time messages but not run-time messages are another bit of a bind. Fine, tell me gently at compile-time that I need libX and exit neatly. But why not do the same when I move that binary to another machine that doesn't HAVE libX, instead of erroring as above? More people RUN programs than COMPILE them. People compiling programs usually have the sense to sort such problems out for themselves (and such a trivial error is nothing compared to some of the doozies that you can get when compiling software for yourself!), ordinary users can't.

Similarly, dumbing down error messages too much is a major mistake:

"An error has occurred."

What error? Why? Whose error was it? Was it my fault? Was something wrong with the machine? Where did the error occur - in the program, in Windows, in something else? What can be done about it?

As a replacement, how about:

"An error has occurred in PartyPoker. [[Note the friendly program name]]This appears to be a problem with that program. You can try running it again, or checking for a PartyPoker update. If you still recieve errors with PartyPoker, the program gives the website address www.partypoker.com/problem as a source of help. Click below for a file which will help the program author to determine the cause of the problem."

Or:

"An out-of-disk-space error has occurred. Windows is showing you this error because it did not have enough room to create a 50Gb file on drive C: as requested by the program Nero Burning ROM. You have only 10Gb free space on drive C:. You can try:

- Clearing up 40Gb of space on drive C: and retrying the operation
- Instructing Nero Burning ROM to use a drive with more space (for example, D: currently has 100Gb free)"

6) Fix itself. (Prime Culprit: All OS)

Windows.
Windows Last Known Good Configuration.
Windows Safe Mode.
Windows Recovery Console.

Where's the "I need to get to my files" option - with a minimal desktop that uses NO programs, services or other information in common with the main Windows install and lets you copy your files off the computer before it dies completely? Where's the "Run Diagnostics" option to let Windows have a go at trying to find out what it actually wrong rather than blindly looping through a list of Windows "versions", each of which gets less and less drivers loaded? While we're at it, where's the "Check My Disks/Memory/Hardware" option in that list?

Where's the "Right, last time I crashed loading the graphics driver, according to these logs - this time I'll ignore graphics and just load a basic VESA driver and see if I can get further" logic?

And then we have the fantastic idea to include an option, which is usually the default, to automatically restart Windows on error (great, so you can't even SEE the BSOD when it whizzes past, and then Windows will blindly sit there trying to get into Windows every time it reboots until you come and fix it - it'd be better just to turn itself off!). Yeah, there sometimes a need for a watchdog on a high-availability server but on an OS designed for Home Desktop use? And what's the point of it just infinitely restarting at the same place unless it LEARNS from that mistake, especiallly if that place is before it evens gets to a desktop?

That's just the start of my list. Hopefully, I'll finish it off soon.

What does the Linux desktop need? Those who say "I want, I want..."

2007-10-19T10:24:00.000+01:00

I've just read an article linked on LWN.net entitled "What does the Linux desktop really need?".

Let's veer slightly and ask a similar question: "What does my car need?". I'd say it NEEDED a lot less rust, a sunroof that isn't held in by parcel-tape, a new wheel-bearing (AGAIN) and something to make my lights turn on at least once in every ten tries.

Personally, I'd say it could also do with electric windows, heated front windscreen and a CD changer/MP3 stereo. But wait. Hold on. If we're saying that I can have anything I've seen on a car... I "need" braking-power reclamation, a hybrid engine, 0-60 in 5 seconds, a five-year warranty, finger-print recognition for starting the engine and GPS vehicle tracking.

Now, that's ridiculous, because my car doesn't NEED those last things, but that's basically what this article was saying. The Linux Desktop doesn't NEED anything else. It's there. It's a viable alternative to Windows. It can do anything that Windows can do (given the developer time investment). Years of development on the Windows side is now recreated in a matter of months on the Linux side - take drivers for things like newly-released wireless cards, some of which have to be reverse-engineered before a driver can be made, take some of the fancy graphical effects present in Vista, some of the desktop "features" of MacOS and Vista and there are already equivalents and copies available for Linux that can do just the same, most of which were started AFTER someone had seen those features elsewhere.

There isn't a type of application that can't be run natively, in theory. Given enough horsepower, we can even replicate the majority of Windows functions enough that high-power applications and 3D games can be run in a Windows-userspace recreation (Wine) at astonishing speed considering the technical problems of doing such things. Not only that, Linux can do virtually everything that Windows can do natively, and usually does a better job at it. There's nowhere to go from here apart from getting people to a) use the thing and b) develop for the thing, both of which are mutually dependent.

Reading LWN comments on the article are even worse... it "needs" Photoshop, Office, Games... No, it doesn't.

It's been proven - it's technically possible to write top-class 3D games and powerful image-editing programs for the Linux desktop. It's not even any "harder" than doing so for Windows. When Adobe want to do it, they can. In fact, Linux is more standardised for such things. You don't need to worry about ATI vs nVIDIA vs Intel - just let OpenGL sort things out for you.

The fact is that the desktop doesn't NEED anything, unless you are intent on recreating Windows on Linux. That's the problem - the Windows mentality isn't suitable, or compatible, with the way Linux works. Windows people want firewalls that don't disrupt their games and let any application request an open port via uPNP. Windows people want antivirus because they think they need it. Windows people want perfect connection to the heap of junk that is Active Directory. Windows people don't want to enter passwords or manually configure their hardware in order to do dangerous things, like overclock their graphics card or turn their firewall off. You can't change those people. Not without a big stick.

The way to get Linux onto a desktop is not to perfectly emulate every undocumented Windows bug and quirk when connecting to an Active Directory server for login so that some poor sap can run Outlook the way he likes, but to build a Linux equivalent that has clear advantages - faster, smaller, easier to manage, more transparent, easily portable, easily extendible and which can do stuff not seen elsewhere. The people who are more likely to make decisions based on those criteria? Large organisations. Who use networks. Which are run by a poor sysadmin somewhere who "knows" Windows but only "plays" with Linux. They don't care that Linux can detect and use 99% of all PC hardware - they care that it takes an hour to set up a new type of PC to the way they want it to be, rather than a five-minute copy of a well-known model's hard drive.

Imagine a Linux distribution. You install it in a "server" mode via a menu. Then you install it on a client machine via the same menu. At no time did you have to install drivers for monitors or some such rubbish. You don't HAVE to license it. You don't HAVE to spend days setting up the user group structure and policies to a safe default. Yeah, they'll be parts of the machine that won't work without proper drivers but that's not important. Really. These sorts of places SPECIFY the machines. They say what hardware it will or will not come with, down to the individual components. Compatibility with some cheap winmodem is not their problem - they buy a different modem, especially if it affects their security or technician's free time.

Anyway, you've started a client and server from barebones. Then imagine that you have automatic, cross-network authentication to that server, client logon, desktop settings and "policies", which allow the network administrator to change every single setting and restriction on the clients in almost every program via one directory interface. Imagine it works just as well over wireless, VPN, a Bluetooth interface or a USB cable. Just as automatic. Just as simple. Just as fast.

You can throw software across the network by just clicking on a machine in a tree-diagram on the server and deploying a package (so it'll be an RPM, not an MSI, but who cares?). Managing a thousand users on a hundred workstations becomes a cinch. And as a bonus, the machines automatically share work between them when they are idle. They automatically discover each other (with according administrator control) and use each other's storage as a mass-RAID for the network data, including encryption to stop people looking at other people's work. It does it all without needing a millions ports open. It does it all without major security problems. It works just as well from outside the network, when one of your staff takes a clieent laptop home - they plug it into their broadband, maybe they have to click an option to connect remotely instead of locally, and bam! - it's just like they are at the office.

Now imagine that you can do all that on lower-end machines than Windows could. And you can do more, stuff that just isn't possible on Windows. You can plug four graphics cards into each PC, four USB mice and four USB keyboards and now four people can use the one machine without even knowing. And their CPU power is being shared across the network, with all the other four-man machines, maybe even with the server itself doing some CPU work on their behalf when it's not busy with other things. And you wouldn't even notice that was what was going on. We're *not* talking thin-client - but you can do that if you want, too. You just tick the "thin-client" option when you install the client and the system does the rest for you.

Now imagine that not only does it do all that but you can also trust the server to backup those clients too, whether they are working locally or remotely. The server remembers individual machines and any quirks you've had to apply (that binary modem driver, that setting on boot that prevents the ACPI problems etc.) and when you rebuild them you can re-include those quirks too. Saving data to the network is transparent and not only does the server RAID all it's data, but it shares it out with the network. Server blows up? No probs. Stick the CD into any machine, create a server, maybe supply it with the old servers private key and bam - all the data feeds back from the clients to the server and the network rebuilds itself.

Well... the problem is that most of that stuff exists in one form or another. Certainly everything listed above is perfectly "do-able" and there's at least a few bits of software for every single component of that sort of system. They're not all tied into one distribution (that I know) but they are there. The most "together" distributions are the paid-for ones, Red-Hat etc. But there is nothing there that isn't possible, it might take a few months work and you could probably do it all without nothing more than existing downloads and kernel patches and a bit of bash-glue. But it's not around. You can't actually get it. And most Windows admin's won't even try it while it involves a lot of messing about. Have you seen some of the HOWTO's? Have you seen the number of steps needed to get Kerberos, LDAP, Exotic Filesystems, remote-control, VPN's, etc. all working your way? Windows is no easier, either, so you're left in the "what's in it for me" valley.

What's needed is not more and more replication of existing features but new and exotic uses. What's the most interesting part of Google? The Google Labs. What's the thing that people ALWAYS buy an OS for? The new, interesting features. Yes, when Samba can perfectly manage every aspect of AD integration, it'll be sought-after. But people scrambled to Vista "because". There wasn't anything complicated in it, there was little groundbreaking stuff and popular opinion now says that Vista is more of a pain in the backside to run for the average user than previous versions. But it was bought because it "could". It could do "new stuff" that Windows people hadn't seen before. Remember Windows 98SE that could "do USB".

People are already talking about the next version of Windows Server because of what it can do. Not about how well it does it. Not even about how easy it is to do, that's normally left until review copies appear in the hands of magazine reporters, but about what's new. And, stupidly, not even about what it doesn't do any more. The fact that every single version of Windows Server has had a hundred features announced that have never appeared is overlooked. The hype surrounding it by the time it comes out MAKES people want it. Vista was supposed to include database-style filesystems, a download manager, filesystem "filters" (Libraries), Palladium "trusted systems", integrated anti-spyware, UEFI support, PC-to-PC Sync, a full XPS implementation, to have a better UI, to perform better than Windows XP, and that's before you even get into all the capabilities that they physically removed from the OS that were there before in XP.

And the fact is that Windows Vista was just a small upgrade. If it had had ALL of those things, it could possibly be the best OS in the world. And Linux CAN have the majority, if not all, of those things. Most of them even exist for Linux right now. We just aren't using them.

People who "push" developers to make a Linux-Windows just don't get it - Linux is already in front in terms of features and technical details. We all know that. It wipes the floor with it's "main" competitor (although, to be fair, so do a lot of other operating systems). It's not that we're not "there". We are. Something else is holding Linux back. Firstly, ease of use. That's usually a big trade-off with not only compatibility and security but also with system performance. However, Linux has power to spare. And then it's just a matter of making things work without a million settings. I'm a big fan of command-lines and text-based configuration files - there is no reason to lose them. But they don't have to have vi as their only interface.

The main thing it's missing, however, is a short, simple, easy demonstration of powers that Vista and even future versions of Windows either can't or don't have. It's needs a show-distro to turn up, either from the depths of one of the established ones, or out-of-the-blue. It doesn't need that distro to say "Look at me, I'm just like Windows, only slightly better", it needs to say "Why on Earth would you bother to look at an OS that can't do X, Y and Z", where X, Y and Z are things that either have never been done before, or always been "promised" or "desired" and never materialised. And I don't mean a flashy-gui interface. The nearest we ever got to that sort of hype was the Kororaa Xgl Live CD and look at what it did - very little of any practical use. But it was NEW. It was even NEWER than what Windows could do at the time. So it got a lot of press.

Being able to access an AD domain isn't something new. It's not impressive to people. It's not even that innovative - there's a major OS that does it automatically and (fairly) reliably. What's needed is to play to Linux's strengths - flexibility, malleability, speed of development, freeform and accessible API's. That means coding quickly, easily, without barriers and restrictions and expensive SDK's. Just get in there and write stuff. In half the time it's taken Windows to get where it is, Linux has replicated and/or surpassed every aspect of Windows. Now it needs to overtake it - you can't do that by blindly copying features from Windows, or even other OS's.

Now, the article doesn't push Linux for anywhere near as much as the comments on LWN. To them I say: Just because Windows does something, doesn't mean that Linux should follow suit. It that were the case, Linux would BE Windows. I don't WANT my Linux desktop to have a built-in GUI firewall that's difficult to configure the way I want. I don't WANT automatic update dialogs that are a pain to turn off. I don't WANT something to automatically detect all wireless networks the second it sees a wireless card.

On the software front, what would be the point of "getting" Exchange, Adobe, Office as Linux-native versions or equivalents. By doing that, you would have to integrate a significant portion of Windows infrastructure, including Active Directory and DirectX. So what you've done is made a "free" version of Windows. Whoopee. Everyone who's currently using Linux is using it NOW while it's not a version of Windows... why? Because it's BETTER. It isn't bound by some stupid corporate decision or two decades of backward compatibility quirks.

Take a look at some edited highlights of Vista SP1:

Performance improvements
New security APIs
A new version of Windows Installer, version 4.1.[47]
Users will be able to change the default desktop search program
Support for the exFAT file system (FAT extended for USB sticks, basically)
Support for 802.11n.
Terminal Services can connect to an existing session.
IPv6 over VPN connections.
Support for booting using Extensible Firmware Interface on x64 systems.
An update to Direct3D, 10.1.
Support for the Secure Socket Tunneling Protocol.

What's there that Linux won't have by the time it comes out, if it hasn't got it already? What's there that Linux couldn't do? Nothing. And to be honest, as a changelog for a major upgrade to even a stable release of an OS, that's pretty pathetic. What about Server 2008? It's all pretty much the same. There's nothing in there that Linux doesn't already or couldn't do with a year or so's work.

Let's stop faffing about asking Windows users what they think they need from a Linux machine. Let's SHOW them. Let's just get stuff done and forget emulating Windows. We all know that Windows has it's death coming to it. The longer we give it credibility by attempting to copy everything it does, the more time we waste away from the interesting stuff, the stuff that will have people hooked. We have SELinux, we have file-server compatibility, we have directory management software, we have all of this but nobody cares. We need to show stuff that Windows can't do.

We need a five-machine network that can outperform the best Windows servers and individual desktops, when both are running 20 simultaneous clients (as in four people ACTUALLY WORKING on each of the five machines, locally). We need filesystems that "heal" (and not like self-healing NTFS in Server 2008 which is basically thread-safe Scandisk), network filesystems that can let Google do it's job without worrying and with which small companies no longer need to worry about tape backup (although, obviously, they still could) - which adds 50% to the price of any server.

We need perfect, logical, simple directory systems that can do stuff that Windows AD can't even dream of, in an easily editable/recoverable/backup-able format - it doesn't matter if it's Fedora Directory Server or Apache Directory Server - no-one cares. We need it all to run, automatically but securely. We need automatic secure communcations across a network to pick up new machines and integrate them directly into the Directory. We need systems that (with proper admin control over the process) auto-patch underneath systems that are still running. We need one-click Setup, Trusts, Backups and merges of entire Domains.

We need client systems that can repair themselves from known-good images (which, hell, should be stored in the cross-network filesystem) while they are still working - no, we don't acquire viruses but you still need to Ghost stuff back sometimes. We need machines that detect faulty hardware and compensate automatically - memory just failed in the server? Fine. Isolate the memory areas responsible (BadRAM), alert the admin, allow them to work-around the problem temporarily until they can get a replacement, restart and check all services and then carry on like nothing happened. And all the time you spotted it where Windows would have just crashed.

We need systems that can tolerate as much failure as possible. Primary hard drive or RAID array failed? Warn the admin, carry on as normal, read what you need off the network. Network failed? Route around it, over a USB connection if the admin only has a USB cable left, or FireWire, or wireless, or Bluetooth, or Serial, or Parallel, or... We need a real, "intelligent" help system. When it sees that admin hunting through menus looking at DNS settings, it tries to (unobtrusively) help. It brings up a checklist and works through things one at a time by itself until it says to the admin "The DNS server is fine. But you forgot to point that client machine at it." or "The DNS server doesn't have a reverse-DNS for that IP, that's why what you're trying isn't working".

We need systems that collectively monitor, detect and shutdown other rogue systems within their sight, a kind of distributed-IDS built into the system. We need systems that do all this 100% securely, with full certificate-chains and verification and let the admin control exactly what's going on if he wants. And when someone breaks that particular method of encryption? Ah, just choose one of the thousand-and-one encryption methods and do a one-time re-encryption to change every server, client and software over. Well, yes, do pick up local Windows systems and tie into them as much as you can but forget making that a priority. Set NEW standards. Make people say "I absolutely NEED a system that can do that." Let the other OS manufacturers play catch-up for a change.

Let's stop playing catch-up. We already won that one, there's no competition there any more, there's no more fun to be had. Let's start wiping the floor. Let's get JUST ONE feature in that people decide they absolutely NEED. And let's do it before Windows can even get a sniff. Let's do it so that, when the time comes for Microsoft to replicate it, they want to be able to read OUR code in order to get it done well enough. Let's stop playing about asking 90-year-old grannies why they don't like Linux when they know nothing BUT Windows... their answer will always be some variant of "It's not like Windows".... either that or "That penguin is scary". Let's make the people that are really scared of the Penguin be Microsoft and Apple. Because, at last and for once, they can't keep up with Tux.

Essential Linux Utilities

2007-07-20T19:15:00.000+01:00

Ever since setting up three Linux PC's in a row, I've realised that I've grown dependent on a few pieces of software for Linux, above and beyond what comes with a standard distro (or, at least, Slackware).

Beep - a tiny util that can beep the PC speaker in a variety of ways, perfect for headless systems. I use it to give a warning tones inside boot scripts and also to provide a rising or falling tone on the start or end of certain tasks, such as booting or shutting down. Because it uses the PC speaker, it doesn't interfere with ALSA, works on even the oldest of PC's, doesn't necessarily require an external set of speakers etc. Beware using it, however, on multi-user installations - I tend to keep it restricted to the audio group of users only to stop people messing about with it.

Ether-wake (available from various places, originally by Donald Becker) - the ultimate power-saving util... this is a Wake-on-LAN packet broadcaster to wake up computers that support WoL from their deep sleep (i.e. turn them on so long as they are plugged into the net and have a power cable in them). With this I keep my home network largely turned off and "wake up" (i.e. turn on) particular PC's as and when I need them. And larger scale experiments have shown that there's nothing better than the sound of a room full of PC's all booting up simultaneously at the click of a single button / cron job.

HTop - a better version of "top" that I find easier to use. Shows processes and RAM usage in a nice controllable text-mode GUI that allows you to kill individual processes, scroll up and down etc.

rc.firewall (See this post for a mirror) - a perfect, simple, one-file iptables firewall that works well as rc.firewall in Slackware. Works for single computers, NAT'ing routers, multiple network cards, multiple-networks-on-a-single-card, and lots of other configurations. It uses a simple syntax for even multi-port port-forwards, has many simple options for various things such as allowing or deny ping's or cross-network traffic, has a very strong default configuration and can be reloaded at the drop of a hat at which point all the detected network interfaces are re-firewalled.

x11vnc - This is one of those utilities that few people ever use. It's a vnc server for X. But it has a vital difference... it's a VNC server for EXISTING X sessions. Most people are familiar with xVNC which allows you to spawn an entire X-Windows system where each "screen" is actually a VNC session (thereby providing instant-VNC-thin-client) but that's not much use to someone that has a single-user Linux PC who wants to log onto their home PC and click on that link that they left showing in their browser. x11VNC does just that - the command-lines get horrid very quickly, you have to pay close attention to the security of the thing (because now connecting to the PC on port 5900 is the equivalent of logging in as yourself on the local PC!) but it's a great piece of software. The author is also working hard to make VNC-wrapped-in-SSH a cinch, even from Windows PC's, by extending the TightVNC clients to incorporate SSL tunnelling. Yeah, you can now do this with some things like KDE's Remote Desktop functionality but I've been using this particular utility for so long that I have scripts which build-on to it and it also has some features that just aren't present in other imitators.

knockd - a simple port-knocking daemon implementation which can be triggered remotely using either a tiny utility that works on Linux/Unix/Windows or by simpler tools such as telnet. Perfect for securing a server for remote access (and incidentally the best way to stop random port probes to your machine - my SSH logs were filling up until I found this) as you can just put the portknock client on a usb disk or a website and download it from wherever you happen to be or you can even "bodge" one in a real emergency. Also, the configuration basically consists of port-sequences and names of scripts to run. This means that it's easy to configure it to see port-hits on ports X,Y,Z as an instruction to run an "open" script and then you can hit ports Z,Y,X to run a "close" script. And because you can have multiple port sequences running, it's very easy to have all sorts of different things happening. See my article here for a bit more background on my use of this utility.

Mirror of Projectfiles.com / lfw.sourceforge.net rc.firewall

2007-07-10T20:53:00.000+01:00

Having just completed a set of instructions for a group of Linux newbies on how to set up a firewall, I then discovered that my favourite Linux iptables firewall script has all-but gone from the Internet. I checked Google, both "official" websites (including the Freshmeat.net mirror) and archive.org. Still no joy. Luckily I had kept a copy of this GPL script, which I have mirrored.

For those people who have had trouble finding the script that's been hosted at both ProjectFiles.com and http://lfw.sourceforge.net you can download the rc.firewall script at the following address:

http://www.ledow.org.uk/linux/

This is the 2.0 "final" version. I have the documentation mirrored too. Oh, and I assume that the reason that the archive.org site has no mirror is that the author wants no more to do with it. So be polite if you do need to contact them (the above file has their email address etc.) and don't bother me for support, either! (You probably couldn't afford me!).

The key to learning (and teaching) Programming

2007-05-15T17:22:00.000+01:00

The subject's come up many a time in my life about how people program computers. A lot of the time it's seen as a mystical art and some lay-people even confuse "programming" with "operating a computer", or even just "changing some settings".

The question always comes up about how to "learn programming". Besides the obvious problem with the wording of that phrase, it seems that people still haven't found an effective way to teach children how to program computers properly... and more and more teachers are skipping every bit of the subject that they can. My interest was caught by this subject again recently by a story on BBC News and by a subsequent appearance of the same story on Slashdot concerning a graphical programming language specifically aimed at younger children.

Programming is quite a strange discipline. It is the laying down of logical, unbreakable rules and actions to arrive at a predictable and reliable answer. However, it also requires, more often than not, ingenuity and creativity in order to achieve those aims. In some respects, it is similar to the cutting-edge of mathematical research (and, in fact, mathematics is probably the most complementary subject to computer programming, with the possible exception of electronic circuit design).

The problem with teaching programming is not that children cannot be brought to understand the subject, or even that the tools aren't available for them... most children of an age where they can comprehend what programming is are capable of actually programming. The problem is, in fact, the established complexity of the systems that they use each day and the over-simplification of their given tools. Additionally, the fast change of consensus on a "suitable" language stifles the teaching of programming, when in fact every programming language proposed has problems whether they be from an over-simplification of programming or a steep learning curve.

Also, the program which dumbs down programming to the point where children can "understand it" is probably one of the more complex programs to write. The abstraction required to allow a child to just drag-and-drop items into a graphical interface and thereby create a set of instructions for the computer to follow relies on decades of other people's code in order to display itself, allow the child to manipulate the mouse, and do so in a high-performance manner on a modern machine.

This is, in some way, similar to mathematics again... we all get told that it's pi-r-squared but finding out WHY takes more advanced mathematics to discover. Unfortunately, the modern consensus is that we have to assume that "it just is" for several years before we can get to the stage where we can work out exactly why. This doesn't present a problem in teaching, however, as almost all teaching comes from the concept of "good enough for now" or "lies for kids".

Teachers are also drawn into the same trap that stops children from being able to grasp programming: You can teach children for-loops and variables to your heart's content... the fact is that they will not be able to effectively link those concepts with the word-processor that loads in a fraction of a second with a thousand different features or their favourite 3D game that simulates realistic physics without a complete understanding of some quite complex mathematical principles. Most teachers, if pressed, will happily explain that "it all gets turned into 1's and 0's" (thereby introducing even more questions about how 1's and 0's can look like Mario running across a platform in time to their controller movements) but few can explain to a child's satisfaction just how it does that.

And when the children attempt to reproduce that 3D game in BASIC or Logo, they will be sadly disappointed that the best they can manage is a few hundred lines of code to make some circles appear at random locations on the screen (but it would only be pseudo-random, which is based on polynomial iteration, which requires advanced mathematics... and the circles would be only grid-based representations of the formula for the graph of a circle, which requires trigonometry to grasp completely... the traps are always there). Yes, there are such things as Dark BASIC, but again we're just abstracting away absolutely everything into a black box that "just works", rather than letting the children find out for themselves in due course.

That's not to say that there isn't some effort being put into teaching children programming, or even that that effort is failing. There are some very effective tools to demonstrate programming in a context where even the youngest child can interact and play with a system that they can understand. The best example would probably be the Lego Mindstorms kits - some building blocks and motors can be turned into anything once a child has learned how to make the control blocks do what he wants them to do. A common first foray into the programming world for many primary school children today is a Lego Mindstorms kit that they first build and then program to achieve an aim - whether it be controlling the traffic lights at a busy "junction" or raising and lowering a gate on a railway crossing.

Unfortunately, some of the Lego software is abysmal to understand on first glance, even for a seasoned programmer. Clunky interfaces, unintuitive icons and "settings" and an extremely limited instruction set, all of which are supposed to help the child understand.

When all you have to work with is "Output X on", "Output X off", "Wait X seconds", "Wait for input X to reach Y" arranged in blocks where the syntax is horribly restrictive and forces you (from a logical and interface point of view) to do things such as "wait for input NONE to reach NONE" on dozens on instructions where it would just be more intuitive to introduce some concept of our old friend IF instead, programming starts to lose its fun.

Rather than forcing the logical equivalent of null IF statements onto every line of code, a much better idea would be to merely teach the children how to program the old fashioned way - groups of english-like instructions, executed strictly in order. Unfortunately, the languages of today all have syntaxes which require pixel-perfect placement of semicolons, parentheses and quotation marks in order to write even a simple Hello World program. And although there are ways and means around that (such as seriously relaxing the syntax), people seem to have the idea that having to use WORDS is somehow dirty, when it comes to programming.

As a youngster, I was only ever officially taught (in this order) Logo, BBC BASIC and Java. Bearing in mind that the first was in infant school (early 80's) and the last on my degree course (early 00's), that's a huge and slow gap to a programming language of practical use (i.e. one that I would have available at home, one that I could make useful programs in that I could use each day, one that I would be able to swap, create and share programs with other people). It was only by my own experimentation that I was able to learn other languages myself and I was programming competently in the last two of those languages before my teachers even mentioned them. For my A-level's, I actually taught my own classmates for several lessons because the teacher believed I could do a better job than he, having been programming in the course language longer than he had been programming.

Logo provided graphical (or physical) response, but all code was textual. BBC BASIC *could* be graphical but was mainly textual and all code was textual. Java, again, *could* be graphical but all code was, again, textual. However, a lot of modern-concept educational programming revolves on getting away from actual words (the Lego Mindstorms "language" being a perfect example). Textual code is not something to be scared of. In fact, in other subjects, we treat textual lists of instructions as vital. And isn't that, when you get down to it, all a computer program is? We need to bring back the old-fashioned text languages, without the historical baggage of text-mode editors, strange delimiters and line terminators.

To clarify, I'm not seriously suggesting that learning C++ at a young age would have helped me or any child. That's a ludicrous assumption, similar to saying that being taught quantum mechanics as a child would help me understand Newtonian physics - of course it would but the complexities of teaching at (and learning at) that sort of level would far outweigh the benefits of such an education. My own teachers struggled with every language I had to be taught, until I hit university. There's no way on Earth that we could expect teachers to teach at the levels required in order to allow that to happen, at least not in the forseeable future.

What's needed is not a *primarily* graphical system. What's needed is a way to easily construct lists of plain-English instructions without needing to worry about perfect spelling, excellent grammar (the absence of which is, in programming, a vital learning experience) and lots of typing. Drag-and-drop keywords are a good start.

The language itself, however, needs to be less strict or, at a minimum, not allow beginners to make most of the classic mistakes. You can't mispell a dragged-and-dropped keyword and with proper graphical interfaces you are able to show scope, loops etc. effectively without having to worry how many invisible tabs you have inserted on the left of the screen.

Data-types, although important, do not need to be set in stone. Children SHOULD learn the difference between a string and an integer variable as it is of critical importance at each stage of programming. Visual Basic's "Variant" does an excellent job of masquerading as multiple types simultaneously but introduces new problems in itself because certain distinctions are not made. However, a simple indicator of Constant vs Variable, String vs Number, should always be included.

The programmer should also be responsible for creating the variables that they are going to use. Variable scope is vital knowledge. If this means clicking a new button and selecting a type, be it "Number" or "Animated 3D Penguin Character", the kids should be the ones who create him, the ones that drag him in each time he's needed and the ones who (critically) should remove him from the project if he's no longer necessary. This means not only in whatever GUI the programming language is set in but into the actual program itself. They should insert the Animated 3D Penguin not only into the project but also into the correct parts of the program to "create" and "destroy" him at the right times.

[[ On a side note: They should also be able to name him whatever they like first... they are kids after all. In fact, he shouldn't be able to be created without being given an assigned name by the programmer. No "Animated 3D Penguin Character #1" etc. ]]

Ideally, his birth and death should be graphical each time the program is run. This will teach correct construction/destruction techniques (because Pingu just popping into the screen half-way through the game isn't good but creating him at the start and just keeping him off-screen until needed is a good way to learn - similarly this will also show the brighter kids that you don't need to create everything at the start and slow the whole system down, if you manage the entrance and exit of data effectively).

Abstraction of complex functions, e.g. 3D libraries etc. also help the programming become interesting much more quickly (I can remember the wow-factor of my first coloured circle on a black background but children today would hardly be impressed with that). However, many of these can, in fact, be expressed in the language itself (albeit complexly).

Most importantly, a language should be complete and self-hosting... that is that each part of it should be capable of being written in itself. The beginners will wonder how this is possible but this is true for almost all programming languages. Except, strangely, the educational ones. By hiding the complexity within the language itself, you strip away the hard work into black boxes.

For instance, the "Create a new penguin called Bob" instruction should not be the end of the matter. You should be able to drill down. That instruction should have the capability to be broken into several hundred parts, each of more complex and primitive instructions, but all in the same "language". The hidden properties of him should be there somewhere... accessible to the more advanced children who want to do something that the basic levels of the languages don't allow.

A multi-tiered GUI is the perfect way of doing this. On startup, you have your "Penguin Bob" buttons but if you "break" them, they split into the code that creates Penguin Bob, in a simple-enough set of further instructions. And you can break them multiple times, into ever-more-complex code until you hit some equivalent of assembly language... most probably some sort of sandboxed, interpreted code similar to Java. Why? Because this is how ALL code works... the BASIC interpreter is probably written in C. The C code would be executed as machine code. Everything is based on an more complex underlying level. And as an extra added bonus, this means that the language that kids learn in Year 5 to make Penguin Bob run across the screen and giggle is still the same language when they hit Year 11, 12 or 13 and have to submit an independently produced program of a highly complex nature (for instance, their own word-processor).

It doesn't have to be fast... in fact it SHOULDN'T be, for the basic levels. This teaches the users quickly that unnecessarily looping ten thousand times over a hundred lines of code is bad practice. And when they grow up and are able to break the code down to it's components they will not only see WHY but be able to generate a much more efficient code that is functionally equivalent.

With widespread use, it would also encourage the greatest principle in programming that has been present since the first line of BASIC hit our eyes... collaboration. Libraries of code. Open source. Those BASIC listings in ancient computer magazines. All of them were about collaboration and sharing of code. Imagine if your GCSE project in Year 11 was to create a documented, stable, fast set of replacement functions that the kids in Year 7 would use to make Penguin Bob do a somersualt.

There also needs to be real, physical feedback. A coloured square on a screen means nothing anymore - especially when it takes £2000 or computer to do it, from a child's perspective. Logo (and some of its later clones, such as RoamerWorld) knew this... many a British child grew up by slamming a glass-domed turtle into the teacher's ankles. Lego Mindstorms spotted this. To incite enthusiasm for programming, there needs to be a response... not just from a computer screen. Things such as PIC-chips and simple robotic circuits make this a breeze.

And, with something along the lines of RoboWar, where students could compete in a school-wide championship to generate the "best" robot by programming him, you could involve the whole school. In one fell swoop, you could have programming on multiple levels (from all levels of AI to sandboxing), electronics, control, communication and design (by applying the "Robot Wars" principle to a full-scale, physical version of each robot), competition and publicity.

Each educational language seems to have one or two brilliant ideas and then to fall flat on it's face in all other aspects. Considering the computers are ever-more complex, ever-more prevalent and are not going to be going anywhere in the foreeseable future, our children should have an understanding of exactly the principles that underly them.

They will learn WHY computers cannot be trusted to run the world without an awful lot of testing (because of the absence of "intelligent" computers so often seen in Hollywood), they will have the ability to not have to rely on multinational companies to write a simple letter. ICT is already compulsory in many aspects in almost every school lesson, and it creeps into our lives more and more each day. Unfortunately, nobody is being educated how to make all this technology do our bidding, instead relying on a few good people to do everything for us.

Retro-gaming value for money

2007-04-14T22:11:00.000+01:00

Take one old PC (considered obsolete by its previous owner and consigned to the scrapheap), add a lashing of a spare operating system (Linux is fine but, in all honesty, Windows is easier for this project). Mix in some random game controllers chosen by lucky dip into your "parts" box (anything from an expensive Playstation joypad with a USB adaptor through to an old two-button gameport joystick is fine). Add a dash of emulators for older games systems that you knew and loved. Sprinkle with a hefty swig of configuration, plugging in and cursing. Decorate with a lashing of ROM downloads of games that you still currently own but can't be bothered to play because the setting up of the games console that runs them takes longer than completing the game.

Finally, download lots of popular freeware games, give them a try-out, then have a retro/freeware-games party with anyone that has a favourite old game they haven't played in years. A TV-out capable card will help matters immensely, allowing you to play them on the telly in the front room, just like the old days. If not, a nice monitor will do just fine. Or for those lucky enough to have a projector, this would be the best way to keep a room full of big-kids happy.

Seriously. I've just done the above with my own ingredients... a 1GHz PC with 512 MB RAM (although you could probably get away with even less). I slapped a spare copy of Windows XP Home on it (I always end up with a little collection of Windows XP licenses from the various computers that I repair or, if they are beyond repair, recycle). I "upgraded" the onboard graphics to an ancient and obsolete AGP 4x card because I wanted TV-Out (the less said about that the better - old TV's that can't display the refresh rates you want without flickering can ruin hours of careful planning and playing with settings).

I threw in a spare 4-port USB card and as many reliable games controllers as I could find. A cheap cable to connect the sound through the television, a few extender cables for the various controllers (because in the heat of a gaming party someone WILL tug on them) and a cordless keyboard/mouse set to control the system from the living room sofa. A little utility called Joy2Key also came to the rescue, especially if you do what I did - I made a tiny Visual Basic util to turn the machine into a primitive "who pressed first" quiz buzzer - more on that later.

The emulators available to such an "obsolete" machine cross all sorts of systems (but only those ones that I ever played on or still have... I have somewhere a CD with Commodore 64 emulators and ROM's but I don't think I've ever even loaded it because I never used to have that particular piece of gaming history, so I have no interest in seeing the games it used to play.) Currently, my particular setup has (in order of age), ZX Spectrum, Sega Master System, Nintendo Gameboy, Sega GameGear, Philips CD-I, Sega Megadrive, Super Nintendo and Nintendo 64 emulators, all of which have several popular games (everything from the previously mentioned JetPac through to Super Mario 64) loaded and ready to play, full screen, full speed, on this old relic of a PC. Anything of the Nintendo 64-era or less is fully playable on such a machine, given the right emulators, and more powerful machines will laugh at them.

It even has DOSBox running several old DOS games that I've only just consigned to the "cupboard of slow, ageing death" where games live that are no longer easy enough to play on modern systems but which I still have a fond memory for (Syndicate, Cannon Fodder, Command & Conquer, Settlers etc.)

And it plays them all fantastically. The foresight of using my best two games controllers (a USB PC joypad made to duplicate a Playstation Analog Dual Shock and an actual Dual Shock with a Playstation -> USB convertor) even allows precise analog control of Mario in Mario 64, or while driving in Mario Kart. And with a press of a button and a change of emulator, it's suddenly a Kempston joystick in a ZX Spectrum.

It's been great to play all the classics. JetPac just isn't JetPac unless it's played on a TV and is emulated so accurately that all it's little palette flickers happen just right. And Target Renegade is still a fantastic game for picking up for only a few minutes for a quick bash. Super Mario 3's Battle Game (the version from the SNES remake) is a great party game.

So every game I've ever owned on a games console is now sitting on one easy-to-manage, quick-to-boot, no-fiddling-with-TV-tuning PC with gamepads enough for a four player battle in any emulated system that supported it. When I originally had this system set up, it was on an equivalent laptop and that was even more fun... there's nothing better than being able to whip out your laptop at someone's house and launch into four-player frenzied gaming of some of the classics.

And I also loaded this PC up with some other games that I had lying around or had downloaded previously...

- Slicks 'n' Slides
An old DOS racing game that I was a registered user of, way back when. With a utility like Joy2Key, you can play four-player with ease (beats having to cram four of us around a single keyboard like we did when we used to play this in Maths lessons in school!).

- XQuest 2
A marvellous freeware game that requires precision control of the mouse

- SCUMMVM
An emulator for various point-and-click adventure games, which comes with Beneath A Steel Sky as a freeware game

- OpenTTD
A remake of the classic Transport Tycoon Deluxe

- Abuse-SDL
The dark and fantastic Abuse from Crack dot Com.

- Rocks 'n' Diamonds
Another remake, this time of Rockman.

- PySolitaire
A card-game to wipe the floor with all the rest (but you can still add Solitaire and Hearts to your PC "console" if you want) - hundreds of customisable card games in one program.

- Liquid Wars
A brilliantly original game, part strategy, part fast reactions.

- Battle Painters
A wonderful game to leave the kiddies with - each player has to paint as much of the screen as possible before the timer runs out. I use Joy2Key on this as well, for the same reason as Slicks 'n' Slides, though this is a Windows game.

- Super Mario War
The best Christmas present I ever gave my mum - she is a massive Mario Battle Game fan and this gave us a night full of entertainment. Mario-themed but the number of games alone is enough to keep the most ardent Mario-hater busy. And such four-player mayhem is incomparable.

Not to mention TuxRacer, Pingus, Gate 88 and all those other great freeware or open-source games.

It's the best games console I've ever had. I even ended up naming the PC "CONSOLE" in it's configuration. It doesn't have network or anything else set up - it was worth sacrificing an old PC for and, as far as I'm concerned, the best use of a Windows XP license ever.

I turn it on. Within about 30 seconds it's in Windows (no net = no need for lots of utilities / services that slow it down) and ready to play. That 30 seconds would be taken up by switching to the right SCART socket, untangling the joypads etc. anyway, so you don't even notice it. Then, when you're in, you can play some old ZX Spectrum game while you wait for your loved one to settle down with a controller that they don't mind using, then you can both have a blast on Mario Kart 64 or play some silly freeware game without having to switch machines. When she goes to bed, you can load up a bit of Syndicate or finish off that Red Alert campaign.

You can have "Mario nights"... Start with the original NES versions, then do the Gameboy ones, working your way up to Mario 64 and filling in the time with some Battle Games, a quick blast of Mario Kart or a four-way battle in Super Mario War. And when anyone comes over and says "Do you remember ?", you can just load it up and play it.

It's great and it's the best investment of time I've ever put into a Windows system. I was going to download one of these menu systems that they have for multi-emulator setups but they were all pants and I found that a desktop folder called Games with a shortcut for each machine type is more than good enough. When you have a lot of games on the system, you would spend longer setting up a menu system and then selecting them than you would just loading up an emulator and typing in the name anyway. You can assign shortcut keys to each shortcut and, if you want, use something like Joy2Key to load up particular ones from the joypad!

Joy2Key is a bit of a fiddle but it's only really necessary for comfort on old games or emulators that can't reassign keys to joypads easily. Do it right and you have the "best" joypad be the controller - the one that can pause, open games, exit emulators etc. by pressing the right buttons on it. Or you can just delegate the keyboard to someone who knows the shortcut keys for each emulator.

I found that the Playstation-style joypad controllers were the best to use. Enough buttons for almost any game system (with a few spare for vital functions like Pause etc.), digital and analog controls, cheap, comfortable, easy to get hold of and easy to find adaptors for (in the UK, Game sell them in all their shops). They're also especially close to the original SNES controllers that I end up playing more than other types of games.

The most time-consuming bits were:

- Making it boot fast
Don't install anything that you don't need to (networks etc. included). Clear out all your startup entries. Set all BIOS options etc. to be as quick as possible. Make your desktop as plain as possible and completely empty your system tray. Make sure everything is plugged in each time you use it. You don't care that the system looks ugly, but if it jerks in the middle of a four-player game you'll be regretting putting that large theme on your desktop.

- TV-Out
My own fault really, because I was trying to use what I had to hand. Only when I was several days into getting it to work did I realise that all my problems stemmed from a television that just wouldn't support non-standard refresh rates at all, not even by the slightest degree, but didn't moan about them. I have an external D-Sub -> SCART convertor that just wasn't up to the job of changing refresh rates without having MASSIVE jerkiness (single screen games worked fine but anything that scrolled horizontally or, worse, vertically looked absolutely crap).

So I tried a video card with TV-Out which was perfectly smooth at doing that job, but which had BIG problems with colour display on my TV. Turns out that the old TV just doesn't support colour over a S-Video connector, even if it's connected via SCART. That wasted HOURS of fiddling, soldering, setting-tweaking etc. and on any other TV it just worked fine... so I used a different TV. My next step will be to get a nice flatscreen for it. The most important thing is that everyone can see it, so a small flatscreen with a limited viewing angle is a no-no unless you have some kind of dual-head setup and run one screen for each "team". (P.S. A very brief experiment showed conclusively that playing Jetpac on a 10-foot-wide projector image connected over S-Video is the coolest thing ever, but a very expensive way to run a games machine, so it's just not practical).

- Emulator configuration
Every one had it's own idea of what was Joypad 1. Every one had a different set of buttons on it's emulated controllers. Quite a lot have some weird options or programming that make them totally unsuited to the task. Some quit out if you pressed the same button on the joypad as selected the menu too many times (highly annoying when in the middle of a game). I had Gameboy emulators that couldn't emulate 4-colour Tetris as fast as a Nintendo 64 emulator could emulate Mario 64 on the same setup. I had Megadrive emulators that ALWAYS tore the screen no matter what VSync/Resolution/Buffering/Refresh options you chose. I had a CD-I emulator that will only accept mouse input. I had DOS emulators that needed settings tweaked for each and every game to get the best out of them (and even then at least one of them needed Joy2Key to make it easily playable). ZX Spectrum emulators offer emulations of so many different types of joystick (some of which conflict) that it's a pain to configure them all. Some I fixed, some I persevered and did it the hard way, some I ditched and replaced with other emulators, some I set up with certain limits (i.e. no more than four players, etc.)

- Distractions
Every time you set up an emulator, you just HAVE to test the games several times to make sure that they are all right, don't you?

- Programming
Yep... I did a bit of programming. It's a party machine, it's obvious from the outset. This thing is great at parties. And the best bit is, it's multipurpose. The hardest thing in the world to get is a decent Quiz Buzzer - one of those "who buzzed first" things. So I wrote one. I misused Joy2Key so that any button on the first joypad "typed" a 1 on the keyboard, second joypad 2 etc. and then I wrote a small VB program that just waited for input after the quizmaster had "opened the question" and then whatever character was typed first was declared the winner. It probably wasn't perfectly accurate but it did the job and took about twenty lines of BASIC code and a freeware Joy2Key to do it.

It's a good project to do. It requires limited resources compared to other PC projects (office and schools are throwing out PC's that can do this, TV's are perfect for playing old games on because you don't need the high-resolution of a monitor, any gamepad or joystick will work etc.). It is immensely fun and can be as simple or as complex as you like. It brings people together.

And I guarantee you that, if you are of the age that you can remember playing these sorts of consoles and not just Playstations, you will get ten times more play out of it than spending the equivalent amount of money on the newest games console. For the price of a new games console and a single game you can have every game you've ever played on in full better-than-the-original glory.

Jetpac

2007-03-28T17:52:00.000+01:00

Seeing how I've been looking back at some of my older games recently, having been disillusioned with most modern ones and without a computer to play them on for a while, I was horrified to spot this bit of news:

Take a 16 kilobyte game, of classic nature, with fast, simplified gameplay, clear objectives, simple controls, that used to run on a processor of only a handful of MHz.

"Upgrade" it for modern machines, in the process making the screen hideously difficult to see what's going on, full of fancy unnecessary effects, several of which do nothing more than make the ENTIRE gameplay field obscured from the player's view, add extremely primitive multiplayer into a game never designed to have it and only release it (online) for one of the most powerful modern games consoles.

And in the process remove years of good game-playing memories.

Linux - Good enough, Easy enough, Supported enough, Common enough.

2007-01-07T01:27:00.000Z

I was recently sitting on a bench on the platform of a local train station when a large man, who was sitting next to me, started talking to me for no apparent reason. This isn't unusual - I must attract local nutcases - but I worked out who he was when he asked if I was "into computers". Seeing as I had no indications on me that would suggest I worked with computers, I quickly realised that he must work in the school that I had just finished working at.

I was right, he was a maths teacher there, and once I had realised this, we introduced ourselves and got talking. First, my answer to the question whether I was "into" computers was a little understated. You can't move in my house for flashing lights and humming fans and it's been that way for nearly ten years. It's my hobby, my job and most things that I do in some way tie back to computers.

Anyway, he started to demonstrate his knowledge of computers, a trait that I find common among people who discover that I "know" IT. The first words out of his mouth took him above the average Word-letter-writer and placed him into the "geek" bracket - Java (in terms of programming environments and not just "that bit in IE that makes fancy websites"), LaTeX and Linux. For those unfamiliar with LaTeX, it's a typesetting language designed for hard-to-typeset symbols, usually used with mathematical formulae. Tying this together with my earlier discovery that he was a mathematics teacher at the school, I quickly realised that he a) actually knew more than average about IT, b) he was researching things to help him work and that c) he was already aware of free and open-source OS's and software.

(Incidentally, he was shocked that I had a degree and further shocked that it was in Mathematics and Computing - he hadn't expected that in a lowly computer technician. That made me into "someone he could to talk about mathematics without having to dumb down")

This mention of Linux and LaTeX further elevated him to "person that *I* can communicate with on a technical level". The fact was that he was using Linux, LaTeX and Kile to do his job, without anyone "making" him doing it, without anyone even knowing he was doing it and without any sort of help from other people got us talking all the way to our station.

He had a Linux desktop (although his technical knowledge hadn't quite reached the level of compiling his own software, so everything was RPM-based), he had found a Linux-based program that he found interesting and useful to his own work and he was trying his best to get things running as smoothly as possible. He asked if I would mind if he popped into the ICT office the next time he was around in order to ask some more questions to help him get enough things working that he could demonstrate Kile to the rest of the Maths department in the school. Immediately, I said "of course not". And why not? Because not only was he POLITE, not only was he TRYING to do as much as he could understand (the two main determining criteria on whether or not I go out of my way to help somebody), but he was also trying to use Linux to get stuff done, which isn't the easiest thing for a relative novice to do.

Similarly, my brother has recently had to help the local Scouts complete their IT proficiency badges. My brother is quite a Linux fan but has never really "run" his own Linux system. He has a router/firewall/storage server/print server/emergency-desktop system that I set up for him some years ago and is capable of using it and managing it himself, so long as there is someone like me or a Google result that helps him find the command he wants, once he's discovered he needs to do something.

The Scout IT activity badge is remarkably well-designed, unlike some computer-based achievements for children, in that it does not make mention of ANY particular computer or operating system. Criteria such as "Create a simple website" or "Take part in a video conference" are worded so that, although they may suggest examples for possible software, they are not heavily biased in terms of the platform on which the activities must take place. Most, if not all, requirements could be completed on something as antique as DOS or CP/M!

This is especially good given the average budget of a small Scout group who would be required to arrange such activities on a fairly regular basis. A few old, obsolete machines loaded with even FreeDOS would be able to complete enough criteria for most of the stages of the IT badge, without unfairly hindering the children required to complete them.

It was with this in mind that my brother, a born-educator, wanted to provide a challenge to the more "cocky" Scouts. Those who have been trying to show off that they "know" computers when in fact all they have ever used is a Windows XP PC which was already installed and set up for them would, on the day, be faced with a Linux-KDE desktop. To the experienced, and genuinely talented, children this would be little more than a cosmetic hinderance. To those who have just memorised that they click the "Blue W" or the "Blue e" to get things done, it would be an awakening.

By the time these children are in the workpool, it's possible that the OS's that are commonly in use today would be completely obsolete and not just from a versioning point of view, but from the methodologies and techniques used. As an example, when I was at school, for my "official" education, I was taught to use BBC Micros with 5.25" floppy disks, followed by Amstrad PCW512's with 3" floppy disks with integrated CP/M, moving onto Windows 3.1 machines (with MICE!), which meant drag-and-drop, windowing metaphors, multi-tasking etc., and finally onto Windows 95 (which introduced me to the world of computer crashes and diagnosis like never before). By university, we were introduced to Linux, Apple Mac and SGI UNIX workstations.

So, if I had "learned" computers only from those that were taught TO me from my school days and then been thrown into the workplace, I would have to relearn everything that I knew. The only way that I was able to keep up and still stay ahead of the class (and the teachers) was to not "learn" specific operating systems, terminology, methodology or icon locations but to spot the patterns and learn how to operate any generic computer. This was mainly aided by the fact that I was exposed to lots of different OS's and architectures before I had left secondary school. Since then I have applied these skills to operating systems that I had never seen before. I have managed networks that I have never had any formal training on with only a few seconds in which to "learn" the system. And I haven't managed to blow anything up yet.

My brother learned the same way, that a broad, general education is far better than a targeted, by-rote education, and wished to convey this to the children at the same time as doing their ordinary badges but without hindering them in such a way that it would interfere with their actual results.

Thus, he wanted to trial a small Linux desktop system to use on the day.
He set aside an entire evening for installation, not including hardware setup (inserting hard disks etc.), plus an old 833MHz computer with 128Mb RAM.
It took us less than an hour to install, from a blank harddrive and 2 Slackware CD's.

The next thirty minutes he spent marvelling about how easy the setup process was, how much of the hardware it supported, how much stuff was "pre-installed" with the basic system (enough to do everything that he intended to do, even though he had initially wondered about installing OpenOffice, the built-in KOffice was consider more than adequate), how "obvious" the setup questions were (although he is more than computer-literate, he still loved the explanations given for options that he was unfamiliar with - most of which told him exactly what each option did, why it did it and what he should choose if he was unsure), how customisable the whole thing was, how secure (when running as an unprivileged user) it was and how quickly and smoothly it all worked. The step-by-step instructions held his hand just enough (although he doesn't really need it and isn't put off by reading a HOWTO or asking on a forum or even just experimenting) at every single stage and I usually made him make the decisions about what option to choose - if in doubt, he chose defaults.

Considering it was Slackware (a server-based distro, really, and an older version at that), on an old machine, using more swap to get stuff done than real RAM, with the default 2.4 kernel, with the default VESA drivers on a crappy 16Mb graphics card, without any sort of real configuration, turning features off or having to fiddle, he was impressed at just how fast and usable a desktop system could be created so quickly. There were some rough edges, which was the main reason for my presence as he was perfectly capable of trialling this himself.

For one thing, he had to type "startx" at one point! And then later edit inittab to make it always boot graphically. He had to select hdb in one place instead of hda despite there only being one drive present (weird ancient BIOS assigned the first hard drive to hdb when hda was empty?!) and I advised him to install LILO to MBR instead of the other locations. That was about it. And none of those problems would be present on any desktop-targeted distro.

However, once in the GUI, he created his own, locked down users (i.e. normal users other than root). He had office suites ready and they were trialled at the touch of a button. He was able to change the clock that was out of date - although he was asked for a root password as he was logged in as one of his restricted users at the time. At no point was he asked for driver CD's, Windows updates, or to reboot. Nothing took over his hard disk or wiped out boot sectors without asking. Functionality that could only be enabled with the addition of extra software on his Windows was there by default - multiple desktops, multiple clipboard entries, etc. Everything ran smoothly without any knowledge of the hardware involved (although, admittedly, we didn't do anything more complicated than join the local Ethernet).

And I know from experience that sound (had this computer had possessed even a built-in soundcard) would work flawlessly without "drivers", printer and scanner support would be the work of a few seconds even on Slackware, never mind an auto-detecting desktop distro, wireless would be just as easy and well-supported. The only problems that MAY have appeared would be with exotic or entirely obsolete hardware - and hardware that exotic or obsolete would either not work at all in Windows or certainly not work without installing lots of drivers. With that effort, they could almost certainly be made to work on Linux just as quickly.

We seem to have reached a plateau. Linux is "good enough" for most tasks, "easy enough" for most people (technically-minded or not, especially if pre-installed for them), "supports enough" to make it run on virtually any hardware no matter what its vintage, "common enough" that even relative novices are hearing of it and using it (and in fact most places are using it whether they know it or not in the form of in-house black boxes, routers, TomTom kits, ISP's, webhosts, firewalls, etc.), "supported enough" in that the simplest of Google searches will throw up hundreds of places to find help (whether by yourself, directly from other people or from somewhere that will sell you support).

And it seems increasingly true that Windows is fast becoming "not enough". Not secure enough (Let's not even get into that debate - I'm taking Windows + supplied software + latest updates against Linux + supplied software + latest updates). Not fast enough (with modern Windows' hardware requirements). Not forgiving enough (of sloppy users, old hardware, etc.). Not usable enough (with more restrictions, problems, confusions, distractions and idiosyncracies). Not simple enough (seven different versions of Vista, difficult to install from scratch on "unusual" hardware, harder and harder to get simple stuff working, more and more complex to secure). Not cheap enough.

The only question remains, how long until you've had enough?

Slackware 11.0 - first experiences and upgrade process

2006-10-03T13:41:00.000+01:00

Well, Slackware 11.0 is officially out after an exhaustive batch of release candidates and (due to some hideously attentive monitoring of the ChangeLog in recent weeks) I've already upgraded to this system. In fact, I've been downloading each new package for the last few weeks as and when they changed the Changelog, and about once every few days, I'd create a DVD and install it onto a copy of my primary partition to find problems before I took the plunge and actually started using it as my primary desktop (replacing an up-to-date, clean Slackware 10.2).

The upgrade, as ever, goes like a dream so long as you follow instructions *very* carefully - don't omit any steps. We won't mention my moment of idiotic forgetfulness where I forgot to upgrade the rc.udev file once I'd installed it or even failing to copy the initial /dev from my existing partition to the "mirror" partition that I was upgrading, both of which caused Slackware to fail to boot... One of those is mentioned in the upgrade.txt (transfer ALL the .new files across! In a moment of blindness, I omitted rc.udev.new), however the other was just common sense if you intend to work from an accurate copy of your existing system! Those installing onto a clean partition should have no problem at all.

After many, many tests (I was, after all, performing a major operating system upgrade on a system that was still being used for "real" work), I copied my Slackware 10.2 main partition to a blank space, freed up 1.5 Gb on it (as leeway for new packages, upgrades, temporary files etc.) and installed the upgrades over the top of this copy. Once I'd followed the upgrade.txt (which, at the time, was the 10.1->10.2 upgrade.txt but still the principle is the same), all I had to do was recompile my kernel (Slackware 11.0 now ships with GCC 3.4 which means you also have to recompile any custom kernel or kernel modules that you may have had from Slackware 10.2, which only used GCC 3.3), reinstall lilo and it all just worked.

However, be very careful if you have extra modules in your kernel (e.g. nVidia/ATI drivers, out-of-tree wireless drivers etc.) as they *will* need recompiling. When you are using the same compiler and kernel on two different machines, the modules are usually transferable between the two machines, but Slackware has changed the default compiler so this time-saving trick no longer works. Failing to recompile them *will* crash your machine, maybe not immediately - for instance, in my initial testing on a blank partition, failing to recompile the nVidia modules and instead "copying" them from a previous installation crashed the machine hard as soon as OpenGL was used, but had functioned perfectly until then (even accelerating X and video flawlessly).

The crash was so hard that the (journalled) filesystem stopped halfway through a write and corrupted the partition - proof, if ever it were needed, that the use of proprietry modules removes any guarantees of stability and also that journalled filesystems and RAID are no substitute for adequate backups. That's also why you should always backup and/or test on a copy of your primary partition before you do stuff like this - I did it out of academic interest and was surprised that X or the kernel didn't throw up more warnings.

Applications, however, should not require any re-compilation at all, unless they are very tightly integrated into the kernel or statically build from the supplied libraries (something that they shouldn't do for most purposes). I haven't found anything that I've needed to recompile except for kernel modules but I'm sure I will find something that will have stopped working - a lot of stuff uses the kernel as the definitive source of information on things like kernel structures etc.

KDE was neatly upgraded to 3.5.4 in the process, all my old settings just ported over without any hassle (although a few KDE-specific tweaks, such as what the taskbar looks like and how multiple-desktop thumbnails work had reverted to a new default - easily changed and they were the exception rather than the rule). And it still runs like a dream.

Given a 1Ghz, 512Mb RAM machine, there is no significant detrimental performance difference between Slackware 10.2 and 11.0. In fact, because of both the KDE and X.org upgrades, programs under X run noticeably smoother - this is an old machine and small optimisations make a big difference when you don't use eye-candy like transluceny and anti-aliasing. Even code like a static-QT installation of Opera 9.0 is seeing responsiveness improvements compared to before. Given that Slackware is not designed as a desktop OS, it functions admirably under such circumstances and the system requirements are minimal.

Because it's Slackware, most desktop software will require, at some point, extra libraries or installations (for instance, mplayer codecs and OpenOffice.org are not included) but everything will usually compile cleanly from source without any patches or there is always Linux Packages.net for packages of any extra software you may need. Slackware's main install is only around the 3-4Gb mark when installed (depending on filesystem and block size) so on a modern hard disk, there's plenty of room for extra software. My main partition (not including personal files in /home directories) rides comfortably around the 10Gb mark and there's always plenty of space for a full install of Slackware plus Wine, Crossover Office, Microsoft Office, OpenOffice and many, many other large pieces of software.

One word of advice - take upgrade.txt's suggestion to just "install the rest of the packages" lightly - in fact the best method, especially if you are short of disk space, is to go through each package directory one-by-one... e.g. upgradepkg --install-new /root/slackware/a/*.tgz etc. Not only does this make it easier to omit the KDE/Koffice internationalisation packages for languages you don't speak (e.g. upgradepkg --install-new /root/slackware/kdei/*en_GB*.tgz), or to omit those packages that you don't need installed anyway (TeX or emacs for example), it saves a lot of time and diskspace and prevents you upgrading to a 2.4 kernel (hiding in slackware/k) and then having to reinstall the 2.6 kernel packages from /extra.

In terms of the final product, hotplug/udev is greatly improved and detects most peripherals and uses them automatically - nothing new or exciting unless you've not run any other recent Linux distro, but being able to plug in a USB drive, joystick or mouse and have it instantly recognised and have X/KDE start using it is a welcome return to the ease of a typical Windows installation. This does require a relatively modern 2.6 kernel though, but the scripts are still designed to take account of older kernels (2.4 or 2.6) that are not able to do this. And yes, 2.4 kernels are still the default for the time being.

One other change to the install process is that the sata.i bootdisk is now set as the default for any bootable CD or DVD (even the text on "how to boot in an emergency" on the boot screen reflects this), allowing direct installion on a old-style-PATA or shiny-new-SATA harddrive without having to select a different bootdisk - apparently a code conflict between the modules for the different hardware has now been resolved, making this possible. It's a welcome simplification to the install process.

All in all, the installation was fairly flawless but be careful about your kernel - unless you stick with the default 2.4 or 2.6 kernels (which will be out-of-date within a week or so, if not already) you are going to have to recompile the kernel and any modules and then reinstall LILO or GRUB. The only other thing to remember (and it's in upgrade.txt) is to ensure that all the .new files that appear on your computer have your settings transferred into them and then rename *them* to replace your original configuration - this way you won't miss any new config options that might have appeared.

I suppose the biggest disappointment would be for Gnome users - there isn't a sign of Gnome left in Slackware (the distribution cited ease of compilation/packaging as the reason for its removal in the last release) although you can still get Slackware packages for it from various third-party sites. To me, this went unnoticed as when I first started off installing Linux with X desktops, I tried both of the major window managers at the time and KDE came off best every time. Gnome felt clunky, old, out-of-place, like the Borland Windows dialogs used to back in the early days of Windows... nothing WRONG with them, they just didn't fit.

They are both now skinnable and in fact either can look like the other, so it's not a win-win situation - however, because of that there's also little reason to claim Gnome's loss is devastating... KDE can be made to work just the same and in fact the two projects are collaborating on just about everything these days. The GTK libraries etc. are still installed by default and, in fact, some ancient Gnome-based software that was left on my setup from its previous Slackware upgrades still functions perfectly.

People say that KDE is full of bloat but, I'm sorry, 3Gb for an entire OS including X and an office suite? That's well within the realms of convenience on a modern computer and most packages can be omitted if you really want (you can get a X installation down to less than a Gb if you really try and omit all the rubbish - I'd hate to imagine what the absolute minimum would be - I should think it would be amazingly small). And a 512Mb system showing only 100-200Mb in use when I have several applications open (and a few dozen background processes including Apache) under X/KDE is perfectly acceptable. And with KDE4 currently in development, the introduction of QT4 is supposed to make everything so much faster and leaner. But let's not get ahead of ourselves.

Altogether, Slackware 11.0 is another flawless upgrade of a "clean source" distribution - there are very few patches to the software included on the disks and the kernel is always "pure"... making upgrades, recompiles and troubleshooting simplicity itself. I should also imagine that it makes the lives of the software developers much easier as the bug reports are directly relevant to the software, rather than patches that the distro has tried to add itself.

Computers, gadgets, modern life and control.

2006-09-07T15:09:00.000+01:00

It occurs that I've discovered a major theme in my life - control. Everything that I own has to be under my control. This obviously starts with my PC. Starting out on DOS, I was used to controlling most aspects of the computer's operation; I could control what programs it ran, whether they stayed resident or just ran once, when they ran, etc. Windows 3.1 also allowed such control but moving towards the more modern Windows OS, I gradually realised that I was losing this control. I didn't know what programs were running, when or why. I couldn't change settings, I couldn't rename stupidly-named menus or folders, I didn't get choices over where software installed itself and a lot of the time I couldn't change it later.

So I switched, to Linux of course, and now I have my control back. I get to decide what runs and when. Programs don't mysteriously insert themselves into the depths of my computer without me a) knowing, b) being able to revert the changes or c) having some way to prevent it. Software doesn't taunt me with promises of being able to do something only to stop me doing that exact thing until I pay more money, take out a subscription, click on an advert etc. If it does, there's always an alternative somewhere that I will be able to freely use. This is what used to make Windows more tolerable - If I couldn't control some aspect, such as which programs are allowed to connect to the Internet, I could always find some shareware or freeware that would allow me to control that aspect.

Unfortunately, this modern epidemic of removing control of modern electronics from the user's hands (the person who PAYS for these same products or services and enables them to be produced) and into the hands of the company (that receives the money, could not be in business without the user and whom, without checks, would dictate when, where and how much upgrades and further products will cost) has reached even the humble television.

Televisions and VCR's were always quite complex for the inexperienced user and rather difficult to exercise control over. I've seen at least one TV that, by a single keypress, will wipe out all its stored channel information, seek out every frequency and insert only discovered stations back into a list but in the order it finds them rather than any realistic ordering (such as by channel number etc.) and in so doing wipe out any channels for VCR's, games consoles or camcorders that might have been painstakingly set up.

The programming of the average VCR was legendary but towards the end of its life it became greatly simplified - no longer were they used as tuners but just as plain recorders, thus eliminating one level of commplexity in setting up recordings. VideoPlus codes enabled simple future-recording of programs on any channel for the exact time required to capture a particular program, even in some cases accounting for last-minute re-scheduling. Now most of that same functionality has moved into the era of the DVD-Recorder and HD-recorder although if anything the complexity is even more reduced - integration with automatic digital TV programme guides enables one-touch recording of future programs just from advertisements or trailers on any of a thousand channels. Programme guides also make browsing and recording any program on any channel on any day a breeze.

However, DVD has brought with it restrictions - restrictions which remove our control. You can't play an American DVD on a British DVD player - why? Not because of a technical difference, not because of an incompatibility but because the DVD inventors and distributors don't want to let you. Why? So they can sting more money out of you if you happen to live in such a closed market. The solution? Most DVD players sold are now either multi-region by default or multi-region capable. However, problems still remain with this format - UOP (User Operation Flags - those restrictions that prevent you skipping trailers, adverts, copyright warnings etc.). Again, these are an in-built mechanism designed to do nothing else but ensure that you see those messages.

When you are browsing your DVD's or waiting for the film to start UOP's are usually mainly used on trailers and commercial content that the user just DOES NOT WANT TO SEE, ever, and after they have seen it once, why would they want to watch any of it again? Sure, give them the option to see it again but let them skip it if they want as well. What's so wrong with a menu option on the DVD that lets me see trailers and other notices IF I decide to? I am perfectly aware of copyright law and if I wish to skip that inevitable 20-seconds of static screen I should be able to. Few DVD players have anything that lets you bypass the DVD's UOPS - however most PC DVD players can be patched or have utilities installed that will make them bypassable.

In fact, the libraries and media players that I have installed on my Linux computer just to be able to watch DVD's in the first place automatically do this for me - I just press skip and it skips forward, no matter what the DVD says I can or can't do. My control is restored, just not in my own front room where instead I make a point of noting which DVD's have excessive UOP control and either copy them, removing it in the process, or label them so I know to put them on five minutes before my tea is done so I can be out of the room while they go through their rigamarole. If neither is possible or practical, I merely make a mental note to never buy any of the things advertised and to try not to buy from the same distributor again, or at least not until they sort their act out. I honestly have a product/manufacturer blacklist in my head whenever I purchase a product.

My PC speaker's don't insist on locking themselves to full volume when a banner advert plays sound because it's intrusive, obnoxious and counter-productive (I will not buy those speakers or will stop visiting those websites) so I don't see why I should be forced to sit through ten minutes of adverts just to get to the film that I'VE PAID TO SEE. The same principle applies to cinema - how many people don't go in until the main feature has started, i.e. ten minutes after they say it starts on the ticket?

My car takes a lot of my control out of my hands but for very important reasons (my life, my safety) I still have control over some important points. I don't expect to be able to modify the algorithm that controls the activation of an airbag by any setting or by modifying any software or hardware - I wouldn't want to and there is no reason to. However, I would expect to be able to disable it, for instance, in the case of an emergency or if someone was working on the steering rack. This particular device is something installed to save my life - I don't expect it to be at all tinkerable or for it to be disabled without major interference with the car and major warnings (i.e. my airbag light staying lit on the dashboard).

Additionally, I want to be able to sue the airbag manufacturer if it failed to be deploy in an emergency situation (or my relatives but ideally **I** would be the person suing the company!). Therefore, I don't need to, or want to tinker with the details of its operation but I still have overall say over whether it's turned on or off. Relatively speaking, then, I have more control over my airbag than I do my DVD player, which has no reason at all for failing to let me turn off an unnecessary feature that allows me to enhance the use of hardware that I have purchased.

Would people tolerate a TV that would lock you into a single channel when you had selected to watch a movie and not allow you to change channel or switch off until you had sat through ten minutes of trailers? (The strange thing is that such TV's probably exist or are at least feasible for being manufactured today!) Would people tolerate vacuum cleaners that refused to suck until they had noticed your carpet was dirty, or because you used a competitors dust-bag? (Again, another likely occurence if the current state of inkjet printing is anything to go by) Would people tolerate telephones that auto-answered calls from marketers who had paid a fee to the telephone company and put them straight onto speakerphone?

One of the most popular features on telephones in the UK is a "Do Not Call" list for marketers, with severe penalties for companies that do not take account of it. My telephone has Caller ID functionality - I even get to control WHO I answer the phone to. My front door has a CCTV system - I get to control WHO I answer the door to (and leave marketers standing there in the cold and the wet). My oven and microwave do what I tell them to (within certain set parameters to ensure safety) and don't try to override me.

Most control-restrictions boil down to copyright control and advertising. Now copyright control is something that no form of protection has ever or will ever stop. Cinemas and DVD manufacturers are usually the source of any leak of pre-release films, professional copyright infringers have the knowledge, equipment and ability to bypass anything that the manufacturer may put in their way and those infringing copyright will not be stopped by such petty restrictions. Laws like the DMCA and its international equivalents fail to take account of one point - if someone is willing to break the law by pirating a movie, they will not think twice about breaking a law that prevents them from buying a device or using the "analogue hole" (the fact that if I can view it in any way, there will be ways to record that viewing, even if they are as primitive as using a camcorder to record the image on-screen) to copy the movie in the first place.

Additionally, it's impossible for a device alone to determine whether a copy process is infringing copyright - I am allowed to copy my DVD's for backup purposes in many countries. I may make one copy but then if the original disc breaks or is lost, the backup copy that saved me will also need to be copied, or I will have to obtain a second copy from somewhere. No chip in the world can currently decide accurately (or even just "well enough") whether or not I'm infringing copyright in doing either of those. In fact, in many cases, such backups are necessary. If you have ever let children loose with DVD's you will know that the discs scratch easily and many companies even refuse to provide copies of the DVD if this accidental damage happens to one you own (in the days of the ZX Spectrum, almost every company that distributed tapes would replace them free-of-charge if they stopped working).

The device has no way of knowing whether or not I am copying a DVD that I personally own or one I've rented or one I've filmed myself - in many cases this is the way that most modern copyright control systems end up being bypassed - any copyright flags or restrictions are removed to make the device "believe" it's genuine. Proof of purchase should be all I need to ensure that I can copy the disc and/or view copies of the disc. Even in court I'm innocent until proven guilty so it would be up to the device to prove, legally, that what I was doing was illegal. It couldn't. Ever.

That leaves advertising. Advertisting is not the only way to make money, or make your product known. Obtrusive advertising is a guaranteed way to lose money and make your products and even your own company infamous. If I wish to buy a product then I will research it. Not only will I not take any notice of brand-names or memories of previous advertisements of a company's products but I will absolutely blacklist a particular product, manufacturer or even entire line of products (such as DVD's or Blu-Ray or similar) based purely on how much control they expect to have over me. When I'm buying a product, the information available to me is what I base my decision on and not what advertisements I've seen in the past. When I go to the cinema, I base my decision on what is showing, not on every trailer I've seen since I last went to the movies and the ones before that. Only at the point of purchase do I NEED to know about any product.

As the years go by and copyright-protection gets ever more restrictive and more and more control is taken out of my hands, my mental blacklist grows. HDTV with it's amazing copy-protection? Blacklisted. (and my TV is five feet from my sofa and I don't see either how it will improve my perception of the TV image at that distance or why it's worth paying out for a mere resolution enhancement) Operating systems that want to dictate what audio/video streams I can play, where I can output them to or how many copies I can have installed? Blacklisted. (although I would like to point out that every OS I've ever installed has been properly licensed.) Cars that would only accept parts produced by their original manufacturer? Blacklisted.

And when enough of these blacklisted items are from a particular manufacturer? That manufacturer is blacklisted in its entirety.

The future, with its gadgets and gizmo's which are "vital" for modern life, holds immense interest for myself. As gadgets get more complicated, there has to be a certain amount of internal abstraction. We don't need to know how or why they work, just so long as we control what they do. My PC is the most complex appliance in my house and yet I exercise full control over what it does. My VCR has immensely complex integrated circuitry, although nowhere near that of my PC, that I don't need to understand to be able to use it and still it doesn't remove my control of its operations. Manufacturers would have you believe that "you can't" change what things do, because they are such complex machines. However, we have to keep in mind what motives they have for removing any controls we have - for instance, DVD region control and UOP's. There is no technical, functional or legal reasons (except for enforced compliance with what the manufacturers believe we should be paying for their DVD's in our particular country, whether or not we are legally able to import cheaper versions of the same DVD) why these two features should even exist and they can only interfere with our use of a product that we pay for in the first place and continue to fund by buying DVD's.

Are DVD's a product or a service? Is legally-bought downloaded music a product or a service? Is the computer hardware or software I buy a product or a service? If I stop paying for subscriptions and upgrades, or stop watching the advertisements, should things that I've "bought" be taken away from me? Everyone who sells these things wants them to be a service - they can charge more in the long run and revoke your use if ever you change your mind about paying for them. Everyone who buys these things wants them to be a product - they can control them and do what they want with them inside the confines of their law-abiding homes. Somewhere along the way, someone's going to have to come up with a reasonable compromise.

Slackware 11.0 RC-1

2006-08-15T15:02:00.000+01:00

The next version of my favourite Linux distribution is on the verge of being released. Yes, Slackware has it's first 11.0 release candidate.

Normally, I don't chase the very latest versions of software until someone's tested them for me beforehand - for instance, my latest foray into the world of Opera version 9.0 was a bit dismal... most of the computers I installed it on had no problems at all but at least two showed severe random crashes that I could not track down for weeks. The funny thing was that both problematic installs were on different hardware and yet on two very-similar machines one version of Opera 9.0 worked flawlessly but another didn't.

Anyway, unusually for me, I've been closely tracking Slackware 11.0 since the last stable release, 10.2, which is currently powering my main desktop and a number of my servers and hobby machines (I always track updates to stable versions so my software is never vulnerable but I rarely use "beta" software of any kind). I've actually got an up-to-date mirror of the bleeding-edge -current version of Slackware (that will become 11.0) which I update every time the Changelog changes. I'm hoping in this way to not have to suffer traffic-lock when 11.0 is finally released - hopefully at worst a tiny update of a handful of packages will be all that's needed to create my own DVD-R instead of having to fight thousands of people trying to download all 4Gb of software swamping every mirror with traffic.

I have even gone to the effort of a test install of the RC-1 version on a seperate partition. Linux, and Slackware in particular, demonstrated its flexibility and user-focus once again - by booting from a Slackware DVD, I was able to install the full install to a blank partition without doing any more than a very basic check of the partition name (which I am always very careful to double-check by mounting the partition in question - never take partitioning or formatting of anything on a PC full of data lightly). Once it was installed to my spare partition, I was able to copy the kernel and modules directory from my "stable" partition to the new "current" partition and, with a little LILO magic, boot into the new version of Slackware without touching my previous installation in any way, but with the very latest 2.6 stable kernel and all the software of 11.0.

I have to say that it's not spectacular but it's not spectacular because it WORKS. It just does what you tell it. You boot it on your PC, it detects all your gear, you set a few options and you have a full desktop. You port your old settings and files over and everything just works again.

The simple fact was that, in under three minutes of the installation completing, I was in a fully kitted out Linux desktop with drivers for all my hardware without having to compile a single package - installation consisted of nothing more than an automated decompression of the packages to the partition in question and minor copies or edits of my previous configuration files (such as re-doing alsamixer settings, configuring X etc.). My old software worked (at worst requiring a recompile against latest headers), my settings transferred and my computer didn't crash or have to reboot seven zillion times.

New in Slackware 11.0 RC-1:

* Updated kernels (although the default still looks set to be a 2.4 kernel)

Obviously, although I would say that the kernel is the one thing not worth waiting for a Slackware package to come out for - just install the latest stable of 2.4 or 2.6 depending on your tastes... Slackware supports either seamlessly without needing any special setup (although you may find it convenient to stick with one of the two for compiling anything that reads from kernel headers). Don't forget that Slackware always comes with a .config for it's kernel that's fully modularised and ideal for "make oldconfig" when a new kernel comes out that you need to compile.

* Lots of init script fixes and features

* Updated hotplug / udev support

* X.org 6.9.0

* KDE 3.5.4

No more Gnome in Slackware unless you get it from somewhere else. Not a bad thing for me, given that Gnome always reminds me of the old Borland dialogs in Windows - it always looked clunky and out of place. You can still get Gnome for Slackware from many places but it was removed in the previous version because of an apparently horrid compilation rigmarole.

* Updated versions of just about everything else (Samba, Apache, MySQL, Java etc.)

The changes aren't massive - it's not even as if the previous version has software which is currently vulnerable (despite what some checkers may tell you if they only go by software version number rather than whether they've actually been patched!). The software isn't the very latest (but it is almost guaranteed to be the best tradeoff between features, security and code stability) but it's clean, it's quick, it's simple, it works and it's been designed to run on as many computers as possible by default.

Although not designed as a desktop distribution, it's easily subverted to that purpose by installing the right "extra" software but the fact is that you know what you are getting - a stable, safe system that works and is flexible.

I get to choose and keep my own firewall package - one I've grown to love and have integrated lots of my other scripts into, I get to keep my choice of kernels and even whether to go 2.4 or 2.6. Every piece of software has been updated but all my old settings port over easily (at worst requiring a diff of some sort). Every piece of hardware has modules ready-prepared for it so there's no need to keep recompiling to get support. The kernel is bog-standard kernel.org fare, so there's no vendor patches or compatibility problems to worry about. Everything is controlled by human-readable scripts, which upgrade cleanly over previous versions.

I'm planning on building some kind of workhorse headless server and it looks like Slackware 11.0 is going to be my choice, given it's proximity to release and its easy flexibility to be installed without X or other useless software. This server will be performing lots of tasks which I'm hoping to come to rely on - CCTV monitoring and other household security tasks, intranet web serving, firewall, NAT, printer server, transparent HTTP proxy, automated network antivirus scanner, email scanner proxy, Caller ID announcer, wireless gateway, network boot server and all manner of other custom projects. It will use relatively modern hardware, will require stability (as it will be expected to be running all day long), will not need any sort of graphics capability and have to be secure against attack. I don't want to have to compile anything from scratch or find out that I've forgotten package X, so a full install and then prune will be in order.

Slackware's reputation means that I'm quite happy to have waited nearly a year for this release - I haven't had a vulnerable system in that time due to strictly-monitored security fixes for the -stable version, I haven't had to fight with half-new features in things like udev and hotplug which would have caused me a lot of trouble and I'm going to a system that's just as stable albeit further updated.

A fascination with recompiling kernels

2006-05-09T10:16:00.000+01:00

Okay, another trend for you:

Why do people who use Linux INSIST on recompiling their kernels over and over again? I'm a regular on several Linux forums and I see a lot of questions being asked that are along the lines of "I've recompiled my kernel and now something doesn't work". They are usually not talking about upgrading to a new kernel but actually just recompiling their already-working kernel but with different options (usually to change everything that is modularised to be incorporated into a single, fat kernel).

Now there are several very good reasons for customising a kernel in such a way. Firstly, if you are using PXE boots, USB keys, bootable CD's, (going back a bit now) single-floppy distributions or something else that has filesystem limitations (whether they are direct technicial limitations or, with PXE for instance, something like bandwidth) then having a single-file, small customised kernel is ideal. However, most people who are asking these types of questions are home desktop or laptop users.

When you have Gigabytes of space in which to store hundreds of unnecessary modules, recompiling the kernel ultimately costs you lots of time. Time in preparation, configuration, compilation, testing and, ultimately, problems brought about by missing a vital checkbox when configuring the new kernel.

One oft-touted advantage is speed. Now recompiling a kernel to target a different architecture may or may not give you a speed advantage (there are benchmarks which suggest that there isn't much difference at all between any of the x86 architectures). For general use, you wouldn't even notice the difference. The current theory is that compiling a kernel with only the modules you require built-in somehow makes it faster.

I can't see this myself. A module is ONLY ever loaded when you (or your hotplug programs) tell it to load. Otherwise, it's not even in RAM, let alone taking any of your CPU time. The time it takes to load a module is miniscule and rare... maybe ten, twenty times, each time your computer is booted, if that? A few milliseconds each time once properly cached? And that's usually done on bootup where things can hang around forever detecting hardware. After that, it's only ever done when you actually insert or remove some piece of hardware (and even then it may decide to stick around or be already loaded for something else). Compared to most other optimisations that are possible, that's an abysmally small one. Putting this stuff into a megalithic kernel every time doesn't save you anything in terms of processor time.

However, having a module PERMANENTLY in RAM even when you are not using it does not seem an efficient use of such a vital resource. By loading modules only on demand, surely you are saving enough RAM to cache anything you may want to load off of your disk (e.g. modules) and thereby increasing the system speed overall.

Additionally, having to hand-configure a kernel is an enormously time-consuming task, especially for the unskilled who may omit several vital options. And when you next purchase hardware, you're going to have to do it all over again. When you insert a USB device, the right options better be ticked or you won't see ANYTHING. You borrow your friend's USB hub, which uses slightly different modules, and you'll have to recompile the ENTIRE kernel all over again. You insert a USB device that uses a slightly different module to anything you've got in your kernel and, guess what, you have to recompile.

Why not just use a modularised, standard kernel that has modules for anything you could EVER use (but which are only lounging in a few tens of megabytes of disk), save yourself several hours of configuration, compilation, testing and frustration, sacrificing any miniscule optimisations that are negligible and unnoticeable for the sake of giving you more time actually USING the machine?

There are a lot of people on these forums that are complaining that something didn't work and their eventual solution is to recompile the kernel to incorporate some module that they didn't have to foresight to include. Add up the time taken to discover the problem, the time to diagnose it (which must be non-trivial if they have to post on a forum for help with it), the time to recompile the kernel and reinstall it in the right locations, to reboot and then test.

Is anyone seriously telling me that that is going to be LESS time than the actual productive time you lose you would lose by having to load a single module automatically from disk from their previous standard kernel configuration? I use hardware which, on the whole, is considered obsolete (it's cheaper and it does everything I need it to) and yet I still don't notice any significant improvement by not using modules.

Kernels take on the order of minutes to compile on modern machines but the human element is a lot of wasted time in doing so correctly, installing, rebooting etc.

Also, when troubleshooting omitting problematic modules can be necessary. When modularised this is simply a question of blacklisting them in the right software, editing a simple script to prevent insmod or modprobe from loading them or even just changing the permissions on the module file itself. Doing so on an all-in-one kernel involves, yes, yet another compile, install, reboot, etc...

I use Slackware which comes with the kernel config file used to compile the main Slackware kernel. It incorporates a lot of junk, most of it as modules, but it works on virtually ANY x86 machine (even 64-bit etc.) with little to no performance loss. I gladly sacrifice 50Mb of hard space to be able to buy/borrow ANY Linux-compatible hardware, stick it in the machine (which doesn't always involve shutting it down if you include USB, Firewire, PCMCIA etc.) and it will work without me having to TOUCH the software side of things.

People lend me their USB keys, their portable CDRW's, their scanners, their printers etc. so that I can diagnose them. So long as my software is up-to-date (which takes two commands at most), I know that anything Linux-compatible will run without me having to play with my machine or reboot it. The time I save in not recompiling the kernel each time more than makes up for any imaginary performance saving.

And when a new kernel comes out, the same config STILL works (make oldconfig). I get asked if I want to include X or Y in the new kernel and I always select the best option - the one that lets me use it if ever I need to (though I don't personally think I will have a Gigabit card, etc. in my personal machine any time soon) but doesn't allow it to get in my way - build as module.

Now let's pretend that my motherboard blows up tomorrow. So what? I move the hard drive to a spare machine, it boots up just the same but takes account of the new hardware without panicking, hanging, not having the modules required, or requiring me to play about with kernels and LILO/GRUB just to get the damn thing to boot up.

It's like defragging - there is a purpose to it but for most tasks the trade-off just doesn't add up. Spend three hours defragging to save milliseconds of disk acces time over the course of a month or so until the filesystem fragments again? There are scenarios where it may well be worth it, or have been worth it in the past, but most of the time you're just sitting a screen staring when you could be getting something done.

Kororaa Xgl Live CD

2006-03-17T16:55:00.000Z

I've been reading about it everywhere and overall dismissing it so I thought I may as well have a look anyway. Yes, Novell have released this "next generation" X Server which combines OpenGL and hardware acceleration into a bog-standard X desktop.

Newsforge have an article about it and they're not the only ones.

Firstly, the use of a LiveCD for this is part of what they were designed for - tests of software you wouldn't want to even try and install but that someone's done the hard work for you, software that may well screw things up for you so you can test without having to install it.

Unfortunately, the demo AVI linked to in the above article shows you all it can do. And I have to say, I don't see what all the fuss is about. It's a minor point that all this is only supported on certain chipsets, almost all of which rely on a third-party binary-only driver (don't get me started again). If everyone in the world wants this, you'd better start complaining to nVidia and ATI now to open-source those drivers because otherwise it ain't gonna happen.

If a distribution (like, say, Suse) wants everyone to start running these sorts of desktops, there'd better be support and backing from someone who's going to keep this running once everyone in the world's got XGL... is that support going to come from Novell - I bet not, or at least without a hefty price. Bang goes advantage one of open-source operating systems. And what happens when nVidia change their support for OpenGL? Bye Bye Window Manager.

Anyway, skirting that minor issue - XGL looks very pretty. I've seen better effects in a few PC games menus, though. It's pretty but gimmicky. Having video overlaid over OpenGL graphics on a desktop that can spin around - very nifty. Now what could I use it for?

I've been able to have video play in the corner of my desktop for a while - it's called having a TV-card or a media player. My computer is also capable of transparencies (a feature which, incidentally, I turn off no matter what the operating system as there is very little legitimate use of it).

But personally, if I want something showing on the screen, it's because I'm watching it. If I don't, I don't want to have to squint to distinguish between different windows, or be distracted by a moving image somewhere else. Whether I'm working or just browsing, the normal use of my desktop ranges between one app full-screen and the rest minimized to another app full-screen and the rest minimized.

Before the advent of digital TV, I used to have a small window overlaying the bottom-right of my screen, in a position in which it did not obscure my use of scrollbars in my most common apps. However, I cannot remember a single occasion that I paid it any atttention like that. Occasionally, I would be waiting for a program to come on and would full-screen the TV when it did but there was nothing I did that would be aided by transparencies, hardware accelerated or not.

The only task I do where the apps aren't used full screen is if I'm doing janitorial work on my files. I would have several windows open on the filesystem, and drag-drop between them. Overlaps were eliminated by the simple precept of moving windows about. Granted, having a shortcut key to arrange/zoom windows would help but that's nothing that cannot be done in standard X.

Additionally, moving stuff between desktops is hardly a chore and a rare event - I use virtual desktops for different tasks. If something's not on the virtual desktop I'm using, it because it gets in the way or is part of a completely different task, e.g. games on a work desktop.

I frequently have my virtual desktops arranged by task, Internet, Work, Entertainment. Hence, the organisation of virtual desktops actually negates any need for transparency or some fancy 3D - I don't need to overlap windows if I can just throw them onto another, more relevant desktop. It also means that I can't be distracted while I'm working by stuff that shouldn't be on the screen at all.

Novell's XGL demo also demonstrates lot of other useless cruft - we won't even mention the bouncy windows or "snap to"... this stuff was done years ago and ignored because it's pure gimmick that's already working. It's the sort of stuff that appears in menus to games. Even there, it's only ever fun for the first minute and then you never dabble with it again.

Switching between virtual desktops as a cube - good metaphor but you could just as easily scrap it as an actual cube and just have four "sides" and still get work done without having to remember horrible keyboard combinations or have lots of OpenGL technology in place to be able to switch between desktops. It provides no advantage over much simpler, much less demanding methods.

Now it seems that the best bit of the XGL demos is, in fact, the hardware acceleration. Now that's a feature I won't knock unless it's for the fact that it only works on binary drivers at the moment. If they can resolve the driver issues and have hardware accelerated graphics on a desktop, maybe we can finally catch up the 90's with the rest of the world's operating systems! (And that's not a bash at Linux/X. It's a bash at the driver manufacturers who are holding us in that horrible position).

Novell, it seems, has spent two years making a stretchy GUI. To quote their website:

"Novell is announcing its contribution of the Xgl graphics subsystem and the 'Compiz ' compositing manager to the X.org project. These enhancements open up a whole world of hardware acceleration, fancy animation, separating hardware resolution from software resolution, and more. As a result, Linux desktops will become more usable, end-user productivity will increase, and Linux is firmly positioned at the forefront of client computing technology."

I'm sure that bouncy windows and video over a cube are so going to increase our productivity and make X more useable. I'm sure having to manually install a binary-only driver is phenomenally easy for your average potential Linux end-user that they'll even be able to do it in a bouncy window while watching Harry Potter playing over the top of Ice Age 2. I'm sure that those two years of adding to the configuration options of X.org.conf is going to just have us all blatting out our code twice as fast.

"Under the leadership of Novell's David Reveman, Novell has sponsored and led the development of this powerful new graphics subsystem for Linux since late 2004. Xgl is the X server architecture layered on top of OpenGL and takes advantage of available accelerated 3D rendering hardware. It is designed to integrate well with the composite extension and performs best when a compositing manager is running. 'Compiz' is the new OpenGL compositing manager from Novell and is the framework that enables the development of graphical plug-ins."

Let's get this straight - hardware acceleration is a good thing. I like to be able to run 3D apps as fast as possible. Rarely, however, does a working desktop require 3D acceleration. You might be a 3D designer all day long but your desktop really doesn't need it. If you have power to spare, a few bouncy windows would probably look lovely but otherwise all they've created is an add-on to window transparency, a feature most people have turned off or don't even notice that they can use.

Novell should have put their efforts into something a bit more practical. An OpenGL compositing manager, I'm sure, has one, maybe two uses where it's absolutely indispensible. What would have been much more use would have been simplified X configuration options, maybe even on-the-fly configuration for most options, tighter integration with layers like HAL or even a single damn hardware accelerated open-source driver.

XGL is a gimmick. It may convince some eight-year-old sap somewhere that "Linux is better than Windows" but that's about it. Just wait until he's thrown into the world of having to seperately compile a kernel-specific driver every time he wants to try out the next kernel assuming, that is, that nVidia has supported it at all yet. XGL brings nothing new to the table, no work that people couldn't get done before can now get done. No time is saved, no money is saved, no problem has been solved. All it does is make my computer even damn slower just to show me a file listing.

Windows Vista

2006-02-28T23:50:00.000Z

Windows Vista (or whatever it will change name to seven times before they ever release the thing) is approaching and a lot of people are focusing on what it can do and what it can't do. What they don't seem to take account of is the history. People who complain that MS has given them a bad run in the past are told they are pessimists and that this is the time when everything will be perfect. I've heard that at least five times now, so will I be upgrading to Vista?

Would you buy again from a butcher that, five times, has sold you a bit of what he assures you is "the best beef in the world" only to discover that when you get home it tastes like old boots?

Not a chance.

DOS I adored. It worked. It was powerful. It was simple. It was fast. It did the job.

Windows 3.1 I gladly bought into and began to love. It was small, simple, worked and worked well. It was easy to use and just pretty enough without needing too much from the hardware (386 with 2Mb RAM).

Windows '95 was then thrust onto me by peer pressure; it was okay but nothing special. There was a lot of frills on an OS that was basically a 32-bit version of Windows 3.1. It was also very buggy. '95 OSR2 didn't help matters at all.

Windows '98 was pushed into my hands because '95 was such a disastrous attempt at trying to push a '98-style OS out too early. It improved next-to-nothing. '98SE came out and you were asked to pay again for it. Yeah. That's going to happen. It fixed a few of the problems and introduced a few new features, but nothing that anybody actually NEEDED at the time.

Windows ME I looked at and quickly became a disaster (things like major components of Windows supporting '98, 2000, XP but not ME... .Net anyone?). It was '98SE-with-knobs-on and didn't manage to do anything particularly exciting.

So from '95 to ME, there were very few, very small improvments in an OS that incorporated at least three major paid-for upgrades (as in major release versions, not just updates) over and above the base price. It was then that I told myself that I would not upgrade any further without good reason. My computers worked, it had cost an awful lot of money to get that far and I hadn't seen much improvement in my actual productivity over previous versions. They all ran the same software, at the same speed, with the same features, with little or no improvements in the areas that mattered - stability, compatibility with older software and hardware support.

Okay, '98SE+ incorporated USB mass storage properly and a nicer driver model but in essence it also killed ISA cards stone dead without giving people a say in the matter... is it really that hard to support a standard that was still in use at the time, had stabilised and standardised itself on things like hardware autodetection, and still works to this day in the Linux kernel which has a much stricter requirement on what stays in the kernel? To be in Linux, the hardware has to be stuff that's used, has support from several programmers willing to change their code constantly, work in the kernel at all times and get updated in line with everything else, from people who are not getting paid to do that job. Anyway...

I stuck to '98SE and I spent most of those years chasing Windows Updates, free antivirus, utilities to manage the computers, anti-spyware, etc.etc.etc. Windows 2000 I skipped entirely - it removed support for a lot of my then-current hardware and only provided a small stability bonus. XP was a "necessity" to run one single game that I wanted on one single machine and has also turned into more hassle than it's worth. XP I see as basically a games console - a bloody complicated and annoying one at that.

I didn't pay for XP, it came with a second-hand computer I was given, one thing I was glad of. XP offered me nothing over 98 except more restrictions, more problems, and much less system transparency - even the filesystem was relatively unreadable outside of XP without expensive utilities (though that's not so much of a problem now but still it's hard to correctly write to NTFS without buggering something up).

Despite several methods of recovery in case of system problems (Recovery Console, Safe Mode, System Restore), it was still perfectly possible to total a machine by installing an official update that would take more hours to fix than the computer was worth. Suddenly, I needed Ghost around constantly whereas before I'd only ever re-installed Windows '98SE from scratch once (and I later found out how I could have fixed that too). It wasn't something the average Windows '98SE user could do but I brought that OS back from numerous permanent blue-screens, booting problems etc. without having to worry that I wouldn't get the system back up and running.

There isn't going to be another chance for MS. This isn't blind MS-bashing, I've just had enough. There's posts on this blog telling you how I kept my own personal '98SE machine in tip-top condition from it's release to mid-2005 and even recommending that people stay with it.

I've always noticed it and never give it a second thought but now I can see the trend in MS OS's:

- More new features that I won't ever use and just get in my way. I end up turning half of them off within the first few days, the rest as time goes by and discover they are causing me problems. I end up setting half the settings to Classic or some other sort of compatibility or failsafe mode because that's how I liked it. Control Panel was prime candidate in XP, along with Autorun. Also disabling of things like power-save settings and screen blanking.

- More restrictions, barriers and brick walls, each of which stops me doing something I WANT to do and can CURRENTLY do. Connection limits, raw sockets, driver signing, not having to activate, the list goes on.

- More time and money, not just on the OS but its supporting programs to get it into a vaguely useable state. Anti-spyware, anti-virus, firewall (because the MS ones I won't trust to be any good from experience and may well be the next anti-competition case against MS), startup controls, Ghost (because, again from experience, the chances of any type of system restore working as it should are extremely minimal). Again, the list could be endless.

- More integration with stuff I don't want (starting with IE and WMP). I don't want stuff connecting to the net unless I SAY so and unless it's ABSOLUTELY necessary (i.e. it's a web browser which has been asked to connect to a website by me personally, or an autoupdate that I'VE scheduled to autoupdate). I don't even LISTEN to music, and I certainly don't want rubbish trying to get album covers and other nonsense from the internet just because I'm testing a drive with an Audio CD. I don't want my browser to even be ABLE to execute code directly in the webpage, or choose a search engine without asking me what one I want to use.

- Nothing that I absolutely *need* when it comes to upgrade time. My computer does lots of stuff already. What can I do in Vista that's totally 100% impossible in 98SE, XP or Linux? If you discount hard-coded restrictions and programming laziness, nothing. Vista is not a quantum computer conversion - it still does the same old stuff the same old way.

- Missing or just starting to introduce a lot of obvious stuff that SHOULD already be in the OS (**why** do I need a completely seperate, non-MS utility to tell me everything that's loading at Windows startup? Why have I gone from Windows 3.1 to Windows XP without MS incorporating such a simple, useful utility? Why can I not also click a button that LOCKS anything else from inserting itself into startup and kill half the spyware/viruses in one fell swoop? And yet they are bundling rubbish like media players and internet browsers that I DON'T want at all and have never even used)

- Still playing catchup to other systems. A database of your files that updates in the background and you can use to locate your files quickly? Got it, except my one doesn't slow the system down when I'm using it like Find Fast and the other MS "inventions" do. Admittedly MS may well be ahead in terms of hardware driver support but considering my Linux machine doesn't NEED half that new hardware and won't do until it's properly supported under Linux anyway... where's the incentive?

I quit Windows about a year ago hopefully forever. I was tired of my computer not doing what I tell it to. This is my biggest, absolute killer for not running Windows... if I say shutdown, you will shutdown, if I say delete that file, just delete the damn thing... I'm not an idiot, I know what I'm doing. The chances are that if I force a shutdown, there's a reason for it. It may not be an important one - I may be rushing to go out for the evening and want to make sure it's off - but that's not for you to decide. Unless I'm going to do permanent, irreparable damage just do what I say, and even then just make sure I'm AWARE of that. My OS of choice will *not* argue or crash or wait for every program on earth to voluntarily allow me to shutdown unless I ask it to.

I'm tired of having to be at the forefront of technology just to browse a simple web-page at a decent speed. I'm tired of "limitations" like XP Home's connection limits, raw socket limitations etc. when there is no technical or practical reason why they have to exist. If my OS is capable of it, it should offer it. It should not say "I COULD but... I'm not going to let you until you pay me money". It's like running a shareware operating system, except I've already paid for it.

I've worked as the only support for many years for a few hundred XP, 2000, 2003 and older machines and yet have only ever used XP on one laptop personally (my "games" machine) and on my girlfriend's computer (it came supplied with the computer and it was easier just to leave it on there for her... she had to "learn" Windows 2 years ago so learning a Linux desktop isn't a big problem at all... it's just easier for when she wants to play The Sims and other rubbish). Windows is "easy" until you need to maintain the thing and then it becomes a nightmare. My choice of OS at home reflects just how good Windows is - I work with Windows all day long, even recommend Windows systems and yet I won't touch it with a bargepole at home any more. On another note, the more broken Windows is, the more money I make because I have to then be paid by numerous schools to fix it for them. And I get paid by the hour. ;-)

I've lost count of the number of computers I've brought "back from the dead" by removing viruses, spyware, too many startups running, etc. When a user can sit at a new, fully-patched, antivirus-ed, antispyware-d machine and, without intent and within a matter of minutes, infect the machine so that it barely loads up in half-an-hour, taking hours to fix, is when I give up on that machine. What a user does SHOULD NOT affect the machine as a whole, only that user... even as a "limited" account on Windows you can wreak havoc.

Windows has an after-the-event method of fixing problems - once the virus is on there, and lots of people have also got it, some company might send out an update that may or may not catch all variants and won't help control the damage the virus has caused. Vista even includes special integration for antivirus apps. Do people not realise how ironic it is that the OS that "invented" the problems with modern-day viruses and spyware even has a special place that you can install anti-virus into so that it will integrate nicely? It's like having a car that comes with an easily accesible tool specially designed with the sole purpose of putting the wheels back on should they fall off on the motorway. So reassuring.

(Yes, DOS had viruses. DOS was back in the era of one-user full-admin home computers without sharing of disks or internet access and was a design disaster from the start... at least it bloody worked though. Sensible people had worked out in the 70's that that was just a stupid idea for multiple-users or internet-facing machines. Windows caught up with them in Windows XP/2003.)

There is actually a page on a website belonging to a Linux security enchancement package called SysMask that actually allows you to upload ANY bash, C or perl script. When you do, it compiles it, runs it and shows you the output! It will voluntarily and automatically run ANY code that ANYONE asks of it as an ordinary user because it's so sure of it's security, just to prove how good it is. This is on the same bloody server that runs their own website where you can download this code for free. It's never been taken down.

Like this site, I want before-the-event fixing - even IF someone runs some dangerous software deliberately, researching the latest holes, it can't affect the machine as a whole, can't destroy other people's files, can't put me in a state where I have to hope I have a recent image/backup. I don't trust Vista to do this... Windows 2000 was supposed to stop this. As was XP. As was 2003. Backups are for restoring files after unavoidable hardware damage - nothing else.

Now, on Linux, the damn computer actually bloody does what I ask of it. I don't have to be too careful about checking licensing for the software I install because it's *all* GPL or free (yes, I still check that it's GPL or otherwise free, though)... I'm not distributing my changes so it's all free for however many computers I want. No more license-counting, no more fighting activation systems that think they know better, no more serial codes, no more.

I used to spend HOURS on Windows hunting down decent freeware to get stuff done without having to shell out even more money but now I don't have to fill every system I own to the hilt with third-party freeware just to get the damn thing into a usable, secure state. It actually comes with everything I need, by default, installed securely.

At aboslute worst, an automated update command (one that WORKS, does it when it's convenient FOR ME, doesn't force updates that are dangerous and doesn't kill one machine or another on a regular basis) keeps me up to date. Rollback? How about a complete uninstallable plain TAR archive of every update I've ever installed, along with a copy of every single package ever installed on the machine? Any package I want, I install. I don't have to con the software into thinking it's NOT installing over a later version, not already been uninstalled, requiring the original setup disk etc.

It's also quite difficult (without doing something incredibly stupid and deliberate while logged in as root) to ruin the actual software on the machine. Windows relies on so much being intact to even boot, Linux just wants any half-recent kernel boot disk to get to a fully functioning command line and repair system (including uninstalling/reinstalling/upgrading/downgrading any single software package individually on the entire machine).

I get to choose what software runs without some arcane registry entry loading up something I'm not aware of, and am not even sure if I need it at all. Same for "services". Additionally, if I want a ten-second boot, I can have one. If I want flashy graphics, I can have them. If I WANT to boot into a command-line only environment, I can. I have that choice available. And you know what? From that environment I can control every single setting that I could control within the GUI if I wanted to. For every user. Without learning hexadecimal or what arcane GUID in the registry it's stored it under.

I can actually TRUST linux, from it's filesystems to it's hardware support to the individual software components to the firewall. I know that someone isn't going to say "well... we COULD let you have five users connected to your shares BUT we're not going to LET you". If something said that, the source code wouldn't know what had hit it after I'd put it back the way **I** want it. You're *my* computer, you can only do what you are told to do and **I** am the one in ultimate control of every single piece of software on my machine. If that means editing source, so be it. If that means I want to voluntarily install some binary (and therefore risk incompatibility, forced upgrades and undiagnosable problems) to get my job done, that's fine.

I don't have to feel like a criminal because I want to use one OS on two computers. I don't have to check in with mothership every time my motherboard changes (which is quite often because the only thing that's constant about my machine is it's data - the drives change, the hardware changes all the time; I've still got data from my DOS days on my current hard drives).

There's very little hardware I own that Linux doesn't support, and all of that is non-essential and easily replaceable (one USB IrDA adaptor, one 56k Winmodem out of eight). I don't need to have drivers on hand for each and every part of it, or a checklist of which manufacturers bothered to pay MS to get their drivers certified and which didn't. I don't need to worry about the drivers interfering or only being able to run them with the most horribly annoying pieces of GUI software known to man (HP printer drivers, some of the arcane school-specific hardware I have etc).

If I get a crash, there is something real, something productive that **I** can do about it. Someone, somewhere will be vaguely interested in finding out why my machine crashed and, hopefully, fixing it. There are constantly new free upgrades to try out, there are config files to play with, there is source to look through, there's one of the most complex debugging systems known to man sitting on my computer already waiting to find the exact spot that something crashed and why, there are many unique, discrete components that can be eliminated one at a time to diagnose and I can even single step individual changes to the kernel to find out which one caused my problem (git bisect's etc.).

I don't get (and could easily discover anyway) obscure problems like a certificate in a JAR file associated with a famous piece of UPS monitoring software expiring and thus killing the entire system without warning or a single error message, taking 100% CPU and stopping approximately 50% of programs from running at all.

Who knows, I may even be able to code a fix myself without having to wait a year for the manufacturer to even acknowledge my problem.

And at the end of the day, there's nothing I can't do on my machine that I ever wanted do on Windows. In fact, most of the tools I use now are so much more powerful it's saddening to think of the time that I've wasted trying to find Windows programs that could perform the same tasks. I **liked** batch files, I **wanted** to tweak every entry in my AUTOEXEC.BAT and CONFIG.SYS to get the most out of my very expensive hardware. I want to be able to choose and change between using my RAM for virtual storage, caching my drives when I organise all 500Gb of data on them, displaying a GUI so that I can get work done etc.

My hardware is, to put it bluntly, crap yet expensive (to me). A 1GHz serves all my needs but may well have cost me two years-worth of donated/disposed of hardware (which means several "free" jobs fixing other people's computers and a lot of effort and petrol), plus several hundred pounds of my hard-earned money plus the time and effort to get it working how I want it.

When £1000's of hardware is sitting there and telling ME that it won't do something because I haven't phoned Microsoft or haven't bought the right version, I find it diabolical that my most expensive appliance in the house is not controlled by me.

Windows 3.1 I bought into, 95/98 I used and tolerated for a LONG time, getting many useful hours out of it. By the time '98 was obsolete I'd fallen for MS's spiel far too many times and was getting tired of computers. An OS actually nearly put me, a computer fanatic, off of computers. I didn't believe in or buy 2000, or XP, or 2003 and I won't be doing the same for Vista.

I'll still have to use it, in work if nowhere else, but I'm hoping that I'm going to have made the right move here by moving away from cash-driven OS's to ones that are driven by a yearning for freedom, control, pride in their work and technical prowess. Not that it's got a new glass interface that looks cool.

Linux desktop update

2006-02-22T11:47:00.000Z

First off, the computer is still running fine. Problems encountered since last post - umm... none? I updated a load of software (in keeping with my usual habits), everything from K3B to PHP even though I hardly use some of them. K3B is my primary CD writing app so that obviously had to be updated, the rest were just for my peace of mind. I very nearly downgraded K3B by several revisions after Swaret found a "new" version on a Slackware mirror but I already had installed a much higher revision from LinuxPackages that Swaret didn't seem to pick up on. Fortunately, I was watching out though and have confirmation turned on for every package upgrade Swaret tries.

Even if I had gone wrong, a simple upgradepkg command would solve the problem. I keep a directory full of packages that are installed on the machine (from LinuxPackages, my own, elsewhere etc.) seperate from the official Slackware packages so that I can upgrade, revert or remove such software. This is again kept seperate from software which I've had to manually compile to install on the machine so that I can always find either original source code or a package for anything I find on the machine.

If you remember, I did a full Slackware installation and that's EVERYTHING. I've got things like LaTeX installed which I haven't used since my university days but seeing that even with everything installed Slackware is still smaller than an equivalent Windows partition, I haven't bothered to remove anything (it's not like they are running as a background service or anything and I keep them up to date anyway so it's not a security risk).

I've been doing a lot of converting/copying/writing Video DVD's just lately which means that I've had to hunt down a suitable program. In the end a few choice command-lines did pretty much everything I needed them to.

Generally, I need to be able to convert anything (DivX, RealMedia, WMV, ASF, Quicktime, etc.) to MPEG-1 or MPEG-2 for putting onto a VideoCD or DVD-R for playing in ordinary DVD players. I also sometimes needed to copy a DVD when I didn't have any DVD-R's so that meant MPEG-2 DVD to MPEG-1 VCD conversion. We're talking home movies and web clips here, so there was no subtitles, chapters, multiple audio tracks or menus to worry about, just straight film clips. I'm sending them to Kuwait for my girlfriend's dad so they have to work in any region DVD player, his laptop, his school's machines, etc. without worrying about extra software, codec compatibility, regions or anything else.

In the process, I spent days looking for a program that could write correctly-formed MPEG's onto a CD in Video CD format (something which was never that easy in Windows anyway as you needed to have stuff not only in the right MPEG format but also a strict filesystem layout) until I found out that, if you don't want menus or anything, K3B can do it for you. I'd been using it for months without even knowing it did that!

K3B handles writing to DVD just the same once the data is in the correct VOB etc. formats and I've got QDVDAuthor to do that for me.

I solved a tiny minor problem to do with the clipboard contents transferring between TightVNC remote sessions and remote Windows computers (which I needed quite badly since I've logged into the machine via VNC every day since I installed x11vnc). Installing autocutsel solved that problem instantly. I like the idea of having seperate selection and clipboard buffers on Linux/Unix but if you haven't been brought up on them, they don't get used properly. Autocutsel synchronises the two and lets you just have a "normal" clipboard.

I also got NTP time synchronisation working after a "doh!" moment when I realised it needed UDP port 123 inbound to be open to the servers I wanted to use, not just outbound. A few good servers and it's ticking along nicely.

I've rewritten all of my firewall scripts so that now I can open ports on demand (for stuff like bittorrent to help it go faster), forward them to my girlfriend's machine, etc. In the process I "homogenised" all the scripts so that they are used on startup, from my rc.firewall, from my portknock daemon and from the command line. This means that I only have to maintain one script for all actions, so opening the SSH port to my work IP on bootup is using the same script as when I portknock from somewhere else or if I need to open a port to external access for remote VNC connections so that I can fix people's PC's. I can use a remote portknock to close a port that I opened from the command line locally without worrying about whether the rules will be implemented in the right order, whether the correct rules will be removed, unintentional doubling-up of iptables rules etc.

Because of the VNC setup, I am able to sit on a remote machine, securely access my network, take over my girlfriends computer to help do the bulk of things like MPEG conversions (her computer is 3 times faster than anything I use as I don't generally need CPU speed), that computer reads from and saves it's results to a Samba share on a journalled filesystem on the linux machine (which has the most disk space and which I can also control simultaneously to put that same files onto a DVD or VCD when they have finished converting).

I've also got UltraVNC running under Wine so that I can accept UltraVNC SC connections to my machine. So if a school has a problem they can login and double-click an icon that I've left on some of their servers, which will initiate an encrypted reverse connection to my machine which will then take over their machine and let me fix whatever the problem is. When I'm done, I close the connection and the software their end returns control. The beauty is that with UltraVNC SC, it's a single executable on the remote end that does not need configuration or installation and cleans up after itself when I'm done, so it's the sort of thing that I can tell people to download on the spur of a moment and, if they have a broadband connection, can easily fix their machines without leaving the sofa.

Because UltraVNC is Windows-only and uses non-standard VNC extensions, I had to use the Windows client for it under Wine. I've already got Crossover Office but I was hearing interesting things coming out of the main Wine releases so I decided to install Wine too. It ended up being easier than I thought and they didn't interfere at all after a bit of PATH-juggling, so now if I type wine, I get wine but my icons for Word etc. still use Crossover Office for which I can get support. Hence, the UltraVNC icon now uses Wine while the supported Office apps use Crossover. (I did try Word in Wine and it seemed fine but I'd rather stick with something that I know works, is a supported configuration and has someone I've paid money to on the other end so that I can shout at them if it goes wrong).

The computer consistently achieves 40-50 days of uptime and would be permanently on barring hardware failure were it not for my insistence on playing about with scripts that load at login so that, when I do next have to reboot, I don't have to worry about whether I enabled x11vnc on startup or configured the firewall to let through SSH connections from my work IP. I also upgrade the kernel to the latest stable release whenever I can, which means LILO changes and reboots, so a reboot once a month or so is no big deal, especially seeing as it is ME deciding that it needs a reboot (I still can't believe the number of times a Windows machine has to reboot from initial purchase through to a working system with all your software).
[On a side note - I noted the other day that my print server achieved over 380 days uptime being used quite a lot EVERY SINGLE DAY by myself and my girlfriend. Considering the fact that the lights go dim and the UPS switches to battery about twice a year, that's quite impressive, and it's not even running through the UPS.]

I've configured stuff like SMART and motherboard sensor logging using lmsensors (a long time ago) and now have more peace of mind that I did with Windows as I can see the exact factors that affect the values - this is very useful for hard disk temperatures and fan speeds. I can actually see which components produce the heat, which are cooled if I open a side panel, which ones are more sensitive to CD-Writers spinning up etc. My case is crammed full of hardware and cables and this is quite vital as there is no room for proper airflow in the case and I can't personally afford to upgrade when this system already works well within safe parameters.

I already have a hardware temperature/fan monitor which is seperate from the motherboard ones so that it throws an absolute wobbler if a fan does not start when the CPU is turned on. This happens sometimes (the fan seems to have trouble on startup on occasion - about once every 20 or so boots) and the computer is actually quite happy without that fan spinning at all but it's much nicer for me to know that it's not and to power down again. The hardware monitor was cheap but works on a very simple system (thermistors and fan connectors connected to an external chip powered by a drive power cable) and doesn't rely on my ageing BIOS having to notice the problem (which it generally doesn't with fan speeds) to shutdown the machine.

That same hardware monitor will also beep like hell and shut the power off if the power supply goes over-temperature (I use a fanless power supply so this was just another piece of paranoia). Additionally I now have motherboard monitoring and SMART monitoring (including disk temperature) which gives me peace of mind, especially considering the age of most of my hardware.

If any major component of the computer overheats, goes overvoltage, stops working, I KNOW for sure that either Linux, the hardware monitors or the UPS will shut the computer down. This is very important to me given that this computer runs 24-7 in a household environment. I doubt that Windows would shut you down if your drives starting to fail or go over temperature unless you spent a lot of time and effort to get some software that did it for you.

SMART also runs self-tests on the drives overnight (when things like slocate also do their business and update the filesystem search indexes for me) and constantly updates me on every performance change that occurs (for some reason one drive flickers back and forth between two consecutive values for Seek Time Performance which I assume is just natural variation) so hopefully I would catch most serious drive problems early enough to replace and restore the drive.

My girlfriend (someone who didn't know what Windows was until she had to use it on her law course a few years ago) is quite capable of turning the machine on or logging into it, doing whatever she needs to in Opera (web, email, etc.) and logging off again. When her computer's down and she needs to enter results for work, the Linux computer is always there and just works for her.

I've got OpenOffice installed now too, as an office backup and also to use for the spreadsheet as I have a licensed copy of Word for the Linux machine but nothing else (yes, I actually have a hologrammed original MS copy of just Word 2000 on CD). Because I am now also using Portable OpenOffice.org on my USB key this is also for compatibility and to familiarise myself with it. Something that's quite funny is that OpenOffice.org spreadsheet program manages to handle the complex XLS spreadsheet I use for my invoicing with the same functionality and without any of the weird "out of resources" errors I get with Excel (despite following every advice known to man on combatting that error in Excel). It's not even THAT complex a spreadsheet, it's just got a lot of conditional formatting to highlight monies owed to me etc.

Wireless works, when I need it to, and I'm thinking of having it permanently on now that I'm sure of the firewalling. This would be primarily so that I can set up an old relic of a computer in our spare room to form SSH tunnels over the wireless to the main Linux machine so that guests can check email etc. without me having to run cables upstairs. The setup works, I've tested it, but I've just got to shrink the machine a bit as it's only a small spare room and a big chunky desktop case is over the top for a remote-access port. If I had unlimited funds, I'd get a mini-ITX computer up there and I'd also fit it with a WinTV card and a security camera so that it can feed the signal back to the other computer in the house that's running a security camera and motion detection software.

Still no show-stoppers. In fact, if anything, my lack of disk space is my greatest problem at the moment, mainly due to the fact that slackware only uses a single 10Gb partition so I've filled the rest up with junk just because it was convenient. Stuff like Gb's of DVD VOB's and source MPEG's that I've already converted and have elsewhere but just haven't got around to deleting yet.

My own piece of Linux "evangelism"

2006-01-14T13:25:00.000Z

It's no surprise that I like to sing the praises of Linux. I've been using it, in one form or another, since the day I discovered it's existence.

Increasingly, however, when I read articles about Linux I am constantly annoyed at people's frustration that it's not how they want it. They read the part that said that Linux is Open Source (I don't usually capitalise those last two words but I feel I should start doing it) and can be customised and therefore they expect it to automatically do whatever they want.

I don't know when my annoyance at the lack of understanding of this particular "mantra" started but it has been recently exacerbated by articles on binary kernel drivers among others. I regularly contribute to several forums on Linux and School IT in general and a lot of people just don't seem to "get" Linux at all.

The binary kernel drivers argument was one of the first arguments I've had online where I've been so annoyed at the lack of understanding that I've pursued the question but only after going away for a while to make sure that my reply was calm enough. The discussion centred on the fact that Linux does not allow a stable kernel interface for binary drivers.

Now, the entire legality of binary drivers within Linux is one which hasn't surfaced properly yet and I can see that one day someone is going to get some nasty jaws appear out of the water and take them by surprise because they've misread the GPL. Leaving that aside for one moment, binary drivers are the bane of any Linux supporter.

Binary drivers are those without source code, like almost every hardware driver that exists for Windows. Companies like nVidia release drivers for their hardware in this form to avoid losing their precious patents etc. to a bunch of people who have bought their hardware and just want to use it. Fair enough, they have to make a living and if part of that living involves never releasing source code, that's up to them.

As a case in point, there do exist Open Source drivers for nVidia cards but they just don't feature the 3D acceleration that the binary drivers do (so you can use nVidia cards in a Linux system, but they won't have the same speed when playing games. The average desktop, however, would run just the same). Therefore, in this case, the drivers for the nVidia cards are only used by people who have spare 3D cards lying around in Linux machines that they are using at least some of the time for gaming. Professional users of 3D would probably not be using the nVidia binaries or even Linux.

nVidia achieved this marvel of modern technology (running 3D applications on a 3D compatible system with an nVidia card, where someone has already wrote 99% of the other necessary supporting code for them) by using binary drivers, which plug into the kernel at certain points. They evaded most of the technical and (possibly) legal limitations of interfacing with various versions of the linux kernel by releasing an Open Source kernel "wrapper" which lets the binary driver load itself without caring about what kernel it's actually running under and without the driver having to be rewritten for every kernel change.

The argument I had on Slashdot centred on people releasing binary drivers for Linux. The general overview was that Linux people were hostile and unhelpful to people writing binary drivers, that the manufacturers constantly had to keep updating the drivers for various kernels and that a stable binary driver API would help matters.

Needless to say, my reply was less than assistive in getting support for binary drivers in an Open Source kernel ("First, I think you're missing the fact that, overall, Linux doesn't care that you can't put your binary-only drivers on it").

The argument centred on the fact that companies will usually only release drivers as binary modules because they spend so long developing and testing their drivers that they are wasting an extraordinary amount of money if they then throw all that work out for anyone to copy it.

They don't seem to take account that Linux is entirely built on years of work that people, including many large corporations such as IBM itself, routinely give away for "nothing". Do you really think that the trade secrets in your hardware design are so fantastic that a) nobody has thought of them or that b) nobody who has your binary drivers and hardware couldn't reverse engineer them, legally, anyway?

Many of the network card drivers in Linux, for example, are reverse-engineered or written from open specifications and then placed under the GPL or other Open Source license. I can't see where this could pose a problem for the hardware manufacturer as they are effectively getting "free" drivers written for them, at little or no expense, as well as having those drivers supported and maintained for the forseeable future (at least until the hardware is considered obsolete and possibly even after that).

To quote Donald Becker's page (although this quote was written at an indeterminable date): "Linux now supports almost every current-production PCI Fast and Gigabit Ethernet chip!".

So every manufacturer of Ethernet cards has effectively had Open Source drivers written for them and distributed worldwide for free. I don't see any network card companies complaining about the fact that pretty much any network card inserted into a Linux machine is detected and used without having to download and install any driver, binary or otherwise.

It may be that the patents and trade secrets covering a network card are far fewer or of less importance than those covering a 3D graphics acceleration card. However, given the number of patents on items like TCP offload engines and the like, it seems unlikely.

If you are writing drivers for hardware, surely you'd be glad that someone is willing to ask you for the specifications of the hardware so that they can write and maintain such a massive, difficult piece of code as a hardware driver for another platform that you probably will never be able to support as well as your main platform?

The main misunderstanding comes from the binary driver "API" idea, the vision of a single standard interface to ANY piece of hardware within a computer and all of the associated kernel functions that will NEVER change. That sounds almost easy, no?

The Linux kernel is never easy. It changes every single day of its life. Unlike the major desktop operating system of Microsoft Windows, Linux is updated almost every minute by someone, somewhere. When the Linux IDE code was considered obsolete, unmaintainable and unsustainable, it was rewritten from the ground up. When that effort failed to stabilise quickly enough, it was restarted again on a smaller scale.

When the scheduler started experiencing problems on systems with hundreds of CPU's, it was ripped out, modularised and put back in. When USB and Firewire were introduced, they were bolted on to the existing frameworks and then rewritten time and time again to obtain a set of code that was maintainable and extendible for new standards like USB2.

At each point, everything was redesigned, not just re-engineered. People went back to the drawing board and said "Why are we still doing things for hardware we no longer support?", "Why are we bodging CD-writing by making an IDE-SCSI hybrid module?", "Why can't we use the SCSI code we already have to support these new fangled USB mass storage devices as well?"

Each time, any stable ABI would have broken. Each time, a new version of the stable ABI would have had to been released. It's not a stable ABI if it keeps breaking. On the other hand, if someone had said "SCSI works this way and you must not change it" then many things would not have been possible or would have meant reinventing the wheel for them to be supported.

When you consider the number of hardware interfaces that Linux supports (PCMCIA, PATA, SATA, PCI, ISA, MCA, PCI-E, PCI-X, AGP, USB, Firewire, I2C, the list goes on and that's just for the x86 platform) and take into account the amount of drivers for each style of interface (some of which share something like 99% of the code, most of which are completely individual) the fixing of a stable interface gets harder and harder without bringing the source code to an unmanageable level.

Yes, Windows does it to an extent. However, try to run a Windows 98 scanner driver in Windows XP. Most of them won't let you do it. The interfaces changed and are no longer compatible. Try and install an ISA card in a machine running XP, it won't recognise it. Microsoft obsoleted ISA cards because they felt like it. That's another issue, but what if Linux were to obsolete a major subsystem? First, there would be outcry, secondly, the code would be around so that people who WANTED it could still use it.

Try and get USB Mass Storage devices working on 98. You usually cannot without a specialised driver and even then, only on 98 Second Edition because the USB in 95/98 did not support it properly. The standard interface they chose in 95/98 WAS NOT COMPREHENSIVE ENOUGH to support Mass Storage Devices and had to be changed. 98SE made it possible by introducing new access methods but the drivers are usually totally different to those used for the same hardware under Windows 2000 or above.

Try and get most older games working in Windows 2000 or above. It's possible for the vast majority but far from easy and it's usually easier to run an emulator like DOSBOX or QEmu to do it because the interfaces and standards used in the DOS, Windows 3.1, Windows 95, 98 etc. era were obsoleted and changed and updated and even removed because they were incompatible with the "new" ideas going into later versions of Windows (e.g. early Windows versions had no real concept of multiple users on a single machine, early DOS games expected complete control of a processor in order to run and exact timings which aren't practical in a modern multi-threaded operating system).

Throughout the history of any operating system, and sometimes even applications, the set standards that seemed so perfect 10 years ago are never used properly or have to be worked around to make them work properly (consider things like 48-bit LBA drive access) and usually that means having to change the interface or corrupting it to your purposes.

Linux does not want to spend the next twenty years supporting it's own dreadful mistakes and misjudgements. Being Open Source, it does not need to. If something's wrong, they can change anything they like because no part of the system has to stay as it is. Linux is a liquid concept. However, should such changes occur many of the current binary drivers for Linux are likely to need substantial support in order to continue working properly, if at all. That support can only come from people with the driver's source code, i.e. the manufacturer.

When a kernel interface change stops PCI-Express cards from being accessed the same way they used to be, nVidia may be able to bodge something in their kernel wrappers or they may have to recompile their binary drivers to take account. Either way, they would have to spend a lot of time and money to support a change that is completely outside of their core business. People would be moaning at them for not doing their job. They would have to keep up, as they do today, with every change. And ten years from now, when they go bust, none of us will be able to use nVidia 3D acceleration on anything but the last kernel they supported.

Then again, they may just decide that it's too much bother and stop producing drivers for Linux. Were they Open Source or, ideally, in the kernel, the updating would *probably* be done for them automatically and without charge. They would be tested, without charge, by far more people than any beta test could summon up, with far more exotic configurations. Every time the kernel changed, they would be kept working until the day that there wasn't a single competent person in the world who wanted to keep supporting their hardware.

When IPv6 came along, Windows and Linux supported it by redesigning every piece of their networking code to take account of it. When IPv7 or whatever is next planned comes along, you're going to have to redesign everything all over again or put in some horrible backwards compatibility kludge to help older programs use it. That means that you will forever have to carry your older systems with you and all their backwards-compatibility layers, or you could just redesign the networking code to take account of it all for you so that old programs don't need to change and new programs can use the new features. They may be entirely seperate systems but why introduce a whole new layer if you could just slightly redefine one that's been working for years?

Binary drivers die a death as soon as the manufacturer stops updating them and are wounded by every kernel upgrade. Open Source drivers live for as long as there is a single person in the world willing to support them, barely feel a bump on a kernel upgrade and will stay in hibernation for as long as their source code exists, ready to be resurrected by anyone who wants to try them out (say, a computer museum curator who wants to run some ancient hardware card that has to be soldered onto a modern connector and have it's drivers tweaked to support the new, bodgeful interface).

A stable binary interface is impossible and totally against the idea of having a system that anyone can submit an idea for improvement to. When someone invents a better way of running hardware, all the internals HAVE to change or you end up with a mess of code that nobody in their right mind wants to touch and certainly not one which you would want people to be learning *from*.

Linux is bigger than an operating system. It's bigger than the companies that use it for commercial gain. It's bigger than the millions of people around the globe that use it every single day whether they know it or not. Linux (and Open Source in general) is about making things work, making them work well, making them work for the forseeable future, letting anyone see how they work, letting people come up with ideas for how to make them work better and keeping them working. None of those goals can be reliably met by using junk like source-less binary drivers or "stable" binary interfaces.

The annoying part is not that people disagree with the above, it's that they demand that Linux should change and not themselves. If Linux does not do what want, Linux is in the wrong. How often do they also swear at how stupid the design of some internal Windows API is? Linux is an emotional creature. It does not care about people who don't care about it.

If you want to rip Linux off and sell it with a thousand binary components and you can find a way to do it legally, even if you sell a unit to every single person on the planet, then you are on your own and Linux won't care that they break your system in their next upgrade. If you decide to Open your drivers and do the same, the chances are that someone will come along and help you to keep your stuff working, especially if it means that they get to play about with your system, ask questions, try new things, fix problems that only they have and can go off and learn from your code.

Open Source encourages software evolution. The better-written and better-performing man wins and their source code gets incorporated into more projects, their code gets learned by more people, who spread it to more code. Before long, every project needs this code to work properly, ensuring its own long life. However, if something EVEN BETTER comes along after that, it will be usurped for the greater good. If it's really better, it will take you over and smother you. If it's not, it will just linger and die and you will remain in your throne.

Binary drivers are one thing that I'd like to see smothered quite quickly. They are not necessarily better written or better performing but are kept there by corporations that are trying to gain money and recognition from Open Source without giving anything back. They legally, ethically and practically hinder alternatives from cropping up to usurp them as they know that they would be quickly smothered and left for dead. They need to maintain their little monopolies over their precious property. There is no analogy in nature for such a beast.

The fact that you can't manage to create a driver for the OS of my choice just means I won't buy your gear, or recommend it, or maybe even consider it. Whining about lack of co-operation from Linux people when I am happily running the product of years of their freely-given hard graft is not getting to get you any sympathy from me. If the drivers for your device came from your company along with the full co-operation of your company, I'd sing your praises and buy your gear. If you just want to cling onto the back of this Linux thing that people seem to be installing more of nowadays but not give anything back, don't expect Linux or its users to do you any favours either. I will only pile my money into something that I know will last me a long time and give me good value for money.

And now an admission... I own an nVidia graphics card - a weird one. It's a PCI Geforce4 440MX. Yes PCI. Not AGP or PCI-E. Bog-standard, old-fashioned PCI. I want to use it (it's my most expensive graphics card purchase ever at £50) and I don't want to replace it. That's not much money but the card is CAPABLE of doing everything I need. If I needed PCI-E levels of performance, I would have a PCI-E card.

I have it in my Linux desktop machine, primarily because that's what the machine used in it's previous Windows incarnation. I used to play Counterstrike and the motherboard does not have an AGP slot. The GeForce fitted the bill nicely. In Linux I have little or no use for it's 3D features (besides possibly the occasional game of TuxRacing) but it runs faster than the motherboard's onboard graphics.

I voluntarily use the nVidia binary drivers. The reason is that they provide better performance playing video, 3D etc. They prove that the card is capable of doing what I want. The binary drivers are not too much of a bind for me to recompile every time I change the kernel. They don't cause any crashes at all and the card works perfectly. If I need to diagnose a problem, rebooting without the nVidia driver but with the Open Source nv driver is not a big deal.

However, if nVidia updates their drivers to a version that doesn't support my card, introduces bugs, etc. I will not be upgrading to those drivers. If the Linux kernel people manage the technical/legal/ethical feat of making sure that nVidia cannot distribute any drivers but GPL ones, I will instaneously revert to the Open Source nv driver unless nVidia DO release a GPL one.

I won't be petitioning the Linux kernel people, I won't be rushing out to buy a new card that has got OS drivers, I won't buy nVidia's newest card that does run on Open Source drivers. I WILL be complaining to nVidia for not releasing the type of driver they should have released in the first place. I will use whatever works best for my current hardware to legally and technically interoperate with the rest of my machine's software.

If that means that, ultimately, my performance is reduced to poor levels because of having to use an inferior OS driver, I will be blaming nVidia for not bothering to contribute to that driver, to enable features that their hardware is perfectly capable of, and will adjust my next purchase according, to a company that does support OS and does not artificially limit the capabilities of a piece of hardware by refusing to openly publish code or specifications for it.

The nv driver already has what I need to run the card. Anything that isn't in the nv driver is due to nVidia not being co-operative.

I will not stop updating my kernel to the latest stable version, even if that means I break the nVidia card... an up-to-date kernel is worth much more than a single, replaceable driver.

I will not allow the kernel maintainers to be blamed for nVidia's lack of assistance. They do not care and never have cared about binary drivers and have stuck to their word on that.

I will not allow the nv maintainers to be blamed for nVidia's lack of assistance. They tried their best to get SOMETHING out of a company that wanted to give NOTHING.

I will accept it as inevitable that I knew I would eventually run into this problem, because I chose to use binary drivers for that component.

I will not be surprised if those same binary drivers stop working or fall into decay one day.

At least if I had open-source drivers which were capable of driving the card to it's full capabilities, I could keep running them through whatever legal or ethical turmoil Linux or nVidia goes through - that's the point of the GPL. If all else fails, I can still change the code myself (and I am capable of doing that) to make it work again. I don't have to rely on company X to keep my card working for me, breaking god-knows-what-else in the process. And I know that my hardware isn't part of some secret cover-up of something that I really don't care about when all I want to do is play TuxRacer.

Laptop security

2005-12-31T13:53:00.000Z

Recently, what with Christmas being seen as an ideal time for theft, I've been in meetings concerning the security of computer hardware, most notably laptops and projectors. Apparently I work in the second-worst location in the UK for thefts from schools.

As some of these meetings were sprung upon me without warning, I wasn't able to think them through as much as I'd like to have. As a result, I've been double-checking my advice to the schools to see if I can come up with any better ideas.

Current advice from the police includes chemical marking of property, securing the building, displaying signs and implementing CCTV (although the later is not really pushed as a solution, more a deterrent).

According to the second-hand feedback I've been hearing from the schools, the local police are having a tough time; schools are having lots of laptops and projectors stolen, the thieves are filing off the serial numbers and then the police are unable to confirm who the property belongs to. There are even stories of the stolen property having to be returned to the thief after a while as they are unable to prove that it's not theirs.

CCTV is proving all-but useless as the thieves are always ready and cover all identifying parts of their body or clothing. Chemical marks are easily discovered with UV lamps and removed even if it means damage to the property. There's also a growing black market in projector bulbs as these are not serialised and are therefore almost impossible to trace back to a source, as well as being an easily removeable, high-value commodity.

All this lead me to thinking about laptop security. Currently, the only physical way of securing laptops are so-called "kensington locks", small standardised holes in the chassis of laptops into which locks can be placed and also easily removed, sometimes without any damage at all to the laptop.

So if you can't prevent them being stolen, is there something else you could do? Each computer processor has a unique serial number burned into the silicon of the chip itself. However, there is usually no way to read this number from the chip as most manufacturers disable the option by default and also the thief can easily disbale the same option. This means that not only is it time-consuming to actually read this number from a laptop on purchase, it's easily disabled too. Although if the laptop is physically recovered the number could be checked, there's no way to read this number remotely.

Lots of software packages exist to "phone home". That is, every time the machine is connected to the internet, the software sends a small packet describing it's location/phone number/other identifying pieces of information to a central server. If the laptop is ever reported stolen, this information is passed to police so the thief is "caught" as soon as they go online.

The major flaw here is that a thief is going to be aware of such tricks and any professional would probably blank the hard drive upon receipt or even replace the entire drive unit and then install a clean version of the operating system. Software piracy would not be a big deal to a laptop thief.

Additionally, any hardware means of doing the same would also be detected and removed/circumvented. Or would it?

Why doesn't someone add to standard laptop chipsets a "call-home" modem/network card? Most laptops have built in modems/network cards nowadays and they would be the devices that actually physically connect to the Internet eventually (I'm assuming that any stolen laptop in use today would most probably go on the Internet at some time in it's life, which is not an unreasonable assumption).

Obviously, the modem/network card would have to call-home without the thief knowing. Let's assume, therefore, that the software driver for the modem/network card comes in two types - on the one hand, it will identify itself as a standard modem/network card, as supported by internal Windows drivers or the same drivers as a non-call-home device. In doing so, it will not give away it's purpose. However, the driver originally supplied with the hardware would also include an option to send a series of innocent-looking AT commands or even packets to localhost. This packets would set a hardware password, and maybe other information such as an IP address or email address, which would be stored inside the chipset firmware itself.

Once the password is set, every time the device connects to the Internet (which is fairly easy for the hardware itself to detect and intervene without software assistance), the device is "activated". From then on, if the device driver does not send the password by the series of special packets/AT commands, the hardware itself would inject packets with the intent on sending a call-home packet/email to a central server.

This central server would most probably be setup by the hardware manufacturer, but it could also be set by the customer themselves to be an email address of their own. Whenever a standard non-password driver is used for the device (such as you would get by a reinstallation of the operating system), it would attempt to send this packet/email, which would include such details as the phone number called or the external IP address or even a short history of phone numbers dialled.

However, even with the "correct" password-driven drivers installed you would HAVE to know the password in order for the device to activate normally (or even activate at all) without sending such call-home information. If the thief was wise enough to know that this laptop contained such hardware, they might try to install the specialised drivers. However, without the password that is etched into the chipset firmware by the manufacturer/owner there is no way the thief could disable the call-home functionality or change the password. This won't have stopped him stealing the laptop but it will seriously limit it's resale value, a laptop without Internet access is severely limited in it's capabilities.

You could even add functionality to the "secure" drivers (the ones that ordinary customers will have pre-installed for them) that the device won't initialise the modem/network unless it receives the correct password from the user. This would prevent the thief from just using the pre-installed drivers, effectively forcing you to "log on" to the modem/network card before you can use it.

With such controls in untouchable silicon on the device that controls the modem, network card, wireless card, etc. a thief would be left with a crippled laptop, unable to go online for fear of being caught.

Even wiping the entire disk would do nothing, the specialised drivers would be gone so the chipset would "know" that it was being used on a machine that may have been stolen and wiped. If the device runs on a standardised driver (e.g. a plain 56k AT command set or an NE2000-compatible network card), then a thief reinstalling the system would be unaware that by using the standard Windows driver they are advertising to the chipset that the system has been stolen. Only the NE2000 driver which also sends the correct password (most probably obtained from the user at boot-time) would be able to circumvent the call-home functionality.

The original owner would, of course, be perfectly capable of reinstalling their operating system as they know the password to the device and be in possession of the drivers to send the password to the device. Even if the original owner sold the laptop, the person they legitimately pass the laptop onto could still use non-secure drivers. The laptop could handshake with the central server to see if it has been reported stolen before sending such a packet or, at worst, send an email to an address whenever it connects. This might even be a good audit tool for companies to see just how much the laptops gets used.

Combine this with the fact that the concept is cross-platform and operating system independent (so long as two drivers exist: a standard one that can use the hardware normally and a specialised driver to send the special commands to the device upon initialisation) and you have a pretty foolproof system. You could ask for the password on boot (most corporate laptops have boot-time passwords anyway and the functionality could be implemented in the BIOS rather than the OS drivers), on login or on use of the device. Inexperienced theives would be caught the second they used the laptop online, experienced ones would be deterred or at least know that the value of a laptop with such a system would be severely limited.

Just an idea I had ticking away in the back of my mind.

The World's Best Computer Games #1 - Quake 1

2005-11-20T23:40:00.000Z

I thought I would try my hand at saying not only which computer games I think were the best, but also how they could have been made better. Firstly, the list of games. Although everybody else in the world is likely to disagree, the list is based on what games I find myself wanting to play when I have exhausted whatever the most recent game I installed was. I'm not a huge games player but I have played an awful lot over the years and have found an unfortunate trend for modern games to be mostly devoid of gameplay. This list should, hopefully, point people at games that are fun to play, engaging, have lasting appeal and are not sold as an advert for a CGI workstation.

I'll quickly list a few of the candidates, however this is not is order and is almost certain to change and be updated as I describe each game in detail, along with where I think their sequels or extension packs should have gone.

The Settlers (PC)
Chaos / Lords of Chaos (Spectrum)
Syndicate (PC)
F29 Retaliator (PC)
Final Fight (Arcade)
Spy Hunter (Arcade)
Quake (PC)

First, though, I'll start with Quake. For the sake of simplicity, I'm going to take Quake 1, GLQuake and both "official" Mission Packs together.

Quake was (I believe) the first ever truly three-dimensional, fully textured game on a standard computer. My first exposure to it was on a Pentium 133MHz . To set the scene, 3D graphics cards did not really exist except in the most extreme of PC's, the Internet was there but mostly limited to specialist uses and the OS of choice was DOS or Windows 3.1.

At the time, Doom and some other minor shareware offerings were the only games to have any sort of 3D. The best 3D offering up until Quake had, in fact, been Doom, with it's textured walls that didn't scale well, 2D-only movement (3D was faked... you could see out of windows and go up stairs but it was all a clever trick... you could never look up or down), blocky sprites and an atmosphere that no game before its release had ever shown before.

Doom was no more that a more convincing version of the "3D" techniques used in previous titles of a similar nature, such as Wolfenstein 3D and Spear of Destiny. Quake, however, managed to invent a whole new level of gameplay. Fully 3D environments, introducing full mouse-look, had most people spending the first hour of playing their new game just looking at the scenery and eagerly hunting out things to look at.

Quake was beautiful for it's time. 3D cards showed off their power and uses like never before and most people who bought a 3D card at this time bought it so they could play Quake. A 3D-card in this era meant that you bought a 3DFX card - no set standard of 3D acceleration in DOS existed until 3DFX's Glide arrived, thereby making itself the standard for years to come. Now virtually unheard of, 3DFX and Glide had set the pace for home 3D graphics. Without them, the nVidia's and ATI's of today wouldn't exist.

3DFX cards were also able to use SLI (Scan Line Interleave). PC's at the time were pretty puny for any serious 3D work and so being able to install 2 3DFX cards in one PC and having them share the work of the drawing helped considerably and at a price point where there wasn't a faster processor available to just brute-force their way through the calculations.

Quake showed off Glide's capabilities to the point where, ever since, they've been basically an essential part of running a gaming PC. Modern computers are approaching a point where 3D and 2D are integrated so tightly, processors are so over-powered and consumers so demanding that most PC's are capable of running even the most powerful games without "specialist" 3D hardware, just an ordinary graphics card. Back in Quake's day, to make people go out and buy an expensive component was the games one main drawback but it easily overcome it by making such good use of every processor tick it could.

The 3D environments were used well. Monsters jumped out of shadows, attacked from every concievable angle, pored out of secret doorways that you wouldn't notice if you weren't paying attention. Secret areas were difficult to get to but many players found them just by trying out things that they'd never been able to do before... jumping across obstacles, leaping onto tiny platforms, looking for inconsistencies in room sizes, hunting for secret buttons above and below them.

Lighting was used to good effect, the rooms going dark and strange noises coming from behind you was guaranteed to make you panic, critical areas were visible from the rest of the map making you try to fight through to get there, slits of light under wall indicated secret areas.

The sounds were incredible but no more scary than the noises of Doom, except now full stereo and an enviroment to support your movements made it even more realistic. But what made the game was its easy introduction to such new environments, the balance of the game, the level design and the relatively new multiplayer facilities.

Multiplayer was only used in relatively basic games up until the likes of Quake and Doom. Quake supported local network play (traditional IPX and newer TCP/IP), effectively creating the first LAN parties as well as modem and serial connections. I can remember playing Quake with certain mod's over a 56,000 bps serial connection (I remember distinctly because we didn't have any sort of connections between the machines beforehand and had to bodge a 9-pin to 9-pin cable using a variety of other cables and adaptors. The final cable ended up over 6 metres in length with numerous adaptors but worked at full speed without any issues - I still have the cable and still use it because, by disconnecting it at certain points, you can form ANY serial connection you're ever likely to need with each end being male or female, null-modem or plain, 9 or 25 pin.)

Talking about mod's... the fact that people could make maps and modify the game added greatly to its use. I spent many hours combining the best parts of my favourite two addons (both of which changed everything from the weapons to the monsters to the AI) and removing features they had that I didn't like (things like instantaneous weapon respawning) while adding in other stuff that I wanted. This wasn't tweaking or playing about, you were actually given a fully functional programming language (QuakeC) in which to do whatever you wanted. It was a little buggy and pernickity but once you got the hang of it you could do almost anything. In fact, that's how the addon packs came about - the better AI in the official addons (codenamed hipnotic and rogue after the companies that made them) worked purely by using new code in QuakeC, with new models and new maps. The Quake executable never really changed at all, except to support loading from different filesets.

Quake was my first exposure to multiplayer deathmatch - years after our first foray with "the ultimate serial cable", we were still playing Quake mod's like Nehara and Wyrm with friends over our 10base2 network and the internet and still they worked just as you'd expect today, sometimes miles better than even many modern games multiplayer implementations!

Even the basic single player game would last you ages, though. You could probably force your way through it in several days of gaming but you never wanted to. Much more fun to actually play the game, find the secrets, kill every monster on the map. Your path was always pretty linear but it was actually intriguing, wondering what new monsters you'd have to deal with around the next corner. Quake was never easy, it wasn't a game to pile through just to say you'd completed it, you could take your time over it and enjoy it.

From a technical side, the system requirements were pretty minimal, even adjusting for relative price increases, and signle-handedly sparked the entire 3D gaming hardware industry - and the compatibility was phenomenal. It ran under DOS originally but then Windows came out in force and Quake still run under Windows, then a Windows native version was released and when the game was over-sequelled, the source code (which was already being licensed into games for things like the Half-life engine etc.) was GPL'd and Unix-native versions came out among all sorts of strange conversions. It also ran on Mac's from early on in it's life.

Gamesaves were simple - one key save and restore from whereever you were. Without those sort of gamesaves, the game would have taken months to complete. Video options could be ramped up or down to suit your hardware and platform. Modern computers just laugh at Quake now as they can all run it in it's top resolution without any sort of strain but back in it's day, some of the options seemed unreachable. The fact that it still scales and plays well on modern machines (and modern OS's) is only testiment to it's high technical quality. You could actually play with just the keyboard if you wanted to but having mouse-look on made it the exact equivalent to the modern FPS's that it spawned.

Network play was smooth, single play was great fun and engaging, the maps seemed enormous and to have no end to their number, downloads (if you had Internet connections back then) would only extend the amount of things you could do and the time you could spend on it. There wasn't really much of a community of multiplay gamers back then and yet there was still a major source of new content for it coming out, from official mission packs just as large as the original game to entirely new ways to play the game coming out of FTP sites.

There was no complex plot; it needed none, it was sheer atmosphere. You could jump, swim, drown (!), burn, electrocute yourself, get squished, use lifts and stairs, dodge flying buttresses and collapsing rocks, watch lava balls leap into the air in front of you, see zombies rise from the water, tear off their arms and throw them at you. The new concepts that arose were just fascinating and that's where the gameplay come from. Mods added things like low gravity levels, grappling hooks, heat-seeking weapons, trap-laying, capture-the-flag games, even primitive physics.

It was like nothing that had been seen before by the average gamer. And all from your home PC. Unfortunately, the number of new features introduced into a dozen modern games is seriously dwarved when compared against those introduced just in Quake. We might have higher-tech games now but, short of more realistic physics, there is little new in them that wasn't present or possible with the now-ancient Quake engine. In fact, many modern games use engines which are based, somewhere along the line, on actual Quake source code.

Trust 445a Router Recovery

2005-11-01T13:46:00.000Z

A few years back, when I first went to ADSL from 56k, I bought a router from my favourite company, Trust. The router was a Trust 445A Speedlink xDSL Web Station, a bog-standard Conexant-based router pretty much indistinguishable from a technical point of view from my previous AMX-CA64E router.

After about 6 or 8 months, however, I tried to change a setting in the router's config without thinking about it (I made the router switch to PPP Half-bridge mode as I was pretty much using it in a Half-Bridge configuration anyway) and managed to make the admin interface inaccessible. At the same time, it stopped giving out DHCP, connecting via ADSL or working at all.

I couldn't find any firmware to restore it from (the one time Trust has let me down by not having the correct drivers) and it seemed beyond repair. I threw it into a back cupboard to use as an emergency four-port Ethernet switch in case I ever needed one. At the time I was a bit short of money but still went out and bought a replacement so that must have really meant that back then there was no way to recover this particular router; I never consign anything to the bin if there's the slightest chance I can recover it.

Digging it up a few weeks ago when clearing out some of my older computer cables, I thought that it was worth another shot. ADSL doesn't seem to be going anywhere any time soon so it was worth getting a backup router or using it if I have to fix someone's computer.

Details were sketchy to say the least. Google only turned up Origo Repair as a likely candidate (which would probably work if you knew how to fiddle it and probably works wonders for people whose routers can take most firmwares and who are willing to try any number of firmwares first.

Instructions for the 445A were much harder to track down until I spotted an old forum entry, hidden away in the depths of a long conversation:

"I have Trust 445a too, and it seems it uses the same chipset CX82310-14 ARM940T Processor as Billion BIPAC-711CE and others. Italians secceded flashing Trust 445a to Billion BIPAC-711CE."

The website linked to contained an instructive PDF, a driver set and a firmware image of a Billion router. The PDF itself was in Italian but with that, the firmware and an online translation service, I was able to get the approximate gist of the plan.

1) Open up the router and short jumper JP1.
2) Turn on router.
3) Connect via USB between the router and a computer with an Intel chipset motherboard (the flashing utility requires certain chipsets, otherwise it can't recognise the USB ports). I found this out on the Origo Repair page which seems to use similar flash utilities: "The only limitation is the Flash program that is used. This seems to be very fussy about the chipset in your machine. It is known to work with VIA & Intel but not with SIS, nForce2 & KT266."
4) Boot DOS (I used the Ultimate Boot CD's version of Freedos)
5) Run the flash utility with the /e switch to erase the current firmware.
6) Run the flash utility again, this time supplying the firmware on that website.
7) Wait until it's uploaded and the modem settles (quite worrying that for several tens of minutes it just looks completely inert but eventually it all completes successfully)
7) Remove the jumper, reboot and test the router.

Additionally, there's a further step to then go on to upgrade the onboard files on the router to support UPnP and various other minor fixes, but that's just the easy part.

After a few false starts (a laptop and a desktop with only SiS chipsets that the DOS utility couldn't use the USB ports on, having it plugged into a USB port which the DOS flash utility did not see as the FIRST USB port, etc.) I managed to flash the firmware perfectly.

A quick reboot of the router showed it working just as it should, albeit with another company's branded version of the admin interface. Basically, from a blank EEPROM, it had restored the 445A to perfect working health and even added UPnP support as a bonus (the 445A didn't support UPnP with it's original settings).

I stood little chance of finding this stuff out on my own and am eternally grateful to the forum poster, the Italian website authors and Google for restoring a dead router back to full health, thereby saving me about £50 in the future.

Now to go and burn that firmware, driver and my own instructions onto a million and half CD's, tapes, DVD's, floppies, USB disks and hard drives just in case the website goes down and I need it again!

SSH port probes and port knocking

2005-10-17T17:41:00.000+01:00

On the topic of people receiving unwanted SSH port probes, I have previously posted a script to catch requests and blacklist the source IP's. Though this strategy is effective, it lacks a certain finesse and it also doesn't prevent what I saw as the major problem about such probes - log flooding.

An average Internet-facing server has room for hundreds of megs of logs, an admin capable and willing to go through the logs en-masse and blacklist troublesome IP's. Unfortunately for myself the only real factor in these potential attacks is in the annoyance that a large log or repeated visible attempts generates.

I am always security conscious and this hasn't changed, however I think it's time for my home machine to stop storing logs of every little script-kiddie that pings my port 22 with their tools and instead just ignore them. That was previously much more difficult for me, especially when you see the same IP try and try again. It's just so tempting to blacklist them and imagine their reaction on the other end.

Previous to this article, my port 22 was open to the world (but no other ports). The control of who got in was left in the more-than-capable hands of a religiously updated OpenSSH, with only one working logon allowed to actually come in, passwordless logins denied and a large private-key that's kept secure, protected also by a long random passphrase. That allows access to an extremely limited account whose only purpose is to allow me to su with the right password and become root.

I use SSH to log in from machines when I am at work (where it's incredibly useful to bypass over-zealous school internet filters, with a little help from a proxy on my home machine) and when I need to fix other people's machines. All the schools I work for are NAT'd across the internet through a single IP. Thus, it would be simple to allow SSH through just that IP. However, when I am fixing someone's machine (which could be at a moment's notice), I need SSH to still let me in. This means that I would have a basically random IP trying to connect to my home machine and it would have to deal with it accordingly. This is why my port 22 was left open up until now.

Previously, I just watched the world and his brother bounce standard logins and passwords off of my home machines in a vain attempt to see if I had left some stupid login or password on the machine that might give them some sort of platform to springboard onto higher access. I was secure in the knowledge that not only would they never get the only working account name by bruteforce, nor would they have the publickey and neither would they have the passphrase which went with it. The worst case was that they exploited some kind of flaw in OpenSSH itself which I consider highly unlikely and easily fixable. In that case, it wouldn't be just me who'd be in trouble but half the servers on the Internet and someone would come up with a fix VERY quickly.

These dumb brute-force attacks left a little bit of a bad taste, though. My logs were constantly flooded by "invalid user", "potential break-in attempt" and so on, all through the day. The machine that controls port 22 is a desktop and toy machine, selectively exposed to the internet for convenience and for trying out new systems and therefore it's quite annoying to have logs flooded by such futile attempts at breaking in.

I have put on this blog a script that I used to use, it basically watches the logs via a regular cron job and blocks any IP that tries too often. This works wonderfully and is the situation I would use if I needed random people from random IP's (e.g. employees etc.) to be able to access the SSH on that machine.

Fortunately, I don't need to. The only person to ever use the machine will be myself and therefore it's much easier to me to use port-knocking.

Port-knocking is the Internet equivalent of a secret handshake. As far as anyone on the Internet is concerned the machine is not responding to any connection requests whatsoever. What it is secretly doing, though, is monitoring all such requests for a particular, pre-determined pattern. When it sees that pattern of requests, it modifies the firewall to open a particular port to that particular IP.

For example, it's possible to set it up so that when it sees connection requests on ports 1000, 2000 and then 3000, all from the same IP address, in that order and without any other port requests in between, it opens up port 22 for the IP that tried to connect. Thus, from any computer in the world, I could take a small utility or even just a batch file and a telnet command, hit those secret-handshake ports in the right order and as if by magic SSH would suddenly be available for me to use from that computer.

A similar "knock" of some different ports would also close the SSH port behind me, too. Using this, it's perfectly possible to rap on the internet port of my home computer, get it to let me in, SSH and do whatever I need to do, then rap a different tattoo and it would close SSH. All the time this is happening, every other computer in the world could be trying to get in and all they would see would be a closed or stealthed port (depending on whatever firewall config you have). Being closed or stealthed would also mean that SSH doesn't even have to wake up to deal with such requests, so bypassing any possibility of having some SSH exploit being randomly tried out on the machine. It also means that people can't even GET to the stage where they can try to log in, so instantly cutting out quite a lot of log-spam.

The utility that can knock like this can be as simple as batch or shell script with certain parameters to telnet, to a custom-made tiny C program or a generic port-knocking utility which you supply the port list to. The ports can be UDP or TCP, they can be any numbers in any order, they could even be encoded versions of a particular passphrase or password. Depending on the port-knock listen server in use, the knocks can also be one-time passwords chosen from a predefined list (so that even if someone sniffs your secret-handshake going across the internet, they wouldn't be able to replay it).

I have now switched over to a port-knock system (a good one seems to be Knock from the author of Arch Linux), if only to make my logs easier to read, and have noticed a myriad of advantages from this system.

1) It doesn't make my system any less secure - the only required software is a port-knock daemon that's trivial in implementation and quite secure by design (all it does is listen to connection attempts at the IP level and then extracts the source IP and destination port. It then executes a shell script of your choice which does all the opening, closing of ports etc.)

2) It makes my system more painful for someone unscrupulous to try to access - ports appear closed or non-existent and there is absolutely no indication that anything "strange" is going on. Knock-opened ports are only opened to the IP that knocked correctly, thus keeping everyone else in the world in the dark.

3) It means that the software behind the port-knock is protected from random, "dumb" brute-force attacks and new exploits. Though this isn't 100% effective (as every security analyst has argued when confronted with port-knocking) it's made another step harder, to the point where you really need to be able to sniff all traffic in and out to be able to defeat the simplest configuration of port-knocking. If someone can do that, you're already in trouble.

4) The knock acts as a magic key. It can be a simple numeric list, it can be TCP or UDP or even (theoretically) ICMP. Equally, however, it can be a one-time password that expires after use (so combatting sniffing/replay techniques to open the port), an encrypted sequence, a time-dependant sequence, an IP-dependent sequence, or a combination of those and many other possible sequences (one based on the last time the admin logged in, for example?). Until the magic key is used, nobody can access any port or tell that anything screwy is going on. As far as the casual "hacker" is concerned, there's no way to tell if the machine is switched on and looking for a portknock or switched off entirely.

5) Anybody trying knocks can be blacklisted in a similar way to we had before, but using the failure logs of the portknocker rather than millions of attempts at usernames.

6) The knocks are immune against every-port scans, like an nmap scan, as the knock has to be done in the right sequence. At no point in the sequence is any clue given to a potential brute-forcer about how far through the sequence, which individual knocks were successful etc. because of the multi-stage nature of portknocking.

Knocking port 1000 in the example above makes everything still look closed down. You knock port 1001 afterwards (or indeed any port but 2000) and you have to start the sequence again to be allowed in. However, you get absolutely no feedback about whether or not each stage was successful.

Therefore the chances of brute-forcing the knock have decreased dramatically. Say you have a three-knock TCP-only rule. Then you would have to try EVERY single combination of three numbers all in the range 0-65535. That's 281,462,092,005,375 combinations, over 281 million million, each consisting of a single packet. And in between each one you would have to scan every port to see if it opened anything that wasn't open before. That's going to need a VERY long time and be more than a little obvious, even on the fastest connection in the world.

But now imagine you have mixtures of TCP, UDP and ICMP, four-knock, five-knock etc. rules, time-based rules and other clever trickery to determine what the secret handshake is. It would be all-but impossible, with a decent well-thought out port-knock system, to get anything to open up at all, thus stopping any sort of brute-force attempt before a login dialog had even been found.

7) The minimal overhead of such a system is valuable too. Possible brute-force attackers take no more bandwidth than random port hitters. No resources are required to track hundreds of parallel connections other than standard TCP/IP. The CPU strain is kept to an absolute minimum and the worst case would be a DoS attack requiring substantial resources behind the attackers to initiate. Again, you're already in trouble but at least they're bouncing off your TCP/IP stack rather than trying to guess your passwords.

Multiple parallel attempts from different IP's would be no more help, either. The knock would have to be completed from a single IP correctly. Additionally, it may well be that the knock is IP dependent or that some IP's are blacklisted from any sort of port-knocking anyway!

Additionally, it's one piece of software on the server and one trivial piece of software on your USB key or in your normal remote kit. Currently I always have PuTTY, my private key and TightVNC as my standard remote kit anyway, so adding a tiny portknock utility is a pittance. The system is simple and completely follows TCP/IP rules and standards. To people who are allowed access, the overhead is just one more command, one more password that may even be linked to the main SSH password.

8) The system is completely customisable, from what the port-knock actually consists of down to what it does. In the end, each knock sequence runs a shell script and so can do anything that the user it's running as can do.

9) At the end of the day, even if someone manages to sniff your network, steal the time-dependence equations or even completely guess your port-knocks, you're then left in no less secure a position that you were before. You can still blacklist, you're still password/public-key protected and no changes have to be made to any of the application software either on the client end or server. Everything is just as it was before.

This system is for the ultra-paranoid as well. As an example, consider trying to attack a system with these layers:

1) No ports are opened to anyone without the special knock. The entire world sees nothing but a blank reply, identical to one which a dead connection or turned-off machine would give.
2) The knock is a hash of source IP, time, secret password and other variables. Nobody would see the knock without sniffing every bit of traffic into the computer.
3) Even when the knock is sniffed, replaying it does not open any ports. With no ports open, nobody can attack.
4) Say someone steals the sheet with all the necessary portknock passwords and equations to open the port (an extremely unlikely but yet still not very dangerous scenario).

This is where any open SSH server on the Internet currently is... SSH is considered uncrackable even when all communcation is sniffed if implemented properly.

5) Now they have to find a login that is allowed in via SSH.
6) There are no passwords on any logins, only publickeys, so they also need the publickey of that user (which cannot be sniffed in any way).
7) The publickey is also protected by a passphrase. They now need that as well.
8) They have all the above and managed to get into SSH and login as that user. Now they could (possibly) do some damage by exploiting vulnerabilities in the underlying system.

What would you do if you were faced with such a system? Personally, I think it would be at least 281 million million times easier to just break into the actual building that held that server rather than even try to access it.

But port-knocking is not just for SSH. As hinted above, because the result of a successful port-knock is the execution of a specific shell script, opening holes in a firewall for a particular IP is the most basic of uses. With the right software or hardware it could do anything you wanted. The right port-knock could do anything from turn your machine off to waking up every machine on the LAN, initiating backups to rebooting, putting the kettle on to sending out alerts to every technician.

CCTV, Motion Detection and Linux

2005-09-16T17:46:00.000+01:00

Over the summer, I found myself with quite a bit of free time on my hands. I have also, for some time, been eager to install a small CCTV camera at my front door to see who's at the door (Quick, hide behind the sofa!). This is partly for my girlfriend and partly for my gadget obsession.

I read on BBC News and also seperately on The Register about a burglar who was caught when he stole the PC which was monitoring a house's security camera. The PC emailed every image of any movement detected on the camera to a remote email address which, obviously, was quite theftproof.

I thought it was a marvellous idea, having a visual record of any event on a security camera sent to a remote email account (far from where the event is happening and also very secure) which doesn't need any authentication to send to (and therefore leaves no passwords on anything that could be stolen) and also requires authentication (which any intruder/thief could not gain from the computer stolen) to be able to delete/view the images in question. Not only that, but the ISP logs and email images would provide quite substantial legal proof in any case coming to court, in terms of verifying times, dates, tampering etc.

Properly set up, only a pre-emptive phone line cut would be any use against it. Even then, however, there's always the possibility of having a mobile phone,possibly even inside the case of the computer or as one of those PCMCIA GPRS cards, dialling up to an ISP, or even more complicated setups like wireless links between friendly neighbours or to a wireless ISP.

Short of covering from head to toe, cutting the whole neighbourhood's phone lines beforehand (an event certain to attract an unwanted amount of attention), jamming the 2.4GHz that most wireless networks run on and making sure to steal the PC's and any video recording equipment which was running the camera, and then wiping that PC with tools secure enough to obliterate any history of any images being written to the drive, there's not much a burglar could do about sending out some sort of information about themself.

I loved the idea of such a system and also that it actually works in practice, as the above story shows. Some months before this news story I had seen a piece of software that did this and apparently that was the one used to catch this particular burglar. This renewed my interest in Motion.

It didn't hurt that the software was Linux-based, free to use, easy to customise and very powerful. Any camera input (USB webcams, networked or wireless PC-compatible cameras, BT848-based TV cards or, indeed, any video equipment with a Linux BTTV driver) could be fed into the system (in fact many feeds are trivially possible), have complex motion detection algorithms run on it, with still images, short movies and even the audio being recorded whenever motion was detected.

These images and sounds could then be stored, transferred, archived or emailed anywhere (I suppose that FTP or SSH is also easy to do, basically the software writes a JPG/MPG and then runs a shell script of your choice on it whenever it detects motion). Additionally, it would be possible to watch through the cameras at any time by using suitable authentication on the web-based interface, showing real-time images to whatever computer on whichever continent you happen to be.

Over the summer, I invested in a cheap CCTV kit with remote 8" monitor. This monitor could not only supply power to and read video and audio from two different cameras, it would also output one of their composite outputs again without the need for further adaptors or cables. This seemed the perfect setup... a camera wired to the monitor so that I can see what's happening in real time, with the output being simultaneously fed straight into a motion-detecting PC setup.

The setup was a cinch, just a matter of dusting off some WinTV cards and adding one short cable to the CCTV monitor. The software compiled and installed and within about 10 minutes I had it emailing images to an email account whenever my willing volunteer waved their hand across the camera. With some fine-tuning over the next few days of settings and image masks to take care of the timed external lights interfering with the setup, the hanging baskets outside moving in the wind etc. I had the perfect test.

We went to Scotland for a week, leaving the cat at home. We arranged for someone to come feed her during the week and I thought it would be a good test. While we were away, I would dial up to a cheap ISP, log into my home machine and watch the images live. I could also browse through my email account and find all the images of movement. I saw the neighbours walk past at 9.58 a.m. I saw the postman come at 7.00 a.m. and could even see the three items of junk mail in his hands as he walked up to the door. I saw our friend come in to feed the cat as promised. I also got one or two images of a plant pot falling over in the garden.

Now intrigued by the possibilities, I'm considering extending the system. We have a car in the private car park behind the house that can be seen from our spare room. I only wish my neighbours could all have identical systems so that when a car alarm goes off we all know who's it is without leaving the house and can ring up the offender and make them go turn it off!

I plan to install another CCTV camera as a spyhole in the front door to capture full-frontal images of the person approaching the door and maybe as many more as I can find USB webcams for (I know I have at least two lying around). All this and it will cost me a little less than £20 extra per camera to cable and put in an old WinTV card. The computer appears capable of running at least two or three more cameras in terms of CPU speed (motion detection is quite expensive in terms of CPU power) and my test/development machines are old, obsolete things that people were throwing out.

I'm even looking into using a wireless setup so that, for example, the computer running the system could be wired *and* wirelessly connected to my main desktop, other cameras or a second system in a more secure location (the loft seemed an ideal place to run the show from, given that most burglars probably wouldn't bother to go up there).

I also remember having some cards that slot into a computers rear slots and the power supply connectors. They supply 9V outputs on standard connectors that plug straight into most CCTV cameras. With those, I could have a cheap, ancient PC or could even invest in a mini-ITX board, that would be using only a single mains socket and maybe a piece of CAT5. That PC would then be connected directly and supplying power to two or more cameras (some USB, some via PCI TV Card) and even microphones, connecting to a central computer which could store and email the files.

It could even text me the pictures to my mobile or indeed ring me up with an automated message, set off an alarm system, email my neighbours to get them to have a quick look, or blast out MP3's of "Wanted", "Rescue Me" or "Stop in the name of love". It could even display the culprit's face on my home TV in full, glorious technicolour with the words "GOTCHA!" displayed over it, while playing a Wah, Wah, Wah, Wahhhhh sound over the speakers!

What an idea. Marvellous what you can do with technology.

Substrate screensaver

2005-09-16T17:33:00.001+01:00

Normally screensavers are disabled the instant I set up a new system and my recent upgrade to Slackware 10.2 was no exception. However, I was looking through the list for a screensaver to test my 3D acceleration to see if my NVidia drivers were working and came across Substrate.

This is a wonderful little screensaver that I find quite beautiful, some say it looks like the evolution of a city from above, others like cracks in a white rock, still others, like a Picasso in progress.

Substrate example images

Either way, it's impressive results from a simple mathematical algorithm. Pity that I can't get a Windows version for my girlfriend as every time the screensaver activates on my machine, she sits and watches it grow.

Edit: I got in contact with the author of a port of XScreensaver, after many, many attempts at trying to find a suitable Win32 version or a version that would compile cleanly under Cygwin or similar environment, and he happened to have a Windows binary of this screensaver just hanging about. So thanks Darren Stone for your help! The version of WinXScreenSaver on his software page didn't have it when I tried but he pointed me to a version of just that screensaver that he had compiled some time ago: http://tron.lir.dk/software/w_substrate.zip

(As of late-February 2008, the above website appears to be down. For now, there is a mirror of this file here: http://www.ledow.org.uk/windows/w_substrate.zip

Another Linux Desktop Update (and Slackware 10.2)

2005-09-16T16:41:00.000+01:00

The Linux desktop machine is not only still going well, but getting better all the time. In fact, Windows has been deleted from my main computer, the result I'd been hoping for and something I'd been wanting to do for years. As far as I'm concerned, Windows in all it's variations is now just another console, good for games, not much use for anything else.

I recently upgraded to Slackware 10.2 on my main computer. Generally my computer is usually near the most updated you can get without having Beta or Alpha or CVS-build software on a machine. When it comes to upgrades, I upgrade to the next version of software depending on:

1) Whether I can run it in tandem with the older version - While this is possible with most things (e.g. Opera, PuTTY, TightVNC), it's not always possible with major upgrades. I want to KNOW that I can run the new version but that if there is a single regression I can still use the version I was using before.

2) Whether I can always revert to the older version if I want - This is where a package management system beats the built-in Windows features hands down. Not only will Swaret find, download, install and check dependencies of any software I install using it, it will also make a backup of the previous version. A simple removepkg/installpkg will get me right back to where I wanted, at worst having to replace any tailored config files from a personal backup. Most Linux distributions have such facilities, RPM, DEB, etc.

3) Other people's experiences of upgrading to that software - Lots of confirmed reports found on Google, relevant forums etc. of no major problems is the best thing, lack of any reports of major problems is next best. The less information available, the less I trust the upgrade.

4) Reputation of and previous experience installing that software - a program which has never had an install problem, upgrades itself neatly and compactly, is able to import all of its old settings etc. is one I'm more likely to upgrade as soon as I can.

5) The severity of the upgrade - how important an upgrade it is will determine how quickly I will be upgrading to it. Serious security updates for critical flaws and serious bug fixes for dangerous bugs will be more likely to be installed that something that corrects a spelling mistake in a filename.

6) The enormity of the upgrade - minor upgrades are more likely to occur, major upgrades may be postponed until I can test them out fully.

Given the above, a Slackware 10.1 -> Slackware 10.2 upgrade snags on 4,5 and 6. Numbers 1 and 2 are dependent on how carefully I think through the upgrade, and information for 3 wasn't available, although many people have been running the slackware-current version between the 10.1 and 10.2 release (I hereby thank all the willing testers for ensuring my machine will be relatively safe by the time 10.2 comes out).

Security wasn't a major problem - I was properly firewalled, my common desktop software (Opera etc.) was always at the latest stable version and nothing exposed to the internet was vulnerable. There were a few upgrades I was looking forward to, most notably a Ghostscript upgrade that made my printer (a Samsung ML-4500... possibly the only laser printer I've ever seen that you can open the specially-designed toner cartridge and just pour in certain toner without having to re-buy the entire cartridge) work much better under CUPS.

KDE upgrades were also on my agenda but not something I was happy attempting on my primary desktop machine by myself. This fixed things like my icons jitting about the place between (fairly infrequent) reboots, konqueror crashing while navigating the filesystem and numerous other little niggles.

It turned out in the end that I managed to clear out my old 10Gb Windows partition (so there really is no going back now!), after throwing a few lifebelts to things like documents, INI files and other stuff that might come in handy someday. With Slackware, unlike Windows, I literally formatted the drive as ext3 and copied the old Slackware install over using cp -a -x to the blank drive. This copied all the files, links, etc. over without modifying them.

A small oversight discovered later was that the -x (stay on a single filesystem) for some reason excluded /dev but even *that* couldn't stop Linux trying it's best to boot (although a lot of drivers complained). That went into the rather short list of "Should I ever have to do this again, remember to"'s.

Once the filesystem was copied across, I booted from a boot disk making sure that the new partition was the linux root. I played with the lilo config to set it up (keeping some older entries to boot back into the original config should I ever need to), edited fstab and then reinstalled lilo.

[[ Side note: My favourite thing about Linux in general is that any kernel can boot any partition on any computer. I needn't have bothered with this backwards compatibility entries in lilo, I could have just booted from any Slackware boot CD that I had laying around and tell it which partition to use as root and I can fix/run anything to get it back up and working.]]

Then I booted into this identical copy of my root and only once I was in and everything was working as if it was my old drive did I follow through the Slackware 10.1 -> 10.2 UPGRADE.TXT instructions.

The upgrade went very smoothly and once the package upgrades were complete, it was merely a matter of setting up lilo again to boot from the 10.2 2.6.13 generic kernel that had been installed (making my own initrd on the way) and then rebooting.

[[side note: Although I think that initrd are a marvellous idea (a small mid-boot ramdisk environment to bung in any strange drivers that may be needed at boot time, e.g. USB/Firewire Mass Storage, SCSI etc.) they can be a pain in the backside when you just want to add a new line to lilo with a slightly different config. That usually means rebuilding the initrd with different root parameters etc. which soon can become a file-management nightmare, having to keep kernels, configs and initrds for every different combination. Hell... it works and you can tweak it to do some complicated stuff, so I'll suffer it.]]

After that, it was simply a matter of seeing what needed recompiling to fit in with the numerous upgrades that were in 10.2 (kernel, KDE, QT, libraries etc.). I didn't even bother to check whether the proprietry NVidia drivers I use (the only low-level non-source-code thing I have installed on this machine) would need recompiling as I was certain they would. This went without a hitch and then I was able to boot into X-Windows on this new system and "see what else had broke".

That turned out to be the driver for my never-used wireless card (which is used for purely experimental purposes as I don't generally trust wireless in any way shape or form, ever since I demonstrated to myself just how easy it is to crack WEP and really interfere with most wireless networks whatever the encryption... the only access allowed over the wireless interface is a public-key authenticated SSH). That caused me some minor problems as it was complaining about the module config of the kernel (a missing file normally attributed to having kernel source lying around which didn't actually compile the kernel in question). This turned out to be a bit of a false alarm as the module would load normally anyway and worked perfectly.

Other than minor issues and a bit of user stupidity, there were no problems. Strange, for an upgrade that's the equivalent of going from Windows 95 to 98, or NT4 to 2000, that there was so little upheaval and virtually zero compatibility problems. And yet, my primary partition was never in any danger and with a small boot from the CD and a tiny LILO tweak, I'd have been back as if I'd never touched the machine.

I have now also tried Knoppix 4.0 on my laptop, similarly impressed that it needs absolutely no prior knowledge of what system it's going to run on. I'm amazed at just how much of the stuff that I have bought or been given "just works" in Linux, at worst requiring me to hunt down a small GPL driver from somewhere:

The QX3 microscope I found at a boot sale.
A cheapy PSX->PC USB controller adaptor (though games are not the focus on this machine, it was just out of interest).
A cheapy USB hard drive enclosure, again found at a boot sale.
My Intel NetportExpress (from eBay) connected so that myself and my girlfriend can stop fighting over who gets to use what printer.
My cheapy graphics tablet that was bought on a whim and never really used

Admittedly, some things I deliberately research to ensure they work on both operating systems; my wireless card, my USB key, my cheapy, ancient printer, my sound card (went through two or three old, donated and junked sound cards which, while they all worked under Linux, all showed performance problems. In the end, I just bought a cheapy SoundBlaster Live off eBay to save me having to use software mixing).

I always knew that Linux had problems with certain pieces of hardware, back from when I first heard about Linux when I was just a lad. Now I think that 99% of the problems are sorted out and most systems "just work". Heck, even most winmodems can be persuaded to work in Linux, even if they do need a binary driver. So far, the only thing I've found that remains resolutely incompatible with Linux is a small USB-IrDA adaptor that was never bought to work with Linux and, though it's detected as a serial port, doesn't actually support all the IR protocols that it would under Windows.

Addendum: And now I'm being asked to install Linux systems for an unattended kiosk-style system and a few small specialised workstations running the QX3's inside some of the schools.

The perfect operating system

2005-06-01T18:00:00.000+01:00

As in other articles, this list will grow as I think of new stuff to add to it.

--- Software for the operating system will come as a single, standardised file. This file will be openly-readable by a suitable viewer/editor and be appropriately compressed.

--- Upon installation, each piece of software will be confined to it's own folder/directory within a system-wide software repository and not need to access anything outside that directory. Requests for system-wide configuration changes, such as file associations, notification of events, binding to internet ports etc. will be indicated by a specially-named file within a specially-named subfolder of the software.

These files are seen by the OS as **requests** for configuration changes. The OS will handle detecting these files and allowing the user to choose which piece of software gets to ACTUALLY change the system in the way it asks for. Facilities would be included to specifically BLOCK individual programs from requesting such changes, if necessary (to avoid spamming the user with bogus requests). Any changes in these files are notified to the user concerned to allow them to decide what to do.

--- No piece of software will be allowed any sort of access to another piece of software's folder. Each piece of software has a folder which will have it's own subfolders, each with a unique cryptographic hash stored for the purposes of detecting changes.

These subfolders will include: Programs (for the program's executables and required libraries), Configuration (for program-wide configuration information and system-wide configuration requests) and Data (for additional data, such as user-created documents, frequently updated databases, temporary files etc.).

--- The OS will control software and the movement of data. Each user will have the same program folders present (although those seen by the users will depend on their security permissions) and each user can have their own unique configuration for each program (again, security policy allowing).

The OS will determine which user the software is running under and create links to the correct configuration for that user. This will be under a copy-on-write basis, with new users getting the basic system install and each user's changes stored seperately. Any unchanged files will use the system's defaults.

Similarly for data, each user will have their own data directory for each piece of software.

The OS will also be responsible for detecting shared libraries between software (using name, internal information such as company, version number etc., cryptographics hashes and so on). Common libraries will be detected by the OS and links created to a system-wide libraries directory, after gaining permission from a suitable user to do so.

--- There will be OS facilities for users to transfer data from one piece of software to the other, but programs will NOT be able to do this themselves as they will NOT be able to access any other software's folder. Ideally, these mechanisms will be transparent and a user will see a "Home" folder which contains all of their data files from every program. These will be links to invidual files within a software's Data directories.

The user is free to arrange their Home folder how they wish... links will stay intact, unneeded files will be hidden from view. By selecting a file, they are able to open it in the software in which it is currently residing or in another suitable piece of software. The OS will handle multiple programs opening the same file in the same way as current OS's use hard links, that is the file is only present ONCE but it's entry may appear in multiple software Data directories. The OS will only show a single copy of any file in the user's Home directory, though. Once a non-default program has finished with a certain item of data, the link is removed from it's Home directory, unless there is no other link to that file.

--- As part of the "file presence" system-wide configuration request mechanism, programs may request the use of specialised libraries and other programs. If granted (either by security policy or by appropriate user permission), the software is allowed read and execute access to that program, it's configuration and it's associated data, again taking account of the user it is running under.

--- Uninstallation of software is as simple as removing it's folder or moving it outside of the software repository. Orphaned configurations/data would remain present in each user's Home directory.

--- The OS will provide a security facility for all password/passkey entry. Whenever a password/key needs to be entered, the user will select the password/key field in the softare requesting it. Upon this selection, the OS will clear the screen and place a special, unfakeable, unchangeable indicator in a position on the screen.

While the system password screen is active, no other program may display any component of itself onscreen and all drawing requests are done off-screen. No other piece of software will be able to overlay or display this indicator but the OS itself. When this indicator is lit, all input device control is passed to the OS but NO OTHER PROGRAMS. The password is entered by the user (or the file/device necessary is selected), then sent directly to the software which provided the initial password/key field.

Additionally, the system security screen will clearly show the name of the software initiating the request for a password/key, the reason for it's request and any username or other information required to provide the correct password/key.

Rules for software

2005-06-01T17:40:00.000+01:00

A list of things that every piece of computer software do. This list will grow as and when I think of stuff to add to it.

1) Context-sensitive help (such as the "What's This" style help), if implemented, should be defined for EVERY button/option, in much the same way as every single image on a website should have an ALT tag.

2) All software should include an "uninstall" option, whether by an operating-system-provided mechanism or by a simple program supplied with the software. The uninstall should NOT require a reboot (except on those OS's where it's the only way to do it), should kill any running instances of any part of the software, clean up after itself (removing itself and the directories it was contained in) and ask the user if they want to remove configuration information for the program, save games, other files it finds in it's install directories, etc.

If the user does choose to keep this information, it should be put into a directory of the user's choice, labelled by default with the name of the software. This information should include any and all registry entries and, at the user's option, removing or reverting back any file association changes etc. that were made by the software.

3) If a program requires a certain file, library, etc. it should explicitly check for it at install time and run time. If any required file isn't found at any time, a check should be made for all required files and a list printed of the files the program needs in order to run. Lower-numbered versions of required files are seen as incompatible and will make the software print the file AND version required. Websites and other sources of these files should be displayed if known.

4) New versions of a program should ALWAYS come with a small utility or facility within the program itself to convert an installation of an older version into a format compatible with the new version. Hence, older config files are translated into any required new format automatically by the software to the equivalent or a safer configuration. These "convertors" should always be run whenever a previous installation has been detected and should be installed no matter what. Users should be informed by the software of any regressions in new configurations.

5) ALL software should co-exist with ANY version of itself without conflicts. On installation, it should not overwrite any previous installation unless asked to although it may steal, for example, file associations from it's previous versions.

6) Any installation changes with system-wide effect must gain the approval of the user before they are executed and provide an option to skip that step. Unless absolutely critical to the running of the program, the software should run whether the users chooses Yes or No. This covers file association changes, default printer changes, URL handler changes, default action changes (such as auto-insertion of audio CD's etc.) and similar measures.

Electronics

2005-05-30T12:43:00.000+01:00

One of my interests has always been computers and I've always known the basic theory about how computers work, how the electronics operate etc. I know how electronic components work and what each does and even the formulae that are relevant but I've never been confident enough to actually dive in and start creating stuff from scratch.

I enjoy the Velleman kits from Maplins... ten minutes with a soldering iron and you have an electronic game or quiz buzzer or clock, but I could never follow how they worked properly. There was always some strange arrangement of components that confused me.

Recently, I was asked by my brother to make a little "disarming" pack for his Scout group for a camp they are going on. I made him one a few years ago that was basically a small circuit with no electronic components that the Scouts had to "disarm" without setting it off. It consisted of a buzzer and a battery. The buzzer was constantly short-circuited by a small resistor on a wire that stopped it sounding. When the wire was cut or the resistor removed, the short circuit disappeared and the buzzer sounded.

That was crude and simple and didn't really work very well apparently, and I was also concerned about how safe it was so had to test it a lot. A few weeks ago, my brother announced that he wants another one and I had enough warning this time to do it properly.

I found the Kelsey Park Electronics Club website, belonging to a school many miles from myself, and it's been a lifesaver, explaining things that I've never seen explained anywhere else, in a simple way, with lovely printable PDF's for the projecta. The author clearly knows his subject and it is a fantastic resource.

Using that website, analysing a few of the circuits and stealing a few of the ideas from there I've come up with a brilliant circuit for this year's camp and have already spent a small fortune at Maplin's buying the breadboards and components and testing it out. Now, if only I could find a louder but lower-power buzzer than the one that cost me a tenner!

Another Linux Desktop progress update

2005-05-30T12:10:00.000+01:00

It's been over a month and I'm sticking with Linux on the desktop. Too many things "just work" for me to go back, my data actually feels more secure than it did before and the computer does what I ask of it, poorly written software aside.

What poorly written software? Nothing too vital. I just wanted to play some movie trailers online with Opera but that's proving almost impossible, even with MPlayer-plugin, plugger et al. Nothing seems to make it work whereas FireFox runs it just as it should with the same plugins.
I've followed every page I could find on getting these plugins to work with Opera but they just display a blank box or throw up lots of stdout errors, or both. That's no big deal, I could just use Firefox, but I like the way Opera works faster for day-to-day browsing and is integrated with RSS, news, mail, etc.

The other program that was giving me hell was KPlayer. Some files it just would not display properly, displaying what appeared to be vsync problems (a single tear at a certain point on the screen while displaying video). I first noticed this while playing a DVD. Putting KPlayer into X11 rather than XV mode solved it but it took too much CPU. MPlayer GUI showed no such problems (despite KPlayer using MPlayer to display) but I didn't like it or any of it's skins.

I could NOT find any difference between how KPlayer and MPlayer were rendering the clips but KPlayer always looked different. Eventually, Xine came as a good middle ground, showing all the clips I want it to while not displaying any artifacts, using the computer's hardware acceleration to it's full and having a usable GUI.

Using ALSA-only for sound is a great leap forward and the only problems I get are the soundcard-sharing issues that are well documented and easy to fix with dmix plugins. I've tried those and they work perfectly.

I investigated KDE 3.4 for a while, even downloading the KLAX live CD (a new version of KDE on a SLAX LiveCD, based on Slackware). 3.4 seemed faster, more responsive, cleaner and more bug-free but I'm wary of upgrading it until I've made a tar.gz of my system as it is at the moment. That's another thing that I love about Linux... the low-level config is all runtime-determined and the remaining config is hardware-independent.

I can transfer this disk image to any other machine in the world and it would boot up (after running lilo) and work just as before, with only a few minor tweaks to support non-detected hardware. The point is that, unlike Windows, I could move this hard disk to any machine should this one go wrong and be up and running again in minutes rather than hours and use the exact same settings as I have now. You just CANNOT do that with Windows. Even changing your motherboard pretty much requires a reinstall but in Linux, it hardly cares about what motherboard you are running.

I've gone onto Linux 2.6 now, given the fact that it supports more hardware, has ALSA built-in at the kernel level, has lots of bugfixes and new features. It meant that I had to tweak the startup scripts somewhat to enable me to dual boot and compile software for whatever kernel I happened to be running (mainly just symlink magic) but that didn't take long and was hardly necessary anyway... I think I've only booted back into 2.4 once.

KNemo has replaced my Zonealarm because now I have the power of an iptables firewall, I just miss the little flashing lights that indicated network activity. :-)

SSH logins are now working flawlessly and, following advice from a number of sites, I've got non-root passwordless SSH up and running, with an su providing all the power I need. I do see this as a little unnecessary root had his own private key that was the only way to log in but it's working now. I can use PuTTY from work and log in to my home machine and laugh at all the login attempts I see bouncing off port 22:

203.73.40.40 only tried 2 times.
210.96.200.24 only tried 2 times.
212.19.84.19 only tried 2 times.
210.204.129.27 only tried 3 times.
211.160.17.13 only tried 3 times.
212.160.93.188 already blocked for 5 attempts.
61.16.165.148 already blocked for 5 attempts.
70.183.189.207 already blocked for 5 attempts.
210.178.215.221 already blocked for 6 attempts.
211.169.117.119 already blocked for 6 attempts.
211.248.77.194 already blocked for 6 attempts.
213.179.250.115 already blocked for 6 attempts.
218.108.89.205 already blocked for 6 attempts.
61.219.67.127 already blocked for 6 attempts.
66.14.195.102 already blocked for 6 attempts.
202.180.175.138 already blocked for 7 attempts.
61.218.8.110 already blocked for 7 attempts.
202.127.24.198 already blocked for 8 attempts.
211.112.77.28 already blocked for 9 attempts.
211.114.170.161 already blocked for 9 attempts.
203.199.69.160 already blocked for 10 attempts.
201.144.107.203 already blocked for 11 attempts.
213.21.187.186 already blocked for 11 attempts.
202.141.12.146 already blocked for 12 attempts.
84.45.142.57 already blocked for 12 attempts.
220.65.55.130 already blocked for 13 attempts.
202.201.0.246 already blocked for 15 attempts.
210.51.12.238 already blocked for 15 attempts.
211.114.177.138 already blocked for 15 attempts.
211.114.195.7 already blocked for 15 attempts.
62.76.207.201 already blocked for 15 attempts.
163.27.7.2 already blocked for 18 attempts.
81.190.223.110 already blocked for 21 attempts.
64.49.222.180 already blocked for 24 attempts.
222.53.117.195 already blocked for 28 attempts.
210.51.25.217 already blocked for 32 attempts.
202.183.229.200 already blocked for 34 attempts.
66.82.4.25 already blocked for 39 attempts.
211.147.7.88 already blocked for 43 attempts.
212.89.111.97 already blocked for 44 attempts.
62.65.85.126 already blocked for 44 attempts.
81.214.131.217 already blocked for 44 attempts.
61.220.76.58 already blocked for 57 attempts.
220.130.3.60 already blocked for 61 attempts.
212.251.61.243 already blocked for 77 attempts.
211.250.189.5 already blocked for 89 attempts.
217.160.130.112 already blocked for 89 attempts.
24.10.130.62 already blocked for 89 attempts.
80.190.249.210 already blocked for 89 attempts.
82.235.174.242 already blocked for 89 attempts.
12.26.192.66 already blocked for 108 attempts.
165.229.65.86 already blocked for 108 attempts.
64.208.57.92 already blocked for 108 attempts.
81.208.31.177 already blocked for 109 attempts.
203.177.36.178 already blocked for 114 attempts.
80.183.225.131 already blocked for 122 attempts.
219.101.165.151 already blocked for 154 attempts.
195.70.198.199 already blocked for 216 attempts.
66.236.248.139 already blocked for 216 attempts.
211.26.36.3 already blocked for 221 attempts.
128.252.171.31 already blocked for 348 attempts.
211.172.225.111 already blocked for 382 attempts.
211.144.101.203 already blocked for 413 attempts.
61.193.182.107 already blocked for 425 attempts.
218.149.85.18 already blocked for 467 attempts.
202.30.198.245 already blocked for 519 attempts.
137.195.182.25 already blocked for 701 attempts.
213.251.132.146 already blocked for 702 attempts.
218.179.255.249 already blocked for 1409 attempts.

That's the output from my custom script which I add new features to whenever I feel like it:

#!/bin/sh
#
# Script to search logs for SSH brute-force attempts and block IP's

# Log a message saying we have started.
logger -t SearchLogs -p cron.notice -- Starting Log Search...

# Search through /var/log/messsages for "Failed" lines from SSH.
# Strip out IP from each line.
cat /var/log/messages* |grep sshd |grep Failed\ password\ for | sed s/.*from\ // | sed s/\:\:ffff\:// |sed s/\ port\ .*// > /tmp/ssh_attempt_ips.txt

# Similarly for "Invalid" lines
cat /var/log/messages* |grep sshd |grep Invalid\ user | sed s/.*from\ // |sed s/\:\:ffff\:// >> /tmp/ssh_attempt_ips.txt

# Similarly for "Did not recieved identification string" lines
cat /var/log/messages* |grep sshd |grep Did\ not\ receive\ identification\ string\ from | sed s/.*from\ // |sed s/\:\:ffff\:// >> /tmp/ssh_attempt_ips.txt

# Sort IP's and weed out any duplicate lines to rid us of multiple IP's.
# Also, add a count of each IP so that we can judge how many attempts
# they had and strip out some whitespace.
#
# Also, ignore any line containing 212.85.1.15 as that's our main IP for
# logging in from, similarly for 127.0.0.1.
cat /tmp/ssh_attempt_ips.txt |sort |uniq -c -d |sort |sed s/\ */\ / | sed '/.*212.85.1.15/d' | sed '/.*127.0.0.1/d' > /tmp/ssh_attempt_counts.txt

# Remove whitespace from beginning of line, place | in between counts and IP's.
cat /tmp/ssh_attempt_counts.txt |sed s/\^\ // | sed s/\ /\|/ > /tmp/ssh_prioritised_list.txt

if [ -f /tmp/ssh_prioritised_list.txt ]
then
# For each line...
for BAD_IP in `cat /tmp/ssh_prioritised_list.txt`
do
# Strip the count from the IP...
COUNT=`cat /tmp/ssh_prioritised_list.txt |grep $BAD_IP | sed s/\|.*//`
IP_TO_BLOCK=`cat /tmp/ssh_prioritised_list.txt |grep $BAD_IP | sed s/.*\|//`

# If a particular IP had more than 5 goes...
if [ "$COUNT" -gt "4" ]
then
EXISTING_LINE=`iptables -n -L INPUT |grep $IP_TO_BLOCK`

# Add to permanent blacklist.
echo $IP_TO_BLOCK >> /etc/ssh_blacklist.txt

# If it's not already on the firewall blocklist
if [ -z "$EXISTING_LINE" ]
then
# Print out a message and add to firewall.
echo Blocking $IP_TO_BLOCK for $COUNT attempts
logger -t SearchLogs -p cron.notice -- Blocking $IP_TO_BLOCK for $COUNT attempts
logger -t SearchLogs -- Blocking $IP_TO_BLOCK for $COUNT attempts
echo iptables -A INPUT -s $IP_TO_BLOCK -j DROP
iptables -A INPUT -s $IP_TO_BLOCK -j LOG
iptables -A INPUT -s $IP_TO_BLOCK -j DROP
else
echo $IP_TO_BLOCK already blocked for $COUNT attempts.
logger -t SearchLogs -p cron.notice -- $IP_TO_BLOCK already blocked for $COUNT attempts
fi
else
# Just warn.
logger -t SearchLogs -p cron.notice -- $IP_TO_BLOCK only tried $COUNT times.
echo $IP_TO_BLOCK only tried $COUNT times.
fi
done
else
echo "Can't read /tmp/ssh_prioritised_list.txt"
fi

cat /etc/ssh_blacklist.txt |sort |uniq > /tmp/ssh_blacklist.txt
cp /tmp/ssh_blacklist.txt /etc/

# Log a message to say we've finished
logger -t SearchLogs -p cron.notice -- Log Search Ended.

#---------------------------------------

I have that running as a cron job and it produces the output you see above. It's amazing how many attempts you get. Looking up the IP's on DShield shows that I'm not alone on being attacked from some of these IP's. It's tempting to move the SSH port just to avoid the log spam.

Overall, thoroughly impressed. I wouldn't class it as a perfect system but it's a damn sight closer than Windows ever was and when you consider that people who are giving their time and effort away are doing a better job than the largest company in the world pumping billions into research, you have to ask yourself whether you've ever been right to give them money.

Some computer annoyances.

2005-05-30T11:39:00.000+01:00

I've had a few pet peeves about how computers work for many years and I thought I'd air them here. Firstly, login procedures.

Why are password dialogs not more secure? How many times have you had some program pop up and ask you for a password while you are in the middle of typing, you've not noticed and pressed some keys and Enter in the course of your work? Or how many times have you been in the middle of writing a password when something else pops up or shifts the cursor focus so you end up typing the password somewhere completely different? (Hotmail's login dialog comes to mind because sometimes it would change focus if you typed in your password before the scripts on the page had finished loading, meaning you typed your password on the login name box for all to read).

In the same way that Windows has the "press Ctrl-Alt-Del" to log on, in order to ensure that your password can only go to the one program not affected by Ctrl-Alt-Del, i.e. the login dialog, shouldn't we have some facility within the OS itself to prevent other passwords being scattered willy-nilly when they are entered?

Maybe a system-default password screen. If the user needs to enter a password for any purpose, they have to click on the password box itself, which is just a large rectangular link. When that is clicked, the screen is blanked, the keyboard is locked to every program but the login system (so preventing software key-loggers) and a password dialog appears asking you to "Type in the password for Opera 8.01 to access the website www.hotmail.com" or "Type in the password for Control Panel to access your network configuration" or "Type in the password for SSH to login as the user root on remote system server.house.com".

This screen would also be secured by the operating system somehow, to ensure fake screens could not be generated (even though they would require user-interaction to do so). This could be by some sort of visual cue which cannot be modified or overlaid by any program by the system itself (in a similar vein to the secure icon in web browser which SHOULD NOT be able to have a website overlay it's image) or by using the clever tricks that people are suggesting web browsers should use such as a customised screen which nobody but the system has access to... therefore the program would have to know what the user's customised login screen looked like. Any deviation from their picture of The Simpsons beating up a computer and the user would know that it was a fake login box.

[Incidentally, I once used a technique many years ago, before the web was popular, to obtain an admin login at my old school. Using Visual Basic and the PrintScreen button, I was able to replicate the look and functionality of the RM software login screen that the school was using at the time, even down to the help dialogs and pixel-perfect positioning, and got my sixth-form computer science teacher to log into the machine. It was set up so that it failed consistently with any login but wrote the username and password to a file.

The computers at the time had a problem that they would fail to log in with the exact same message if they had been disconnected from the network since they booted up, no matter if you reconnected them later on. The way to fix it was to reboot from the login screen. My program simulated this functionality and was run from within any non-privileged user account. Once the admin had typed their username and password, they got the error they thought meant the computer had to be rebooted, they would reboot, come to a GENUINE login screen and log in successfully. However, the username and password they had tried would be logged to a plaintext file on the user account I had run the program on.

Honestly, I let the person in question log in, then immediately told him what I'd done. Stupidly, he had challenged everyone in the sixth form to try to get admin access, seeing it as a learning experience, and had foiled several previous attempts to steal his username and password. The only other success was when someone read his password over his shoulder and then created dozens of admin accounts using it, using their own name. They were quickly spotted and caught. Mine wasn't and would never have been as multiple logins from the same username were allowed and I would have had time enough to circumvent the measures in place which showed up admin users on the system (an easily subvertible two-line "cron job" that showed all admin users on a spare monitor in the server room)]

Ideally, there would be a fourth LED on your keyboard, indicating "secure" status which can ONLY be toggled by the operating system itself, not application software or device drivers. This would light only when the REAL login screen was up and people would be trained to realise that if the light's not on, people can read their password.

This should also stop the problem of accidentally writing your password at the end of the username box. Either a special button or key combination (even as simple as tabbing to the password field in question and pressing enter) would take you to the secure password screen. The light on the keyboard would be enough to show non-touch-typists that they are in the password box, the blanked screen for the login procedure enough to show touch-typists that they are in the right place.

I do consider password showing on the screen to be quite a weak link, though not completely unstoppable. For a busy user, though, especially around kids or teenagers, this would be a great way to stop unintentional password theft.

Another update.

2005-04-22T19:40:00.000+01:00

Following on from my previous post, another update:

Kplayer & Mplayer are working perfectly now that I've got the codecs installed properly, have acceleration working with the right options and I now know to always use Alsa for sound wherever possible. I have one spurious AVI file which identifies itself as DivX which won't play properly under any of the codecs except one but that one's not automatically picked. Selecting that codec manually makes it work and it's only one file that does this.

I've associated most of the formats with their respective players. Upgrading to the just-released Opera 8 was simple enough, no glitches and it's much faster and more stable, although it was too clever for it's own good and tried to use the Win32 Opera directory from my previous Windows install to get its Java from.

The first Java site I visited, I noticed and a quick whereis pointed me to a seemingly good java directory which Opera refused but then managed to find some sort of clue and suggested I try another. The suggestion worked and Java runs better than ever inside the browser.

I've now bought Crossover Office because it does what I want it to, was cheap enough, though I'm not sure about the subscription module for update. However, I was paying for one piece of subscription-based software on Windows and at least if I stop paying for Crossover, I get to "keep" the last version. Crossover runs my Word 2000 without problem, also my Irfanview tool (at least for all the bits I've tried that I would ever need). I'm going to try JASC Paint Shop Pro 7 on it at a later date.

I've configured the login screen the way I want it and this way it has a simple "shutdown" button that my girlfriend can use to turn it off if she's the last to bed at night. Before it needed a console login and a shutdown command and she's not familiar with non-GUI systems.

KPlayer issues solved, I've set it to be the default player for every media file format and I've also tested GSpot in wine and it runs perfectly.

Sound was my biggest problem so far. SDL games were sometimes perfect, next time they were run the sound would lag by over a second. That was unacceptable and annoying. Eventually, after much Googling, an upgrade of SDL, a tweak of ALSA settings to enable basic hardware mixing (I'm not brave enough to attempt the dmix plugin yet and don't really have a need for it) and most importantly, disabling the aRTs system. This hasn't lost me any sound that I would like to keep and stops the sound that I do want from lagging so inconsistently. This is something that I hope upgrading to a later version (whether or ALSA, KDE, aRTs, or even the linux kernel) will fix because it's plainly a problem.

I've upgraded packages (simple) and also installed remote passwordless SSH (more difficult) by a combination of Googling and guessing. Already I've had a few SSH login attempts using basic passwords from various places in Taiwan and China but I'm not paranoid enough to worry about them. Passwordless-SSH was more of an educational distraction than a real need. And, yes, my ADSL router was forwarding unknown ports to a no-longer-existent local IP address instead of to my main machine (which used to have it's own firewall). A quick tweak of the settings and SSH showed up externally as responding.

Further on my quest for "work distractive technologies", I've played a bit with TuxRacer and it looks like 3D acceleration is working properly... :-)

Still fighting with the bloody KDE clock as it just insists on pissing about when set to a timezone that uses BST and says it's applied changes that it doesn't. Wonder if KDE 3.4 has fixed that issue?

Still no problems or major disappointments as of yet but rest assured that I'm hunting for some more show-stoppers..

Linux Conversion Update

2005-04-18T09:43:00.000+01:00

Okay, so I've been running Linux for a few days now. I've managed to get all of my necessary apps and most of my "useful" apps working properly.

KDE provides most of the tools natively but I had to install some things like MPlayer (and Kplayer, a GUI for it), K3B and a few others. I've also downloaded Crossover Office having been disappointed at the conversion capabilities of AbiWord and KOffice (through no fault on the behalf of their authors, I want to add). I haven't got around to installing OpenOffice at the moment and Word 2000 is about the only thing that I will admit that Microsoft has done right so I have no problems with paying money to get that to work. It saves me time playing about, ensures compatibility and means I won't strip or corrupt important info in my DOC files. Similarly for Excel 97/Gnumeric.

Crossover + Word is working perfectly, not a sign of a glitch, but Wine isn't up to running much else that I want at the moment, but luckily most of what I want has nothing to do with actual work. :-) It does run Irfanview, Dreamweaver and Paint Shop Pro, though, and DW and PSP I consider to be my vital apps. I have yet to test it out on every feature of those programs.

I've also installed the nVidia drivers which sped up X's 2D drawing noticeably. That was the only time I needed to shut down X and restart. I'm still using a console login at the moment because I've never been afraid of a command line and I like to see what's been going on at boot before I start up a complex program like X.

Linux still hasn't crashed once so it's definitely better than my previous setup, and even X hasn't crashed out yet (which, from previous Linux experience, I was expecting, especially with a binary display driver).

I managed to download a Knoppix torrent and burn to a CD within an hour of deciding to do it, without meeting any unpassable shortfalls or annoyances along the way (just installed BitTorrent and K3B from LinuxPackages and it all just worked). In fact, K3B is better than most CD-Writing software I've used on Windows and I've pretty much used them all.

KPlayer/MPlayer was a little more tricky but hardly a vital program. The package I was using was missing a few symlinks, which were easily created, a little bit of configuration and I'm only having trouble now with one MPEG-4 avi that seems screwed in MPlayer but works in another, unaccelerated, media player that came with Slackware. I'm assuming that the Win32 codecs are faster but not as well supported as the OS codecs that came with Slackware and that a slight override somewhere will cure this. While fixing this, I did notice that I quite miss using GSpot, a codec-discoverer for any avi/mpg/etc. file. I'll have to find an OS equivalent or get it running under Wine.

Installing new and missing software has been a breeze thanks to LinuxPackages.net, not that I've ever been scared of compiling my own, it just makes it so much easier to keep track of what program is where. Thanks to Slackware's plain tgz packages, if I lose track of where the "executable" went, I can just browse the package in Ark and look for it. That's worth its weight in gold and useful for those packages that didn't have quite as much attention paid to their creation as others on LinuxPackages and might be missing a KDE shortcut icon or similar.

I've been upgrading a few libs like SDL et al and not run into any problems yet, in fact I can say that it's only made things work better. I'm looking at the prospect of upgrading to the latest -current Slackware packages as they include the latest version of KDE. I'm not too worried about the rest of the updates as they are mainly bugfixes and security updates. The computer is firewalled and not running any internet-bound services and the only bugs I've run into are Konqueror crashing a bit more than it should. It's hardly a problem as it doesn't even take down other instances of itself, let alone X or the OS, and a click on the Konqueror icon gets it straight back up.

I haven't managed to test remote SSH yet and I'm a bit worried that my firewall might be being a little overzealous and blocking it as the port appears "stealthed" to any web port-scanner. I'll have to see if that's because of the firewall, the way I've set it up or the SSH config. I've noticed this same problem on another 10.1 machine with this firewall so I will have to look into this. Internet access from the computer itself, though, works just fine, even over P2P and torrenting.

I've been trying out a couple of Linux games and they've been quite fun so far, just the sort of games I like, small, fast, not too fancy but fun. Am missing my other games a bit but knew I would be and will have to wait for a new PC for gaming.

Have yet to try the DVD but I see no reason why it should not work as MPlayer can play MPEG2 files off my hard disk and the computer can see the DVD drive, so there shouldn't be any problems. I have updated packages for libdvdcss etc. on standby just in case.

Had a few teething troubles with the KDE clock as it really messes up when you select timezones, whacking hours on and off of the clock a few seconds after selecting them and simultaneously resisting most attempts to stop it being too clever for it's own good and adding hours for BST etc. In the end, I just set it to a non-timezone until I can be bothered to fight with it again.

Altogether, it's been pretty painless, I haven't lost any work, my main apps are up and running and there's not much functionality that I didn't have before. The computer is more stable, turns itself off EVERY time I ask it to, boots up without issues or any error messages and runs most of the existing hardware (with the only exception being the old scanner) without me having to do anything at all.

Also, I've noticed the benefit of using a local network, printer-server and ADSL router again. To be able to have a network where everything important like printers and internet access are done over a standalone networked device is a real lifesaver but then, Slackware would easily handle my printer and any sort of NAT effortlessly. I have also recently set up a Slackware machine for my brother which does all of the above in one machine so that he just plugs in any computer and it all just works (firewall, NAT, printer sharing, Samba storage, etc.) once it has an IP. This has greatly aided his transition from a 500MHz 98 machine to a new mega-gaming-machine XP.

My entire changeover has gone mostly unnoticed by the other user of my home network, namely my girlfriend. The only comments she had were, when she used the machine once to save turning hers on, that it didn't have a seperate Opera icon for her like the old one did, with her bookmarks and emails. That was because I'd forgotten to transfer her settings across when I'd done mine. That's easily fixed, as all the old Windows drives are still present and mapped into the filesystem and, thanks to Opera being cross-platform again, it's simply a matter of copying them over to the right place.

The other comment was that she couldn't play PartyPoker on it, her favourite online game, because that's a Windows-only piece of software. I had tried that in Wine myself but it really didn't like it at all. She'll just have to stick to her own computer for that, which I knew she would.

Overall, quite happy with it so far and still waiting for the first showstopper.

Plunge Taken...

2005-04-14T13:00:00.000+01:00

Bandwagon firmly landed on with both feet...

Windows decided to play up. I thought to myself "I can fix this". Then I thought, "Why bother? This is my work machine and it should ALWAYS be up". Then I installed Linux. I now have Linux as my primary desktop.

Things I will miss:

- Games (but may well invest in a cheap XP machine for those)
- My plethora of "essential" programs (no more Zonealarm icon flashing away reassuringly, no more need for specialist programs like NAT32, virus scanners, spyware detectors etc.) which have become obsolete or unnecessary.

Things I won't miss:
- Bugs
- Blue screens
- Viruses (Only ever had one, personally, from a respected PC Games magazine CD)
- Spyware (Never had any but always kept checking)
- Endless drivers

I've moved onto a Slackware 10.1 system running KDE and it's working just fine. I plan to use it for work mainly, and to provide a fault-free stable system for the next few years. Already browsing the web, rss, irc & emailling (having Opera be multiplatform is a lifesaver and greatly helped the transfer), icq, msn, yim and aim (thanks to Kopete, the linux equivalent of Trillian without the ludicrous upgrades and skins), printing (via CUPS and the lovely people at linuxprinting.org for the PPD), access to all my parititions, a firewall, a version of PuTTY (yes, I know it's just an SSH frontend but I liked it on Windows and I'm used to it now).

Considering it's running on a plain VESA driver for now, it's actually faster than even my finely-tuned 98. Have still to set up my CD-RW and DVD-ROM but don't see them being a problem, using k3b and mplayer. My scanner is linux-incompatible but I have two others sitting under the desk that are 100% compatible, so just have to re-cable that. My "weird" hardware like my cheapy-RAID card, cheapy USB stick, USB IrDA, Intel QX3 are already supported and auto-detected without me having to touch anything. Will have a look see how hard it is to connect to my Nokia via IrDA and use my card-reader at some point but that's hardly a priority.

Collateral damage is minimal so far, just a lilo change to boot Linux by default. All my flaky FAT drives are still there and accessible. I am considering investing in Crossover Office to run my Word 2000 and Excel 97 combo and possibly even things like Dreamweaver but for the moment, KOffice is holding the fort.

Considering that 90% of my use of the computer is Web, Email and IM, the impact has not been too bad, it took minutes to get up and running with the exact same version of Opera that I was running on Windows and import all my stuff over. Scroll wheel on my mouse and the occasional segfault due to not having any swap were very quickly cured and I haven't managed to crash it since.

I need to switch on APM/ACPI but I haven't tried that yet. Normally a "modprobe apm" does all that for me but it appears to be missing so I will try and track that down. When it didn't work, I was too busy trying out all the silly card games to care. :-) Worst case scenario is that I recompile the stock kernel that Slackware provides to something a little more relevant. The only difference that that gives me to my old Windows 98 is that now I don't have a pretty screen up when I have to turn it off saying "Windows is shutting down..." :-)

I've decided to allocate one month of time to it, to see how I get on with it. I've resigned myself to the fact that it will not run my games but I may well be able to find emulators for my favourite older systems (Spectrum etc.), use things like DOSBox to run some of my older titles, and anything DirectX/OpenGL I can use on some other computer. That should be enough to distract me and I can use an XP machine as a games-console only.

The programs I have yet to find a suitable replacement for are Paint Shop Pro 7 (nice, simplified interface around a powerful image manipulation program), Dreamweaver (nothing quite like it), and a few tiny utilities I like to use.

I'll see how it goes and see whether I can hold of a nice games machine for myself. My ideal aim is to have a Linux desktop for work, browsing, email etc. and only power up a Windows XP machine for games, literally using it as a games console. Even then, what I want to do is make CD images of all my games and mount them over a Samba share via Daemon Tools on the windows side so that I don't have to track down every CD for every single game every time I want to play it. The samba share would be held on either the main Linux machine or on a small Linux storage server with a mini-RAID on it.

The "Perfect" Virus

2005-04-07T12:03:00.000+01:00

On the theme of my earlier article on viruses, I'd like to touch on how a perfect virus is not only possible but also, if ever created, extremely nasty to stop.

PLEASE NOTE: I do not condone the creation of viruses, nor of any program which resides or gains entry to a computer in illicit ways, including spyware, adware, forced updates, etc. This article is targetted at a particular operating system, notably the one most prevalant today and that which is subject to the greatest number of virus attacks... (two facts which are not, despite popular belief, cause and effect)... Windows in all its flavours. Most of the ideas listed below WOULD NOT WORK for any other operating system and the purpose of this article is to highlight just how Windows leaves itself open to mass destruction, and also to just how tame current viruses are compared to what they could be. I accept no responsibility if some idiot uses some of these ideas, in the same way as someone who says "the security on that building looks weak... someone could walk right in there and slip through unnoticed" has no responsibility if someone actually DOES do just that.
==========

Viruses in the last few years have changed tack. At first, the early DOS viruses merely spread over floppies, infecting already-present executables (because of the lack of any autorun facility in the operating system), the startup files or the computers boot sector. Later they became more sohpisticated and started to not only infect executables but to hide themselves in such a way that they could not be detected easily. These early viruses were highly destructive... reformatting hard drives, destroying hardware, even taking out the CMOS settings.

Then Windows become more prevelant and the viruses changed tack. No longer were they destructive, no longer did they have to hide themselves so well or use particular avenues to infect the machines. Network shares, badly coded files leading to security vulnerabilities, scripting and macro languages with WAY too much power over the computer are all open doors as far as a virus is concerned. And now, the viruses could turn your PC into a valuable commodity that would sell on the black market, useful for spamming, covering tracks, attack remote machines and taking down large corporation's websites.

At the same time, virus scanners were updated to cope. Nowadays hardly any virus scanners actually take control of your machine from boot but instead rely on Windows to initalise the anti-virus program. This, however, is a reversion to previous, bad ideas. The best anti-virus scanner is one that runs as the sole program in memory. Other protections were enabled though that did help, such as the BIOS options to protect your boot sector and guard against unauthorised BIOS flashing, Windows and Internet Explorer in particular were updated to stop scripting attacks working so well from remote websites.

Soon, traffic coming over the network cable could exploit the holes in the operating systems services to take control of a machine without even having to be on the hard disk or in RAM already. Firewalls stopped the majority of those attacks but lots of people who were unprotected were caught out by the early variants. Running Windows Update would have prevented many of these attacks but not all and, besides, what use is a cure if you have to open yourself up to attack to obtain it?

So, what would the perfect virus look like?

Hopefully, a number of infection methods would be available. Rather than just relying on a single hole through which to gain access to a machine, several methods would have to be available. The more entry methods, the greater the chances of spreading. Executable size is rarely an issue nowadays, what with broadband connections, so many methods could be implemented in a simple executable.

Secondly, detection avoidance would be a must. If the cause can be discovered, it can be cured. Having got on to a target machine, a passive analysis of it's hardware and software should be conducted. Does it have anti-virus? Are the BIOS options for boot-sector protection turned on? Where can I hide?

After the passive analysis, hopefully we still wouldn't have triggered anything that would alert the user to our presence. But how would we stop things like AV packages finding us? For that, we'd need to revert to the older tactics. AV packages, on the whole, rely on file "signatures", little pieces of identifying strings that match against a particular virus.

We'd have to be signature-less, as best we could. In the olden days of viruses, we had polymorphism: the ability to change size, shape and contents and yet still be the same executable. Look around your hard drive. Somewhere there will be files which are designed to hide their contents, be they encrypted, compressed or obfuscated. Programs like UPX (an executable compressor) are commonly used in viruses to reduce the size but for the same executable they would give the same compressed signature.

There are also such things as encrypted files. One of the lesser-known goals of encryption is to make the resulting file data appear as random as possible. Encyrpted files have no consistent signature between similar contents. A tiny change to the program would render any signature useless.

However, we would need a "key" with which to decrypt ourselves. That key could well be changed at will by the program itself as it transmits itself between computers, reencrypting its raw contents with the new key each time and somehow transmitting the key and the encrypted data inside itself. The code that it would use to decrypt itself would also be a signature, as that could not be encrypted if it were to be executed. Somewhere down the line we'd have to give ourselves away eventually.

Or would we? Surely a method could be devised where the decryption code is some kind of standard file access, such as the decryption engine of a ZIP program, for instance, or some other signature which couldn't be hunted without also turning up innocent programs. Nobody would trust AV vendors if there software suddenly started spouting out virus warnings for a perfectly innocent file. Maybe that could be a possibility. There are lots of genuine program out there that have a built-in decryption engine and some encrypted content inside an ordinary executable file. If we were to hide in one of those files, we could remain undetected at least until we'd had a chance to start executing our code, couldn't we?

Many programs misuse some low-level services within the computer to actually hide the files themselves from virus scanners, intercepting file system calls and substituting fake harmless data whenever the contents of themselves are requested by a program. Once activated, our program could well do this too. Why not? Any step to make it harder to remove would help it stay.

There are several places that a program can hide itself in to be automatically executed when the computer is started, the boot sector, windows itself, as a service or as another program that runs on startup. Why not, if we can verify that nothing is watching us doing so, try all of them. That way ALL of them would have to be cleaned up in order to stop us loading.

Most AV disinfection is done while the computer still has the virus present, i.e. while the infected computer is actually running. For a lot of viruses, the first thing you need to do is to terminate their processes in order to be able to remove their files. If you don't, you run the risk that the virus will notice this and try to compensate by reinstating all the entries that it needs to run.

So if someone is going to be trying to shut us down while we are running, why don't we spawn a few more copies of ourselves, as soon as we possibly can, and have them coexist. Have them watch each other. As soon as any one virus process disappears for no good reason, IMMEDIATELY reinstate another one, or even two. If you do it fast enough you will negate any manual attempt at shutting those processes down.

Sometimes, in Windows, programs are notified that they are about to be shutdown before they are actually terminated. Why not look out for these notifications and, if we get one, execute some code in the brief window of opportunity we are given to cancel the action, or spawn more copies of ourself?

With processes all watching each other, the user would have to be bloody quick to get rid of us all and may even need specialised tools or knowledge to do so. It might even baffle most automated programs that try to do so, at best sending them into an infinite loop where for every process they terminate, two more turn up. That's what the user gets for detecting us and trying to remove us. He'll have to do better than that.

On the subject of retribution there was a brief spate of DOS viruses that, when activated, removed a critical part of your machine's data, usually the partition table, and held it to ransom. "Co-operate with me or I won't put it back. Turn me off and you lose it forever." It might be an idea to do that while we're at it. Given the wealth of vital information on the computer, blackmail might not be a bad tactic for a virus to execute.

Setting vital computer passwords (such as the admin accounts) to random values, removing the password for encrypted files or disks, setting CMOS passwords or hard disk passwords if we think we know where they are stored. The ATA spec does include some facility for encrypting disks with a password that must be sent over the IDE cable before the disk is readable. If we could get that to work, that might be fun.

Of course, we'd only do this if someone tried to remove us. The worst case for a virus is that the computer gets formatted and replaced with a clean copy of Windows. We want to make sure that that is the ONLY way the user could remove us because we want to cause as much inconvenience as possible. That way, the virus gains a name for itself (which is actually a bad thing... much better if people aren't aware we exist) but also provides us with a fresh source of un-patched machines trying to get to Windows update before we can infect them via whatever hole Microsoft may have patched.

Obviously, though, we have more than one entry ticket up our sleeve and we can try a number of different ways to get in. In fact, it might even be an idea to use a plug-in style interface. Have the virus contact some location... or several such locations in case one is discovered and shutdown (file sharing networks, ftp and www sites, already-infected computers, there's no end of places) and download the latest "updates".

They could include plugins for new methods of infection to combat any patches that may become available, plugins for more advanced analysis of the computer to discover it's flaws, plugins which can update the update locations themselves in case the sites all get shutdown, plugins to change the way that the virus polymorphs itself, plugins to cater for more architectures (64-bit or even different processors), more machine types, plugins to take advantage of new technologies, new storage methods, new ways to hide inside different file formats, plugins that interact with your CD-Writing software to make any bootable CD contain the virus and/or any autorun entry to run the virus from the CD silently, plugins to infect over instant messaging network, plugins to swamp filesharing networks with fake files (or even better, real files) containing the virus, plugins to allow backdoor access, using UPnP or discovered passwords to disable or tweak any firewall protection blocking it. There is no end to the number of ways the virus could be upgraded, all of which would make it harder to detect and remove, easier for it to propogate, harder to block its attacks and harder to brush it aside as just a virus.

However, this plugin interface may well be a possible avenue for an anti-virus worm, a self-cleanser that would find virus-infected machines and exploit the virus itself to remove it instead of make it more powerful. Some sort of public-key-encryption signature (suitably obscured or encrypted within the executable itself) would be needed, to verify that any updates have been signed by the virus author themself. Defeating this would be a massive piece of work, involving defeating high-level encryption by working out the private key from the public key, something which only supercomputers are currently capable of. And if such an anti-virus virus should ever appear? Well.. send out a plugin update that changes the public-private key pair to those machines that still remain infected and on the Internet.

As people try to defeat the virus, hopefully new updates to it would ensure that the usual ageing process of a virus from obscurity to notoriety back to obscurity wouldn't occur. Instead of slowly dwindling away, more and more machines would succumb to the virus, taking advantage of every known hole as soon as it appeared. Those who had removed the virus and patched to the latest level would only be safe until the next plugin appeared. Generated from the few remnants of the virus remaining on the older, unpatched, unmaintained machines on the internet, the virus would reappear in new forms, taking advantage of new techniques which have taken account of whatever its previous weaknesses were.

The distribution of the virus would be controlled by a changeable plugin. Once one signature had been spotted by AV vendors, an update could be rolled out to change that signature so that the next generation of the virus would be different as it spread.

New plugins for analysis would recognise specific versions of the tools designed to remove the virus itself. The standalone removers would be blocked or fooled into thinking they had done their job, the major anti-virus tools would have to change signatures often to catch up with each new variant formed by the newer plugins. Newer virus versions would have detailed information on what virus scanners look like and how to defeat them, either by hiding themselves, faking some of their information, or, crudely, terminating the antivirus program (that actually seems quite amateur and noticable and we'd only want to do that if it were absolutely necessary for a particular version of that AV scanner/virus signatures).

Using the above, we'd aim to get onto as many machines as possible, whether we can sneak in past some ancient or poorly-crafted AV scanner or whether we have to get into the machine, terminate everything and attract the attention of anyone using the machine. Preferably, we'd do the first because we also want to stay on each machine for as long as possible, too. We want nobody to know we are there. We can sit dormant and wait for years for someone to hook us up to the Internet so that we can cause as much havoc as we can.

Thirdly, once we're on millions of machines worldwide, we need a purpose. Previous viruses have done everything from re-format the machine to send out spam, or even attack certain websites. Each of those is malicious and doesn't achieve much. Maybe the payload of the virus should be something simple. Demonstrate to people what you COULD have done. Pop up a message once a week showing what you could have achieved.

How's this sound?

"You have been infected with the Super-X virus. This virus was created with the goal of showing you just how insecure your operating system is and how the trust you have put into the company that made it, Company X, is misplaced. Company X left the following gaping holes in their software:

-A vulnerability caused by a buffer overflow in Service X.
-A poorly implemented software firewall which doesn't blocks packets of type X.
-A security oversight in the default settings of program X (created by Company X).
-A security oversight in the default settings of program Y (created by Company Y).
-Poorly checked security restrictions in module X.
-A failure of your anti-virus software, X, to discover my presence despite you paying for an up-to-date subscription, with your credit card XXXX-XXXX-XXXX-XXXX.

Using the above (and other) holes, I could quite easily have done any of the following without you knowing and without your permission:

-Formatted your hard drive, losing all your data.
-Send out spam email claiming to have come from anyone on the Internet, to anyone on the Internet.
-Attack any website that I chose, co-operating with NNN,NNN other infected computers globally, possibly resulting in a complete loss of that website.
-Stolen all your passwords and your credit card numbers from file XXX and website XXX and sent them to random, or selected, people on the internet for them to use to obtain further information about you.

Please inform any of the above software vendors if you are unhappy about my ability to do this so easily or, alternatively, vote with your feet.
"

It would become the most famous virus ever and, like the Millenium Bug and Black Monday, stick in the minds of users who might, just might, then ask for a computer that isn't susceptible to a virus the next time they buy one at PC World.

Can you imagine just what that would do to the Internet, to the way common people work and trust machines or companies and to the way people think about computer security?

Linux to the rescue

2005-04-04T13:22:00.000+01:00

How's this for a Linux advocation? A school wanted to use their Intel QX3 computer microscope (which have a tendency at all schools to fester at the back of some cupboard, despite being a brilliant idea). They wanted a kiosk-style setup so that the computer could sit in the corridor (not networked), the screen showed the microscope view, the kids could play with objects under it but not break the computer in doing so. They also didn't want to have to pay for Windows licenses or buy a computer just for that job.

Pulled out an old 300MHz that they were going to scrap, bunged a bog-standard Knoppix CD in and booted off it. Ran xawtv... voila! Instant full-screen microscope view without having to load a single driver, or even touch the hard disk. It appears that the driver for the Intel QX3 is just an ordinary webcam driver (cpia) that Knoppix has bundled anyway. No configuration required to get the image on the screen apart from running a TV program (also built-in to Knoppix) and pressing F for fullscreen.

Next week, if they are happy with what they have, I'll install it to a permanent location on the hard disk (apparently a one-liner under knoppix), set up a simple script to turn the lights on the microscope on and off (echo toplight:on >/proc/cpia/video0 etc.), maybe even have that controlled by a joypad or similar simplistic setup, and automate the boot so that xawtv goes straight to a full-screen kiosk mode.

Cost: Nothing. Extras required: Knoppix ISO image + 1 CD-R. Time Required: About 20 minutes. Expertise required: A google for "Intel QX3 linux".

Now try that with Windows.

Making your own Firewall / Router / Fileserver / Print Server

2005-04-04T10:49:00.000+01:00

As I have just done the above numerous times for myself and my brother, I thought that a little writeup might help. This is not designed to be complete or simple, but is for someone with a basic grasp of Linux and/or an advanced grasp of PC's in general.

I assume several things. You have a local Windows network. You have ADSL and an ADSL router that is connected to this local network. You know the basics of TCP/IP and are comfortable with command-line interfaces. You have a spare machine that has nothing of any value on it (e.g. Windows, documents etc.) that can safely be wiped and made to become a firewall/router/server. You have a spare parallel port laser printer (e.g. an HP Laserjet) that you want shared over the network. You know how to use Google and look things up for yourself if this article doesn't cover your problem.

First, get a computer and network it. I happen to have a source for old Pentium machines (233's) but for my brother's project, I gave him an old P400 with 64Mb RAM, two PCI network cards and an 80Gb hard drive - smaller harddrives are fine, down to about 4Gb, and may actually be more compatible with a machine with an old BIOS. 2Gb is an absolute minimum but it's then a struggle to keep up to date with such little free space.

[Author's Note: Yes, I'm being lazy and installing EVERYTHING from Slackware, I know perfectly well that you can do this from a single floppy but that's not the point. See Freesco if you want to be an expert and do it properly... I ran a Freesco install for about 4 years, making it do all of the above.]

The PC could really have done with a bit more RAM but it runs fine with just that much. Plug it into your network. At first, you can just add it onto an existing network by plugging into the hub/switch, to make installation easier. Later, we'll turn it into a router that can be put between your broadband connection and your computers to keep them much safer than they would normally be.

Next, blank the harddrive completely (or just be aware that you will be wiping everything on it, even Windows or whatever else is on there) and download Slackware. You can use any recent version but for me the version was 10.1. Download the first two ISO disk images and burn them to CD. CD Burner XP Pro is good for putting them onto CD if you don't have Nero or anything else to use, is free and runs fine under Windows 98 despite the name.

Next, boot the machine intended to be a firewall, file server etc. from the first CD. Follow the instructions until you get to the login prompt. Login as root and you'll be left hanging on a command line. Type cfdisk and delete any partitions on your hard drive (THIS WILL DELETE ALL YOUR DATA ON THAT HARD DRIVE, OBVIOUSLY). Then install one large partition over the whole disk and make it bootable. Exit out of cfdisk and reboot.

Boot back off of the CD again and login, then type setup. Go through the setup process (selecting a keyboard layout etc.), select the partition you just made to install into, don't set swap space, use the default kernel and make sure you install all packages, full install.

After a few hours, you should be done and rebooting without the CD in will take you straight into Slackware Linux. You would have been asked to set a root password. It's a bloody good idea and you'll need it to log in to this computer from then on. Now we have to set up your network card in order to connect to the internet and the rest of the network.

Login as root and type dmesg | more and scroll through, looking for anything that indicates that Slackware has found a network card. If it is a PCI or even an onboard card, it should find it without any assistance. Non-PNP ISA cards need a bit of a nudge beyond the scope of this article (Google isapnp).

If your network cards were successfully detected, type pico /etc/rc.d/rc.inet1.conf. This will allow you set IP's, network mask and (if need be) a gateway address (e.g. of an adsl router or any other device that all Internet-bound traffic should be directed through). You should set one card to an internal network address (e.g. 192.168.0.1 or 10.0.0.1) and the other card to an address which will be "external" (or enable DHCP for that card).

[For an example, my ADSL router gives out addresses in the 10.0.2.x / 255.255.255.0 range, which my server picks up as it's "external" address. The local network uses 10.0.1.x / 255.255.255.0 which the server has an address set for on its other card.]

Hit Ctrl-X when you're done.

After a reboot, both cards should now appear in the output of ifconfig as eth0 and eth1 (lo is a loopback interface that's always there, network or not).

Test your connection by ping'ing a website, ping www.example.com. This should come back with replies. Hit Ctrl-C to stop the pinging.

If you have two cards and the ping doesn't work, make sure you try this with a cable in both cards... sometimes the cards are chosen in a different order to what you might expect. Also, try pinging the address of other machines on the network, your ADSL router etc.

If you have a succesful website ping, it means that the Slackware machine is now able to connect to the internet. Now we can download a little program that will help us later on. Personally, I like Projectfiles.com's firewall... a simple script that enables all sorts of things without the complications of a GUI.

Type lynx http://projectfiles.com/firewall/. (Lynx is a text-based web browser that is simple enough to download files should you need them). Select the link for the latest stable release and hit d for download. Let it download it and then save it. It should have saved it in the home directory for root. Exit out of Lynx.

Copy it to /etc/rc.d (cp rc.firewall /etc/rc.d/)
Edit it to suit your network (pico /etc/rc.d/rc.firewall) using the output from ifconfig to remember which card (eth0 or eth1) has which IP address etc. There is a line in the file that tells you below which point you should not make any changes.

Make sure that you add the local network interface (e.g. eth0) to the list of internal interfaces. Any other network cards are considered external, i.e. anything that comes through them is NOT trusted, as if it were a direct connection to the Internet.

Press Ctrl-X when you are done.

Now, to turn the firewall on, you type chmod +x /etc/rc.d/rc.firewall and it will run at the next reboot. To turn it off, do the same with a -x instead and reboot. If you have done it correctly, you should get no errors and still be able to access the internet with ping etc.

While we're here, we need to turn a few other things on and off:

chmod +x /etc/rc.d/rc.dnsmasq
chmod +x /etc/rc.d/rc.samba
chmod +x /etc/rc.d/rc.cups

For these to work, however, they need some configuring. If in doubt, google for the name of the program or config file (e.g. dnsmasq.conf) and read their documentation.

DNSMasq
=======

This program forwards DNS lookup requests (i.e. resolving www.example.com to an address the computer can use) to an upstream DNS server. This is usually the same machine as your gateway if you have an ADSL connection (i.e. the IP address of your ADSL router) or it can be your ISP's DNS address (they will have these written down somewhere on their website but you may need to login to their website first... usually they can be found inside their instructions for connecting other types of computer to the internet, e.g. Mac, Linux, Playstation, XBox etc.)

Mostly, it needs no configuration at all but any can be done using pico /etc/dnsmasq.conf. It will get the IP of the upstream DNS server from /etc/resolv.conf which you can edit using pico too.

If this works properly, you should be able to set the other computers in your network to use your server's LOCAL ip address as a gateway and DNS server. E.g.

Internet (some address which your ISP has given you)
|
|
ADSL Router (10.0.2.1 / 255.255.255.0 and providing DHCP in that range)
|
|
10.0.2.100 / 255.255.255.0 on eth0
Server
10.0.1.1 /255.255.255.0 on eth1
|
|
Windows machines
10.0.1.x /255.255.255.0 using 10.0.1.1 as their gateway and DNS.

With this setup, the Windows machines should be able to connect to the internet as if the server wasn't even there. A quick trip to GRC to run their ShieldsUp! test should show all ports as Stealth, though.

That's routing, firewall and DNS-forwarding configured already!

Samba
=====

Samba is the piece of software that will allow you to produce network shares, which in Windows will appear as shares under Network Neighbourhood. pico /etc/samba/smb.conf to set it up. Generally, you should set the workgroup to the same for all computers on your local network, which you can do for your other computers using the settings inside Windows Control Panel. I use something like HOME or OFFICE for this so, for example, all the machines in the office have a workgroup of OFFICE. Also, for simple purposes, security=shareis fine.

You should also set a name for the server, which can be anything you like but which is probably better off as "server" or something simple. With just a name and workgroup configured, saving the file and rebooting (all this rebooting is *completely* unnecessary under Linux but it's quicker to type... :-)) the machine should appear under Network Neighbourhood of any Windows machine on the local network with the same workgroup.

If not, check whether you can ping the server from one of the Windows machines (type ping followed by the internal IP address in a command prompt) and also try disabling the firewall on the server (chmod -x /etc/rc.d/rc.firewall). If disabling the firewall works, check your firewall config... all machines on the local network should be able to have unrestricted access to the server if they are on a card which is listed in rc.firewall as being an internal interface.

Also, check any Windows firewall software you might have. Zonealarm, for instance, will sometimes through a wobbler when it sees file-sharing protocol traffic from the server and this may make it block the server... in this case sometimes the internet will work for five minutes or so and then stop. Add the server IP address as a "Trusted" IP address in Zonealarm on the client machines to fix this.

Now the hard part: read through the shares part at the bottom of the /etc/samba/smb.conf file and comment out any you aren't going to explicitly use (except for the printers one, leave that one there) and then make your own, following their example. If you want to create a basic, empty share using the hard drive space that you have left over on the server, that you can store stuff on from your Windows machines, it's best to do this:

Type mkdir /mnt/share. This makes a directory for us to use.
Type chown nobody:nogroup /mnt/share. This gives anyone permission to use that directory.
Type chmod +r +w /mnt/share. This lets anyone who has permission to read from and write to that directory.

Add the following share to smb.conf:-

[storage]
browseable = yes
comment = General storage
guest ok = yes
path = /mnt/share
read only = no

The share should now show up and be writable (i.e. you can save files to it) from the Windows machines.

If you install more hard drives into this machine, you can incorporate them into this share by putting a line for them into your /etc/fstab and making the directory inside /mnt/share.

An example fstab might be:

#
# /etc/fstab
#
#

/dev/hda1 / ext2 defaults 1 1
/dev/hdb1 /mnt/share/another ext2 defaults 0 0
1

To prepare this second harddrive and show it under the existing share, you can use:

mkdir /mnt/share/another (this creates the directory that it will "live" under).
Edit the /etc/fstab file as above
cfdisk /dev/hdb and delete any partitions on this new hard drive. THIS WILL LOSE YOU ANY DATA OBVIOUSLY. Type carefully. (Note that by using /dev/hdb, we are modifying the SECOND hard drive that the computer sees.)

Create one large Linux partition on it which should show as hdb1.

Similarly for hdc, hdd etc. as you add extra hard drives.

Printing
========

First pico /etc/rc.d/rc.modules and uncomment anything under the Parallel port sections (remove the # character from the beginning of the lines). When you reboot, typing lsmod should show up some stuff to do with "parport".

Then, use lynx to go to http://localhost:631 and in there you will find an interface that will let you add most laser printers. Any printers added should print a test page and should then appear under Network Neighbourhood as a shared printer.

If you have a non-laser printer or a strange standard one (e.g. Samsung), you can try with CUPS using Linux Printing.org's CUPS tutorial or use APSFilter:

/usr/lib/apsfilter/SETUP

(remember to cd back to the home directory when you are done).

APSFilter is more complicated and quite tricky to piece together.