Technology for Mortals

One of the biggest objections I have to the typical VC or IT analyst view of the world is this concept of “open source strategy”. If you talk to an analyst or a VC, there is this conviction that open source is some special magic that gets folks working on your project for free, and makes things “disruptive”. The reality does not bear that out. First and foremost, the typical customer (as in, “probably 99% of customers”) could not care less if something is “open source” or merely “free”, except in rare cases (like some Web apps). Why? Because having access to the source isn’t useful:

It takes a large degree of sophistication to modify an application in a way that keeps it from breaking when you try to upgrade it.
• A developer trying to tweak an existing codebase which is unfamiliar operates at a very low efficiency rate.
• Only the best run OSS projects have things like code standards in place (the same can be said of closed source, of course); the majority of the open source code that I’ve personally looked at was an absolute nightmare.
• Most IT shops barely have the resources to keep the lights on, let alone spend time inspecting source code to ensure that it doesn’t have bugs, security holes, etc.

In other words, actually leveraging the source code itself is not happening for the overwhelming majority of shops.

Secondly, take a look at the actual numbers. The bulk of people who contribute to most major open source projects are being paid by their employers to work on that project during their 9 – 5 hours. Even more telling, those paid workers perform an overwhelming majority of the commits to a typical project. The “just so story” of the army of hackers sweating away in their nights and weekends to produce great code for the love of humanity… it’s a myth for the most part.

Thirdly, this classification of software into “open source” and “closed source” is bogus. The status of the code itself (closed or open) is merely an attribute, an adjective, and hardly the most important one. Do you really think that the typical IT department (based on the information I’ve provided) really cares if it is “open” or “closed” source? Not really. The adjectives they care much, much more about are “TCO”, “ROI”, “ownership headaches”, “does it do what we need?”, “support options” and “upfront cost”. Who cares if it is “open source” if it is a huge pain in the neck? What IT manager is going to turn down the solution that best meets their needs because it isn’t open source?

Here is a good example. My employer is a Microsoft Certified Partner, so we get most of the Microsoft apps for free. We will never make use of the source code to an open source app for the reasons stated above. For us, open source apps are equivalent to the Microsoft apps, because they have the same licensing price point (free) and the only guaranteed advantage of open source (access to the code) is useless to us. Even for companies that pay for software, with the exception of certain “enterprise class” vendors that charge a mint, the cost of a software license is a wash compared to the cost of maintenance, integration, and other TCO items in most cases. Compare MySQL to SQL Server, for example… unless you have a large data center, the few thousand for SQL Server is peanuts, unless your company is broke (in which case, you have bigger fish to fry).

Finally, the concept of “open source” is quickly being demolished by Web apps and SaaS, which do a neat little end run around the GPL. Look at Google. Google has done some absolutely amazing things to the open source that they built their empire on. But they rarely even talk about it in general terms, let alone release the actual source code. While I am not a fan of the GPL for personal reasons (I prefer BSD and MIT style licenses, and I have released a number of items under MIT license myself), I understand the motivations behind it; it is designed to coerce a particular behavior from those who profit from using GPL’ed code. By not actually redistributing the applications (and therefore not triggering the requirement to release the source code), people who are in these *aaS scenarious are just as closed source as any proprietary vendor, but they have the benefit of starting with someone else’s code. It’s basically stealing according to the spirit of GPL, but not the letter.

Here is where open source really makes a difference:

• Startups with little or no working capital (the person trying to start a business on their nights and weekends)
• Companies that have the brains and need to make a giant application, and can use open source as the basis for it (Google with their homebrewed stuff is an excellent example)
• A select few applications (usually written in dynamic languages) which are well architected and allow the end user to customize functionality in a way that upgrades to the base system can still be made

As you can see, there is very little evidence for this concept of an “open source strategy” in the trenches, because whether or not an application is open source is just not relevant to the typical business or user.

J.Ja

This article will show you how to build Qt, the popular C++ framework from Nokia, so that it is both small and, if you prefer, available for static linking. Your Qt applications will be smaller, possibly faster, and can be distributable as a single executable.

Also answered: How small can Intel’s C++ compiler make a large library? How does Microsoft fare? Three compilers (settings tuned for small file output) and their resulting code size is compared.

Read more…

Make your entire Qt application work entirely from a single file.

I was downloading Ubuntu Netbook Remix for my EeePC 900A and found that it’s installer is not a CD image, but a raw image meant to be written to a USB FLASH drive. After downloading their handy image writer, I found that this simple utility came with quite a few support files, including a surprising 13.5MB of .dlls.

This image writer tool was written using Nokia’s Qt software, which allows easy development and distribution for all the major platforms from a single C++ source base. This program was compiled in the same way most Windows software is today — with “dynamic linking”, requiring the Qt libraries, among others, be shipped with the program as separate files.

This is fine for large software projects, but it’s a little cumbersome for small tools like this one. Many programs can be distributed such that all necessary files are built into the executable in a process called static linking. Sometimes it is nice to be able to download just the program itself and not have to worry about making sure various DLLs are included (though setting up an installer largely eliminates this problem).

Static linking has its advantages and disadvantages, but when not done, running a program unaccompanied by even one support file will result in a cryptic error for the user:

These 4 steps are all you need to make your entire project result in a single, easy-to-distribute .EXE file:

1. Complete your Qt project using normal debug libraries.
2. Compile the Qt libraries for static linking (needs to be done only once).
3. Add the necessary lines of code to include any Qt plugins you may need.
4. Compile your release version with the static libraries.

Read more…

I finally put my finger on what drives me nuts about KDE. It’s so petty and trivial: all of the applications insist on starting with the letter “K”. Not only is it absolutely ridiculous, but it makes finding anything impossible. “KEdit”, “KMail”, etc. It reminds me of the “Mortal Kombat” games. KSeriously.

J.Ja

I’ve been having a few major problems with my FreeBSD server for over a year now. In a nutshell, a bad CPU caused compiling to fail quite often, which trashed the installed software packages pretty badly. Even after replacing the CPU, I could never quite get PHP working. No matter what I did, it wouldn’t take POST data! I tried upgrading to FreeBSD 7.0, and that didn’t do the trick, and even created additional problems in the process. So why didn’t I just rebuild the server? One word: “qmail”.

See, qmail is a mail system. Like “sendmail” but without sendmail’s insecurity. But qmail likes to be “special”. It’s creator has certainly principles that he likes to stick to, which spill over into the software. In this case, he beleives that the directory hierarchy that he prefers is so much better than the default for your OS (directory hierarchy is a favorite religious arguement in Unix-land), that his stuff is really tricky to make work outside of his structure. On top of that, most of the tutorials out there are from the “compile from source” crowd, so they don’t help too much for people (like me) who use the OS’s package or port system.

To make matters worse, qmail takes the Unix modularization to a truly sick extreme. There are so many sub-programs that it seems like each class in the code became its own module, so the functionality can be “drop-in replaced” at an extremely granular level. While there is something commendable and “elegant” in this, it makes problem solving nearly impossible.

The last time I set up this server, qmail took me nearly 3 days of non-stop effort to get working the way I wanted it to. The first time I did it, qmail took me nearly a week of 20 hour days. when I finally broke down this week and rebuilt the server, qmail took me merely 3 nights’ worth of effort. It is a measure of how twisted this is, that I consider spending three nights’ worth of effort an “improvement”. Indeed, I could have earned enough money in that period of time to purchase a Windows Server SBS license, provided that I had enough articles lined up.

I am not going to go into what went wrong this time and what the resolution was. A quick summary would simply be, “qmail’s strong preference for certain defaults, coupled with qmail’s preference for pure source compilation, multiplied by the extreme modularity of qmail, with a small dose of operator error, created a situation which was untenable”. In other words, the system is so bloody complex that a simple mistake not only makes the whole thing unworkable, but eliminates the possibility of getting any kind of meaningful troubleshooting information, and the resources online are all wildly different as well as often being quite out of date.

If you value your time, stay clear of qmail. Sadly, it is the best game in town for BSD users, but if you are on a Linux distribution that supports Scallix or Zimbra, go with them and save yourself some pain.

J.Ja

So I finally got my FreeBSD machine up & running. I spent all of last night updating all of the installed packages (all to get PHP working right) now that it compiles correctly. And then, I decided to go for broke, and upgrade from FreeBSD 6.3 to FreeBSD 7.0. Immediately before I started, I made backups of everything but I did not test those backups to see if they were good. From the italics, you can see where this story is headed.

Well, the upgrade 7.0 blew up on me. Something went wrong, who knows what. No problems, right? I’ll just boot off of a Live FS CD, run the restore. I’ve done it many times before, even as recently as a few weeks ago. Woops!

For whatever reason, I see to be completely unable to restore from the backups. I don’t know what the problem is, but this time, “restore” is blowing up. So now, I am attempting to rebuild the system from scratch. Luckily, I made a separate backup of the critical data files a few weeks ago, so all is not lost. But needless to say, I have egg all over my face. I know, I seem like the world’s most incompetant sys admin here. Sometimes I am sure I am. But then I think of the mistakes other people make, and I feel a lot better. At least I made the backup in the first place! Ironically, it is the first FreeBSD upgrade that’s ever failed on me, and the first time I ever made a backup before doing an upgrade.

I don’t know what went wrong here; I have never had a problem with the FreeBSD backups in the past. I found the problem; the backup was fine, it was my use of “restore” combined with the Live FS CD that was the problem! But a painfull lesson (yet another) is to be learned here. Before counting on your backups, test them the process from top to bottom!

J.Ja

One day after the Debian Linux project announced a massive flaw where its implementation of OpenSSL key generators only used 15 bits of entropy (32,768 combinations), HD Moore (creator of Metasploit) has released a tool to exploit it.  Nate McFeters has a good write up here on this matter.

Because this bug is involves very obscure cryptographic concepts and the severity and scope of the flaw wasn’t easily understood, it didn’t really get a whole lot of media coverage.  Now that the flaw can be exploited in Metasploit, the issue should get some attention.

The flaw stems from the fact that the PRNG (Pseudo Random Number Generator) was crippled leaving it with only 32768 combinations to test.  That means all RSA and DSA cryptographic keys generated by Debian and Ubuntu Linux distributions are effectively worthless.

The impact of this exploit is massive and it can easily affect non-Linux systems like Windows or Mac if those computers have a Root Certificate generated from a Debian/Ubuntu computer.  Any asymmetric crypto keys generated between September 2006 and 5/13/2008 on Debian or Ubuntu Linux distributions are affected.  Every affected key needs to be revoked and regenerated. System administrator and security professionals everywhere should start auditing their computer for this very serious weakness as soon as possible.

Update 4:32:PM

c0uchw4rrior in comments below asked: “You mention Root Certs in the last paragraph. Does that include SSL web certs from Verisign/GoDaddy, etc?”