Our responsibility as Software Engineers

Do most IT projects fail?
Most of whom I know is inclined to answer 'Yes!'

Is there a problem?
I'll argue against that saying 'failure' itself is relative. However, there are a lot of studies out there that quantify failure. Let us leave that for another day.

What can I do to ensure that IT projects succeed?
Core Principles of Object Oriented-Design
Component Based Design
Design by Contract (Bertrand Meyer)
Service Oriented Architecture
Test Driven Development
Continual Refactoring
Software Factories (more later on that...)

These and more are most likely what I will hear. There are a lot of case studies that employ these concepts to get good results.

Alright, so why do we still have a rather chaotic software development environment in the industry?

In an attempt to try and understand this problem I will choose to compare the 'software industry' to my other passion 'the automobile industry'. No, we are not talking about the widely circulated MS and GM joke.

The automobile industry
Scientists or Automotive engineers 'design' cars. Typically, a modern car has over 4000 parts and almost every one of them is individually designed, independently manufactured and tested. A lot of these products are shared in many vehicles. There are government regulations and mandatory standards to follow.

When something needs to be changed, there are mechanics who can 'replace' parts or 'service' parts. There are laws, standards and political watch dogs. Even after these things reach the junk yard, I personally know people who can retrieve parts from them to reuse them. By the way, that includes me too. :)

A few examples:
Tires come in finite sizes and fit the same 'interface'

The cruise control system is essentially the same system in almost all mid segment cars. Ford, Mercury, Lincoln or a Mazda actually share the same product.


The software industry
Now, I'll draw just a few comparisons to the software industry and leave the rest to imagination: Not many facilities have 'scientists' designing software systems. There is a fair chance that an 'Expert' does the design. Clear independently designed, developed and tested components are possible but not prevalent. Sharing mostly occurs in terms of data, but sharing logic and processing is only catching up. I am not just talking about web services, but actually API availability, some what like Google's. Yeah, and why not, they have all the smart people! ;)

Examples? Here:
How many of you developers have designed and developed an in-house login/security system? If millions are reading this question, there is a good chance that a million heads are nodding. Imagine that! :D

Let's raise our two hands and clap hands in applaud for those who use the MS Enterprise library. If you are not using .Net but are involved in something equally impressive, please find yourself eligible for applause.

Ok, if you have read this far patiently, I am sure you know what I am talking about. Is there a Solution? So, at this time, am I suggesting that we follow the automotive industry? Do we need 'Software Factories'? (Pun intended) Yes and more.

But there are obviously lots of differences. For one, what do most of these Detroiters deliver any way? A contraption that allows you to drive on a road; that's about it. What does software allow you to do? Everything! Nothing short of that.

Some Philosophy
Let me digress and say something here before we continue. Humans have a processor up in their heads. In cave man times, the human being managed to do some things. Then, he created a few tools and 'automated' or eased completion of some of those tasks. Do you realize that a simple knife is a tool that eases your cutting or shearing something, which otherwise would need to be performed with your own two hands? Progressively some evolution plus billions of tools and automations later, we have our current world. Regardless of any political, environmental or social changes humans will keep doing this. Now, I am not going into the topic of is that for better or worse. I am just saying that it has been happening and that it will happen as long as the human being lives.

So, my definition of a 'computer with its components' comes out of this philosophy. A tool that automates certain tasks. Software development is in essence using this base thing computer with its many features and adding one more layer: your software. Thus, there is no real limit to what you can achieve following the path of software development.

Note that I am using 'Software development' in a broad sense that includes all the steps from the vision of creating something to delivering it and maintaining its existence.

Back to where we were
Now, being this software engineer that I am, with the tools I have, am able to create a fairly complex system from scratch and deploy a complete solution, thanks to the 'ready made' platforms available in the market today. No widespread regulations or standards to follow. I am not exactly liable if the software fails. Is that a problem? Hell, Yeah!

And, did I say "not many regulations?" Yes. Any argument? There are a few standards and laws around the industry. But have you ever heard of 'software recalls'? Does a software architect sign his name under a design document? Are there government mandated tests a software engineer has to pass like civil engineers have to?

For the executives
A common complaint is that the executive management is usually happy to see their business requirement but doesn't know or care about the technology. So, my question would be "how many out there understand the function of an IC engine"?, or "the Crank shaft"?, or the "Fuel Injection System"? End users don't care and they won't. Most of the management won't and shouldn't either. But if you have a management that listens to the IT head and understands the utility of these 'tools' that is good enough.

To executives, IT should be an asset, not an academic learning program. They better know what a car can do for them, not how its internals work.

Problem and Solution
Ok, so finally what is the problem? Immense capability in the hands of individual software engineers. Very low oversight and lack of liability laws.

Well, at this point it is not even imaginable for encompassing laws to be developed for the current chaotic yet super powerful software engineering world. So what do we do as software engineers?

As software engineers we have immense power. And you should know what comes with immense power: Great Responsibility.

Exercise restraint. Don't develop just because you can. Don't deliver just because you can. Don't hack just because you can.

Lots of people compare software development to poetic compositions. I have been guilty of that myself. You can do that at your desk, in the lawn, on a sunny beach or if you are lucky at a research facility, but not with commercial projects.

Follow design patterns. Use tested practices. Emulate the knowledgeable. Find mentors. Learn from industry leaders and experts.

Develop custom cars in your garages, but there is a 'Road worthiness certificate' that you need to obtain for it to share the roads and streets with other cars. We as software engineers don't have to get one for software that we build, but act like you have to get one.

There is a something called 'Computer Science' and there are Computer Scientists.

From the desk of a wannabe Spiderman
Since 1991