Frank Drake, in the development of his famous equation in the early 1960s for the search for extra-terrestrial life, included what is known as the “L” variable. This variable stipulates, in addition to the viability of a civilization’s life-span, the concept that if an earth-like planet is found with no life upon it, this lack of life could be the result of the misused development of nuclear technology (i.e., the atomic bomb).
As a senior software engineer professional, I must expand this variable to represent not just nuclear technology but technology in general. Drake and his initial research team, though quite cognizant of the dangers of nuclear technology, were too singular in focusing on one technology, a lapse for which he can be forgiven since studies on technology’s effects on the daily lives of people were still in their infancy.
Today, most people rarely give a thought to the current hi-tech computers and gadgets they use every day, yet all of these technologies have a Dark Side since any technology can be used for good or evil… or both. Where and how one applies any technology also determines its long term success, to include whether its benefits will outweigh its drawbacks and whether its ill effects can be absorbed by society and the surrounding natural environments. A classic example of this was the invention of the cotton-gin, which had a ripple effect throughout American Southern society prior to the Civil War; slaves may have been able to produce more cleaned cotton but this only encouraged further support of the Southern slave culture. The economic consequences furthered the growing conflict between the Southern aristocratic/slave-labor economy and the Northern capitalistic/paid-labor one. This conflict would become an underlying motivation toward the “War Between The States.”
Thus, technologies affecting major institutions and people’s daily lives will not only have profound effects in a society but such effects, unfortunately, often seek the lowest common human denominator. This will always be true as a result of human penchants for individual power.
This power dynamic is especially dangerous when technology speeds up a faulty and dangerous process, as expressed by this axiom:
If a technology is applied to an erroneous process and that process is not corrected, the results will remain erroneous but be produced at a far faster pace. Humans are, by their very nature, highly flawed in their personal perceptions of their realities. Applying technology to such warped perceptions can and does cause terrifying results.
A classic example of this axiomatic collision of human perception and faulty automation was found at a hospital where I worked in their Information Technology organization. It involved a medical analysis and tracking system to authorize and reference blood–sample results. In this instance, hospitals always took a primary and a final sample of blood from a patient to ensure consistency in blood chemistry prior to a procedure being performed. If there was a difference, further testing would be required to verify if the procedure could go ahead as planned, be halted completely, or be modified toward another procedure.
When doing this analysis manually, mistakes were made but they were easy for hospital personnel to detect since the paperwork could be quickly analyzed for validity. Still, a mistake could filter through the process for a variety of reasons if there were no further checks on the validity of such tests other than the assigned nurse or doctor in charge of the patient.
When a new, automated system was implemented, it had a fatal flaw: no validation checks were made between the two blood samples since the system merely updated the final information with the primary information every time data was entered, whether it was for a primary or a final blood sample. Medical personnel simply could not see the inner workings of this system, which meant that they had no idea that the system was producing erroneous results.
For six months patients with serious conditions were being admitted to scheduled procedures without medical personnel having any knowledge of blood test consistency until my colleague, who had been performing analysis on the system, uncovered this flaw. The system was quite complex so it was not something anyone would have found unless they suspected something was amiss.
This kind of error in technology goes on all the time. People unknowingly continue to use data produced by flawed processes. For example, how many professional truckers have crashed their trucks as a result of GPS systems sending them down incorrect routes? It happens far too often.
Why is this allowed to happen if the technical profession is cognizant of such matters? The answer lies in the deeper sociological traits of society itself. Technological innovation is applied both to the individual as well as the society an individual is a member of. However, in this case we have to consider the entirety of humanity. And humanity, being the flawed entity it is, has never learned the critical lessons from its own history. One such criticality was what Roman Cynics were trying to teach people in their day (or the message of Christians who followed Christ): have compassion for and understanding of your neighbor. The avoidance of such a message has subsequently continued to encourage the most primeval of all human characteristics: its penchant for abject and exploitative violence.
The long-term result has been a seriously flawed sociological process toward modernity that has brought humanity to the brink of immediate extinction more than once. If you think that the 1962 Cuban Missile Crisis was the worst case of this in our history you would be sadly mistaken. Few people have read the documentation in the 1980s of a U.S. Air Force SAC (Strategic Air Command) bomber squadron that was headed to Moscow for a nuclear bombing run and passed its two fail-safe points because of system software “glitches” that were unnoticed until a lowly airmen caught them 20 seconds prior to the bombers passing their third and final fail-safe point. Had he not found this situation in time everything we know would have simply disappeared in the catastrophic nuclear response.
There is documented evidence that such events have occurred roughly 300 times in US military history since the continued development of nuclear technology during the Cold War and the development of the similarly complex technologies of the software programs used to monitor nuclear-based activities and endeavors.
There is a very simple reason for such failures: though the Information Technology field has been quite cognizant of the proper processes and procedures required to create high-quality software for many years, technical managers have consistently refused to adhere to such standards. And with few exceptions, this lack of adherence has been a consistent issue across all forms of software development, even when the underlying requirements entail life and death.
This trait of the profession has then consistently produced a pattern of software project failure across all project types that stands at 70%, which is roughly the same rate of failure that was observed over 35 years ago when the tracking of such statistics was first begun.
This is not a technical failure by any means but a human one in which the pursuit of profits, selfish agendas, and self-aggrandizement are found to be far more important than the production of quality. And this cuts across all professions, though unlike software development, many similar technical endeavors have seen a maturation in standards and processes that software development has quite intentionally refused to follow.
Today, software literally runs the large majority of transactions in daily life, but the expectation that it is reliable and fool-proof is not mirrored in reality. Quality and reliability cost time and money, and there are simply too many technical managers who are willing to side-step the critical foundations of quality production, at the potential expense of human life.
The result is that modern life today relies on a foundation that is akin to a very wobbly house of cards.
Steve Naidamast is a senior software engineer by profession and a military historian by avocation.