Complexity

I sit on the Library Advisory Board for the City of Wylie and we had our monthly meeting last night. At one point in the discussions, it came up that a relatively minor technical problem had happened at the library and that it had taken almost a week to fix. In discussing this event, several of us suggested that any number of people could have fixed the issue in a much shorter period of time. However, as it turns out, they were (and are) not allowed to do such things because of what we call in the technical world “process”. Certain procedures and protocols have to be followed in such instances because they exist, not because they are useful.

This is one of many examples of bureaucracy that I’ve run into, not just on the library board but in life in general. Bureaucracy thrives on complexity because it allows inefficiencies in the system to be hidden and avoided. These inefficiencies not only frustrate the people who have to deal with the bureaucracy, they also naturally mean that bureaucratic systems function at a much lower level of output than they could.

Clay Shirky wrote a fascinating article on the complexity of business models earlier this month. In it he argues that the future of media mostly likely lies not with the monolithic media conglomerates of today but instead with new creators of media that do not rely on the built in complexity that current media companies thrive on. One of the key quotes:

To pick a couple of examples more or less at random, last year Barry Diller of IAC said, of content available on the web, “It is not free, and is not going to be,” Steve Brill of Journalism Online said that users “just need to get back into the habit of doing so [paying for content] online”, and Rupert Murdoch of News Corp said “Web users will have to pay for what they watch and use.”

Diller, Brill, and Murdoch seem be stating a simple fact—we will have to pay them—but this fact is not in fact a fact. Instead, it is a choice, one its proponents often decline to spell out in full, because, spelled out in full, it would read something like this:

“Web users will have to pay for what they watch and use, or else we will have to stop making content in the costly and complex way we have grown accustomed to making it. And we don’t know how to do that.”

You can see this type of attitude all around the world, not just in TV companies but in record companies, governments, Goldman Sachs and the Catholic Church, just to name a few that have been in the news lately. Complexity thrives until it doesn’t and then we revert to simplicity, often in dramatic, spectacular fashion. If you’re in the technical field, you hear horror stories about just this type of event any time 1 or more technicians get together. Complexity seems to be the bane of actually getting things done.

So why do we allow it to happen? Initially, increases in complexity tend to impart a competitive advantage on the entity being complexed. Moving from a world where I have to produce everything I need to live to a world where I can write code on the computer and buy everything I need is mostly good, especially on an individual level. The problems start occurring when the complexity reaches a level that parasites can begin to extract value without being punished or even noticed in many cases. Many times, even if they are noticed, they are practically immune to punishment or rejection because of the complexity of the system they live in.

Take for example the Rubber Rooms of the New York City Education system. Here, teachers who have committed infractions ranging from incompetence to the molestation of a school child live out amazingly boring days while waiting on arbitration of their case, all the while earning their full salaries and pension benefits. Their average length of stay? Three years. Even if the teachers are eventually dismissed, they are entitled to their pension. Here, one layer (or 30) of complexity too many has been added to a bureaucracy and now the system is failing.

Or consider the saga of the NUMMI plant, a joint venture between General Motors and Toyota begun in 1984 in hopes of bringing the Toyota system of car making to GM to reintroduce quality cars. The NUMMI plant worked fantastically but when the time came to introduce the model and methods to other plants within the GM ecosystem, it didn’t work because other plants were not interested in more efficient models because it would potentially take away their power and prestige.

In the case of GM, the bureaucracy established by the company and their union eventually led to the company becoming the largest bankruptcy in US history, all paid for by taxpayers. Once upon a time, GM was a shining star of American capitalism but it ended up a disaster largely because of a huge, sprawling bureaucracy that fed on the complexity of the system. There is no guarantee that it will ever recover.

Our financial system is currently a morass of complexity that is so intertwined with our government, another huge sprawling bureaucracy, as to be indistinguishable in many ways. We found this out the hard way when the near collapse of the financial system threatened to bring down our government and many others. The financial system has become a parasite on the economy, one that appears to be providing benefit but is fact sucking the very life blood out of its victim. Once upon a time, banks existed to provide the capital and liquidity necessary to contribute growth and prosperity to our economic system. Now many banks exist as the ends instead of the means, producing nothing other than outlandish profits and bonuses for management and employees, seeking rent from the American taxpayer.

The difference between many large complex systems that have failed in the past and the complex systems of our day is that we extracted a huge amount of value from the real economy to keep the zombie systems alive. Shirky ends his article with these thoughts:

When ecosystems change and inflexible institutions collapse, their members disperse, abandoning old beliefs, trying new things, making their living in different ways than they used to. It’s easy to see the ways in which collapse to simplicity wrecks the glories of old. But there is one compensating advantage for the people who escape the old system: when the ecosystem stops rewarding complexity, it is the people who figure out how to work simply in the present, rather than the people who mastered the complexities of the past, who get to say what happens in the future.

While that may have been true in the past, our inflexible institutions have instead been propped up at vast expense to the middle class in a misguided effort to protect our economy from collapse. The problem with that is no evolution has happened at all, no new ecosystems have arisen to take the place of the failed ones, simplicity has not come to the fore in the absence of the complex. Instead of new traditions and businesses emerging from the ashes, we are still beholden to the complex beasts of yesterday.

Complex systems can only be allowed to exist insofar as they provide more value than simpler systems of the same type. When they do not, the only thing that makes sense is to allow them to die, either quietly or loudly and with much fanfare. But by propping up the dead and pretending they are still alive, we only manage to stink up the place for the foreseeable future.