Centralization and organizational single point of failure

Talk to an engineer or architect and they will likely be aware of the term "single point of failure" and have experience in trying to remove them or mitigate the effects of them. Ask them to give a few examples of such issues and they will frequently be able to come up with several examples, a database in some system, a router in another or a configuration service in some obscure hidden corner of the architecture.

While feared and loathed in tech single points of failure are both accepted and even preferred in other disciplines. Centralized organizational units that should serve the entire company are such an example. Frequently specialization, control and economies of scale are cited as the fundamental benefits of those units. While they can have a place for some problems the negative side effects are often overlooked...

Scalability
Adapting to rapid changes in demand is something that centralized organizations are frequently very poor at. If the benefit was to get a few very specialized persons together when demand change they will likely be idle or swamped, while some idleness might be tolerable a central team that is swamped by demand and introducing latency for countless persons and teams in the rest of the organization can be fatal in times when great change is needed. If you ever see ticketing systems that maintain queues this a sure sign of a scalability issue.

Learning
Frequently centralized organizations are built to maintain highly specialized expertise that is not be believed to be sustainably maintained otherwise. While this might sound great it frequently means that less of that expertise is available out there in the rest of the organisation where daily operational action happens. But people talk and information sharing is cheap, there are other ways to maintain highly specialized knowledge without putting them all in one organizational unit. If the skill is truly important for your company you do not want fewer persons with this skills, you want to have more, and when you have more you will also have a few of those many that are incredibly skilled.

Accountability
When moving people into centralized teams to serve many, typically these persons are removed from any accountability since they increasingly become a consulting function, which means that the most knowledgeable persons in the company are no longer as influential and as engaged in the important decisions in their field. While they might give great advice to many the risk is that they are not as deeply involved where it really matters.

Commoditization
For smart people in a centralized organization it makes sense to invest in automation of their work, providing generic solutions with great documentation and training material that can be reused over and over again throughout the company. If this is done successfully it can be a highly efficient way to achieve economies of scale. You will recognize such a unit by never hearing about capacity issues since they have basically removed themselves from the value delivery, in a good way. But if the problem area requires multiple diverse solutions that can not be commoditized this is very hard to attain, and the persons working in the unit will basically need to be consultants that create value by their own hourly work. Come any change in demand and scalability suffers greatly. If it's not a "one size fits all" kind of thing problems are likely.

Lead times
Traversing organisational boundaries typically have a cost associated, frequently the cost is lead time. As short as that lead time might be, when those small amounts of time are aggregated they add up pretty quickly, if ability to innovate and improve is at all of interest reducing lead times for work to keep loop times as short as possible is paramount.

As devastating as the effects of single points of failures can be it makes sense to look carefully to find them and fix them. When in doubt on how to analyze such organizational issues look no further than your skilled engineers, they have keen eyes to identify and address such issues.

Many organisations are so used to some of the pains that come from a centralized organisation and have spent much effort on mitigation of these problems that the willingness to adjust is fairly low. Frequently some of the problems caused by organisational single points of failure are just accepted as "that's how it is in large companies", but it doesn't have to be that way, different designs are possible.



Comments

Popular Posts