Interesting book going into the difficulties of implementing software projects for the administration of government systems, especially when under duress, or trying to salvage failing projects. The author has a lot of experience in organizing such projects, often as an external actor brought in to save projects after they’ve already fallen into trouble.
Interacting with American bureaucracy is notoriously frustrating and laborious, and this book provides some insight into how the software systems to interface with this bureaucracy end up so messed up.
A good breakdown of the (purported) causes here would be that
- congressional mandates can be needlessly complicated, and don’t take into account that certain minor requirements can be major contributors to implementation complexity,
- regulators are conservative in turning the mandates into rules, and zealously diligent in doing so, the point of even inventing requirements that don’t exist,
- the implementation of the systems is hobbled by counter-productive regulations like the Paperwork Reduction Act which prevent internal expertise from being developed, and shift the job from implementing to managing complicated contracting,
- there is no ability for feedback on the implementation process to percolate up the perceived requirements, the regulations, or the laws themselves.
On all of this I agree with the author. My disagreements are more so on whether these can be reformed. It’s true that lawmakers could be more aware of the implementation work that would be required for the laws they propose, that they could repeal legislation requiring obtuse contracting for getting any amount of work done, that regulators could co-develop both the rules they make and the software making those rules, and that the culture of government could shift from conservative rule-following to pro-active value delivery. There are certainly many individuals motivated to see this happen, and ready to participate in this effort.
The issue here is that this optimism does not take incentives into account. Through willpower, it might be possible to shift government away from its current state, but incentives are aligned to pull it back to where it is now. The failings of American bureaucracy and its handling of software projects are not unique, and we see the same kind of failings across the governments of other developed countries. This kind of stagnation feels like the natural result of the accumulation of rules and complexity, and there are few mechanisms by which this accumulation is hindered.
On the other hand, in private markets, there are powerful forces of Creative Destruction in which incumbent companies can apply improved practices to deliver better quality services. Often, large companies get hindered by this same kind of bureaucratic stagnation, and fail to modernize and implement software projects. Thankfully, newer companies that can more efficiently provide better services are able to supplant the old giants in the market. Ironically, this mechanism is often only curtailed by unnecessary government regulation advantaging old and inefficient companies that have the expertise or special relationships to navigate government.
I take these factors as reason to reconsider the scope of government work, both in reducing the areas we want government to act, but also in what we envision the interface of government in its scope to be.
For example, many government forms would be better in the form of programmable APIs, using modern forms of programmable credentials, rather than confusing identity and authorization. This would then enable competition in the provision of UIs over those APIs, ultimately delivering a better service to consumers.