By Lambert Strether of Corrente.
Democracy Now has an interesting interview with Clay Johnson, co-founder of Blue State Digital, the firm that built and managed Obama’s online 2008 campaign for the presidency in 2008, and then director of Sunlight Labs at the Sunlight Foundation. Johnson has been all over the Exchange debacle coverage (see here, here, and here). I’m going to take a look at some questions raised by Johnson’s very layperson friendly radio discussion.
But first, I should caveat that Johnson — as a subject matter expert in government procurement — was very helpful to me when I was trying to dope out ObamaCare’s CMS contracts last summer. (Alas, as matters turned out, it was dig into that story or — the life of a big-time blogger! — paint the house, so I had to paint the house.) And although in a very much past life I’ve done government contracting, I’m very much a content person, not a programmer. So, two strikes against me for taking on this topic!
So that said, let me start with something Johnson said in the DN interview that absolutely makes sense to me:
But when government is building software like this, it ought to be built out in the open. It ought to be built with a licensing system called “open source,”[*] so that the public truly owns it. You know, if my tax dollars are going to something, then it ought to belong—if the public’s tax dollars are going to something, then it ought to belong to the public.
This seems utterly unexceptional to me (modulo, off the top of my head, grandfathered Commericial Off-The-Shelf Software, and modulo a discussion about software that affects so-called “national security). Public code for public purpose, as it were. However, note that Johnson makes an ethical claim (“ought… ought… ought… ought”) and hence a policy and ultimately a political claim, and not a technical claim (which will become important in a moment).
So how would that happen? Johnson writes in a Times Op-Ed:
The president should use the power of the White House to end all large information technology purchases, and instead give his administration’s accomplished technologists the ability to work with agencies to make the right decisions, increase adoption of modern, incremental software development practices, like a popular one called Agile, already used in the private sector, and work with the Small Business Administration and the General Services Administration to make it easy for small businesses** to contract with the government.
Large federal information technology purchases have to end. Any methodology with a 94 percent chance of failure or delay, which costs taxpayers billions of dollars, doesn’t belong in a 21st-century government.
This again is an appealing vision. In fact, if it were easy for small business to contract with government, then no more Joe Stacks. (See Stack’s interpretation of Section 1706 of the Tax Reform Act of 1986, which got me, too. In fact, I’m surprised there aren’t more software engineers flying airplanes into buildings.)
But now let’s return to Johnson’s DN interview, because here’s where my questions arise; where we apply an open source critique to the ObamaCare debacle. DN, and radio as a medium, perhaps aren’t the ideal mechanism to convey complex technical ideas. So let’s take this as the lowest common denominator expression of a process the public should fully own. I’ll key material where I want to ask questions — and no, I don’t feel the need to break out the color coding markers! — with numbers in square brackets, thus: [0].
AMY GOODMAN: … What do you think happened? Why has this website not worked?
CLAY JOHNSON: Well, government doesn’t have a lot of people to choose from when they’re looking for contractors to build this stuff. And I think part of the problem is that the same people that are building drones are building websites.[1] When government is building a website like this, they have to use a system called procurement, which is about 1,800 pages’ worth of regulation that all but ensures that the people who are building this stuff are the people with the best lawyers, not the people with the best programmers.[2] And so, you know, you have this sort of fundamental lack of talent[3] amongst the contractor ecosystem that’s building this stuff, that it’s bound to be bad work[4]—that, combined with the fact that in 1996 Congress lobotomized itself by getting rid of its technology think tank, called the Technology Assessment Office. So when they’re writing bills, they don’t understand the technology that they’re requiring in their laws.[5] This is what you get when you have a Congress that is basically brainless on technology, and government who can only pick from a few old, stodgy contractors.[6] You’re bound to have this result.[7] And, in fact, the standings group came out earlier this week and pointed out that over—for all procurements over $10 million, 94 percent of them fail.[8]
[1] “The same people that are building drones.” Yeah, Lockheed, ouch. (Though surely not a Canadian company like CGI?) And technically, aren’t drones innovative, and don’t they work pretty well? It’s not Lockheed’s fault Obama’s drone strikes keep blowing children to pink mist. That said, does open source scale? ObamaCare is a big systems integration project, and comprises Federal Departments like DHS, IRS, CMS, the Department of Indian Affairs, the Peace Corps, and Medicaid, at least one credit reporting agency (Experian), and 36 states. That’s just big. Do we have a precedent for a project of that magnitude being executed successfully as open source?***
[2] “People with the best lawyers.” As always, complexity and excess lead to niches for rentiers — ObamaCare itself being a prime example of this, except that the rentiers are government contractors and not the health insurance industry — and in procurement indeed, the “Beltway Bandits” leverage their political clout, knowledge of the process, contacts in government, and even existing contract vehicles to get work they should not get on technical merit alone, at prices that go far beyond the value of the productive work that they do. That said, it’s always possible to make things worse. How many pages of regulation should there be, if 1800 is too many? Does the UK’s Digital Government Service provide a precedent? And language like “work with agencies to make the right decisions” really gives me — as a taxpayer and a citizen — pause. (“If men were angels, no government would be necessary.” ) If I take that language literally, and translate it to the local level, it would mean getting rid of those pesky building codes and zoning regulations and letting the Town Office and the real estate (not software) developers “work together” “to make the right decisions.” That would lead instantly to open corruption and a destroyed ecosystem: decisions that benefited the developers only, and nobody else, and things are ugly enough as they are. Now, surely Johnson doesn’t mean for the language to be interpreted as I just did. But what does the language mean?
[3] “Lack of talent.” Diplomacy aside, I’m just not sure about “fundamental lack of talent.” When I was a contractor — albeit at a far lower level than Johnson — I met plenty of smart, committed people in government, and when I worked in a corporate cube I met plenty of, er, drones. Surely the programmers running the Mars Exploration Rovers are talented?
[4] “Lack of talent” “leads to bad work”? But what does “bad” mean? What if the incentives are structured to make “bad” work more profitable than good? Consider self-licking ice cream cones:
As far as I am aware, the phrase came into usage among knowing observers of the Vietnam War. A self-licking ice cream cone is a programme or policy that costs money and resources, generating a great deal of activity; produces indicators of its own success, preferably quantitative; but does not actually achieve its announced goals. Indeed, a proper self-licking cone undermines the very purposes for which it was created, while at the same time sucking in ever more resources from worthy and effective activities.
In the early 1960s, the US Air Force (USAF) advisory mission was trying to expand its role in South Vietnam. For this it required targets that were helpfully supplied by South Vietnamese officers. The targets were labeled “VC arms factory” or “command and control facility”, but were in fact peasant huts or the most prominent building in a village. The fighter bombers would go out and blow up these “structures”. Damage assessments duly reported a high rate of “success” – blown up huts.
To really grow this cone and set the self-licking in motion, an ever expanding list of targets had to be invented.
More targets meant more bombing, and more bombing meant a bigger role for the USAF. The result was more budget, more planes, and career advancement for all concerned.
The only problem was that the bombing was helping lose the war. Predictably, the peasants turned against Saigon, leading to the introduction of US ground forces in 1965. Like the “body count”, the number of “structures” destroyed had no relation to actually winning the war. It was an indicator of organisational “success” wholly divorced from reality, and from the values that the organization was supposed to be serving. The USAF was killing the very peasants it was there to save from communism.
Self-licking ice cream cones can arise in any modern organizational environment. People dedicated to the values of their vocations always have had to struggle against them–as did more than a few US officers in Vietnam.
Surely self-licking ice cream cones are an issue in software procurement? After all, from a pure profit perspective, a project that takes a $100 million to fail, and then another $100 million to fixand maybe another $100 million on top of that is far preferable to a $100 million project that just works! Now, it is true that I can think of ways that Johnson’s open source proposals would mitigate the self-licking ice cream code dynamic: the code is open, so there are more eyes on it; the firms are smaller and there are more of them, and so there’s more competition (at least at first); open source programmers have a strong sense of vocation. Nevertheless, do not the incentives for self-licking ice cream cones still exist? And do not incentives structure behavior? Here is where a successful example at scale would be really useful.
[5] “Congress lobotomized itself.” There are lots of things Congress has not lobotomized itself about, the campaign cash proffered by a behemoth like Lockheed, which is their “understanding” of technology, being one of them.
[6] “A few old, stodgy contractors.” Johnson’s claim that the primary characteristic of contractors is that they’re “stodgy” and “old” strikes me as not thought through. Surely it makes more sense to treat them as corrupt rentiers? Follow the money!
[7] “This result.” There’s a noteworthy absence here: Like most, Johnson tiptoes around the issue of White House involvement (some would say “political interference”). The White House political operation set the fundamental parameters for building the system: Adopt the Heritage approach and preserve the private insurance industry. That must have translated (depending on methodology) into a complex set of requirements for a complex system. The White House surely set the schedule over the three years since the ObamaCare legislation was passed; the White House set the baseline expectations for success or failure of the project at launch; the White House ran the project under conditions of extraordinary secrecy; the White House was responsible for which requirements got triaged (and which did not); the White House set a drop-dead launch date; the White House changed the web site’s plan selection form four months before launch. We have examples of great success for the open source movement in campaigning; but as we all know too well, campaigning is not governing. Do we have examples of open source successes, at scale, in a context of “political interference”?
[8] “94 percent of them fail.” IIRC, an extremely large percentage of private software projects fail. Are there statistics that show what percentage of open source projects succeed?**** On the scale of ObamaCare? In a context of “political interference”? Presumably there are not, or else Johnson would have supplied them.
Summing up, I really don’t want to rain on the open source parade. I really don’t. I use open source software all the time! And I’d rather live in Johnson’s world, where there are lots of small businesses writing open source software for the government; lumbering dinosaurs lose and agile mammals win!
However, I’m not sure that the technical issues that Johnson raises, though interesting, are the most interesting. The ObamaCare debacle took place — and any government program must and will — at the intersection of technology, policy, and politics, and we might ask ourselves which of those three forces is the driver. Clearly, policy drives technology; if, for example, it’s the policy of the United States government that only bi-planes will be built, then two-winged technology will at length issue forth from the procurement process. Equally clearly, politics drive policy; if it’s policy that only wooden bi-planes be built, and we find that bi-plane production is more or less evenly distributed among all the congressional districts where forest products dominate, then politics is where we’d look for explanations (“A Wooden America Is A Strong America!”). Bringing this home to the health care debate, which would be preferable: An efficient and Agile implementation of the complex ObamaCare exchanges, or a brain-dead and “stodgy” implementation of a dead-simple single payer system? Clearly, the latter — even if that’s a policy/political question — especially if your concern is excess fatalities among those not covered by ObamaCare even when it’s fully implemented. And if you’ve saved money by doing something that you should never have done in the first place, have you really saved money?
A political economy that was capable of procuring software as Johnson so justly would wish it to do would probably be capable of adopting single payer as well. I hate wicked problems…
NOTE * Definition and history of the term:
open source: n. [common; also adj. open-source] Term coined in March 1998 following the Mozilla release to describe software distributed in source under licenses guaranteeing anybody rights to freely use, modify, and redistribute, the code. The intent was to be able to sell the hackers’ ways of doing software to industry and the mainstream by avoiding the negative connotations (to suits) of the term “free software”. For discussion of the follow-on tactics and their consequences, see the Open Source Initiative site.
See also the definition at the Open Source Initiative.
NOTE ** If I were king, I’d give preference to small businesses that were also co-ops.
NOTE *** When I look at the projects listed at the UK’s Government Digital Service, there’s nothing remotely of the scale of ObamaCare.
NOTE **** I don’t know whether the definition of success for an open source project is the same as the definition for a project with proprietary source, whether corporate or government.
NOTE I may have more to say about this when I look at Krugman’s column tomorrow.
56 comments
Comments are closed.
We do now have a good metric.
The IT costs per page of bills through congress are between $200,000 and $500,000 per page.
Clay Johnson is spot on. Everything he says here is simple and self-evident.
For one, its not fun to work for large corporations because the internal politics tend to suck raw. In finance the pay is often double in order to entice decent developers to work with self-important traders and clueless middle management. Govt work may pay well, but not that well. Creative developers don’t want to work for bloated, corrupt and chaotic behemoths if they want to do good work.
What he says about all government software being open source is as elegant as it is obvious.
I wonder how he feels about the “turn” Obama’s taken since entering office… I wonder if he feels taken advantage of, or of having been complicit in something bad without ‘opting in’, as it were.
Anyway, great source Lambert.
The differences between Open Source and Commercial Bespoke software:
– Open Source is grown bottom-up by volunteers. In the case of large integration type projects (Linux being the stellar example here), controlled by one Chief Developer with the overall vision.
– There are no deadlines.
– Development never stops. If a module change is needed by some developer, it gets checked out, modified, tested, and then checked back in subject to the Okay of the Chief Developer.
– It’s not about money, it’s about product usefulness and product quality.
– Open source developers pick up skills as necessary for the task at hand. They are not limited to a single role. Hence one developer will be involved in design, coding, testing, and determining when and whether the module should see the light of day.
– Commercial Bespoke is driven from the top down by a pre-determined contract.
– The Contractor’s objective is to write the least amount of code for the most amount of money.
– There’s always a manager dictating short-cuts and compromises to the developer team.
– Tasks are divvied up in the classical industrial style: separate designers, developers, testers, and most crucially, separate managers who are the arbiters of determining when and whether the module should see the light of day, and when and whether it should be reconceived or discarded.
– Once written, ‘tested’ and launched, most of the developers wash their hands and move on: some to other projects, and some to other companies.
– a primary desire of Developers in Commercial organizations, once a project is completed, or even before, is to avoid working together with a particular manager or a co-developer ever again.
– There’s usually a gaggle of contractors hanging around, who don’t stick around, don’t integrate well with the core team, may not have any commitment to the product outcome.
I’ve done both. Open source development is competence building and confidence developing, and everyone tends to be a respected authority on something. Commercial bespoke has a higher proportion of unsatisfied, unrounded and un-self-confident workers.
Clay is correct. Open Source Software, carefully qualified by the developers and integrators, is available to compete with virtually any proprietary technology. Sure, as detractors love to point out, there are plenty of dead projects with no activity and “start-up” projects whose quality has not yet made their technology viable for production use. However, the mature Open Source Software technologies like Linux, the Apache Web Server, MySQL (now MariaDB), PERL, PHP, the GNU tools, OpenLDAP, and so many more are already running systems of the scale of this project.
Clay is wrong when he says Open Source is owned by the public, however. The public doesn’t nurture, enhance, and correct software so the software is owned and parented by communities of volunteer. But, unlike most proprietary packages, the software is open to the public to inspect, modify, and redistribute. It is a common good in that regard.
However, no matter whether the infrastructure software selected is Open Source or Proprietary, the outcome of a large project like the systems for the ACA (RomneyCare) is determined by both the schedule allowed and the quality of the definition of the requirements. Even the best small developers are stymied by ambiguous statements of requirements and changing specifications. And nobody can make unrealistic development deadlines.
Thanks for the article Lambert! (Disclosure: We are a company that provides commercial technical support for OpenLDAP).
Whether this project could have been implemented open source is a red herring. Open source has no implication of functional difference to closed source, but only with whether or not the work product is visible to anyone with an interest.
It seems more likely this project (and its escalating cost) was driven by an endless stream of change orders, many of them politically inspired by demands of non-tech boot-licks struggling to please demands of clueless political appointees.
If Congress now demands a look at the change order log — which may not exist but could be derived by staff from contractor invoices presented to the USG for payment — they likely to discover why this project has been a blowout fiasco from the point of view of cost, quality, and schedule.
Did this web site really require that much “creativity”? That word seems to get overused these days. There are tons of site out there that sell all kinds of insurance from different companies.
That being said it seems completely reasonable to expect that there will be glitches. I guess we are so detached from the “bricks and mortar” world that the web site becomes the “product”. Quite clearly that is not the case here.
Just wait until the insurance actually goes into effect. I cannot wait to see the right wing bloggers jump through their intellectual hoops to blame Obama for every decision of every private company that has sold insurance. Every claim denied will be the fault of the “government” rather than the fault of the corporate insurance carrier. While the proper conclusion would be that a true public option is necessary, the wingnuts will obviously conclude that only if we unchained these companies from Obamacare the “market” will magically cause these problems to go away. Which it will – by taking these decisions out of the spotlight and back into the closet.
First, thanks to Lambert for all his work on this chimerical phenomenon. This ObamaCare rollout personifies corporatism: the merging of state and business, the exploitation of the citizenry, who are mere consumers. And even calling the public consumers masks the reality of their denigration and exploitation by what passes for government. Since it’s corporatism, which is a perversion of democracy, it had to arrive as a disservice to the public because it was not meant to serve but to extract the fruits of labor.
Yes, creativity could’ve gone a long way in making this particular product a lot more user-friendly. For example, a creatively, or well-designed system wouldn’t be so brittle: if one part of this system fails it all fails, whereas a creatively written system would’ve provided alternate use cases that gracefully enabled the user’s ability to navigate, store and retrieve their research and potential selections. Just for one quick example — even if the system couldn’t immediately verify a family’s income and thus present exact prices, it could offer a range of options given the user’s stated income that could be affordable anyway, without the promise of a guaranteed price yet — graceful ways to work around the obvious choke-points in the system.
Also, they couldn’ve written it such that changes in the rules, which of course will be legion for years to come (if not forever given the way govt loves to tinker) were embraced by the architecture rather than a potential landmine for each change.
There a thousands of ways, large and small, that can make such a project a pleasure or a nightmare for the users AND developers, and creative approaches are key to that.
http://www.bioperl.org/wiki/How_Perl_saved_human_genome
Open source has nothing to do with anything here. The problem is a management issue. The government contracting system as far as IT is concerned must be completely revamped. The government ought to have a technology department staffed by the well-paid professionals who ought to be the project managers of all IT contracts above a certain amount. Contractors will sell uninformed and befuddled clients a million dollar system that could be accomplished by tweaking an off-the-shelf system costing a few thousand–but they have no way of knowing that. The set requirements based on ignorance and mythology.
I was a government IT contractor who sometimes had to come in and fix projects that failed I have seen some really, really, really, really bad sh!t that passed for “work.” I saw deliberate fraud and, in fact, was part of a couple of fraudulent projects (often the programmers don’t know this until it is too late). How this happens is too long a story. All of this could have been avoided if the people making the decisions for the government were technical people.
As a whole, the whole contracting movement has become a millstone around the neck of the federal government in a million different ways. I can give you a list of scams I have seen and not just in IT. Procurement, as a whole, is a disaster as everyone should know–all of this is
reformable if Congress wasn’t as corrupt as it has become.
Most of my working life has been spent on large systems for large corporations, mostly in the financial sector, and I have observed a number of catastrophic failures, which are, in general, the habit of the industry. The problem, as Banger said, is management. Managing large projects is a political act requiring political skills, which are radically different from and antagonistic to the skills needed to do good technical work. As a result, most people managing computer projects do not understand development work and make fundamental errors over and over again. A good example is the present healthcare.gov debacle, which is to be cured by adding more people to the project. As was demonstrated nearly 40 years ago in The Mythical Man-Month, adding personnel to a project after it has begun delays completion. Politically speaking, however, the managers have to do something, so they add people in order to cover the asses; the project therefore must be delayed and, if resources run out, fails. The only large projects which succeeded on time that I have observed were those few which were given adequate resources in the beginning, a very difficult trick to pull off managerially.
This has nothing to do with Open Source versus proprietary code or anything like that. It seems to be inevitable.
It’s enough to make me scream when I think about it now. And the worse project I was involved with went on year after year after year of blind allies and idiots–the project even has a minor legendary status within the contracting community. It was either massive corruption or some weird Satanic ritual–I couldn’t figure it out.
“Open source has nothing to do with anything here. The problem is a management issue.”
It’s pretty amazing this even has to be pointed out.
“In general, open source projects, products, or initiatives are those that embrace and celebrate open exchange, collaborative participation, rapid prototyping, transparency, meritocracy, and community development.”
In other words, the exact opposite of the beliefs held by those in the Most Transparent Administration Evah.
http://opensource.com/resources/what-open-source
The idea of open source doesn’t make much sense when the privacy, and integrity, of people’s health care, and/or financial, records is concerned.
The comment about the problem being “management” is absolutely correct. Large software projects have been failing since the 1950s. The problem is always management. Management fails to specify the problems correctly. It fails to specify the solutions it will accept correctly. Management fails to manage the software development process, and to insure that the status reports are actually timely, and accurate.
Management chooses to believe hype over reality, and fact.
Management is always the problem, in the end.
Open source would actually be an advantage from a privacy standpoint. The concept of open source is not that anyone can view the data, but that anyone can view the code base/process. This would allow for creating software that people could be confident would allow for the data to only be used in the manner for which it was intended.
This is crude and simplistic, but generally software projects fail to meet objectives because:
1)
a. The originator has difficulty knowing/communicating the purpose and description of what is wanted.
b. The implementers fail: to fully understand goals, to design appropriate response, to assess and plan for time and resource requirements
and/or
2) The implementers (all else being equal) fail to follow an iterative process of quality control procedures of appropriate/some/any kind.
Both 1 and 2 (design and testing) are important, though not exaustive by any means, for scalability.
Obama’s priorities are always first and foremost to obfuscate and keep secret what he’s up to. In this case his goal is not to create a software system that helps and enables people to sign up for cost effective health care as much as it is a software system that benefits private monopolistic insurance companies and hides/obfuscates that goal by appearing to provide the user/applicant with all sorts of cost saving choices. And that is something he can’t simply go up to the design team and explain clearly, or have his agent go up and explain clearly, or have congress explain clearly, or anyone, no matter how complex or burdened the procurement process is with red tape; because the principal goal of government mandated theft by private enterprise can’t easily be admitted to without creating additional problems. So the thing is a mess to start with and is simply doomed to huge difficulties in implementation.
How do you implement a lie? It goes without saying that Open Source disqualifies itself from the principal goal; to gull the public into an abusive relationship with private enterprise with the government acting as Guido for those who might question the relationship.
In the case of the NSA and NASA, the agencies know precisely what they want and have had the benefit of becoming expert at asking for it precisely. The implementation teams are both experienced at designing and planning for such requests and they have exceptionally rigorous quality control procedures. All of this reflects very clear priorities of the administration that is ultimately at the origin of those efforts. Of cousre NASA is on the decline now and the NSA is on the rise but that doesn’t negate the points being made.
I think you overstate the President’s understanding and management of projects a long, long way below his purview.
1. Why was it a Canadian company who got this contract when technology is supposed to be American’s premier industry now?
2. Why did it cost 600 million+ dollars?
3. Was the contract directed to CGI because cronyism (Michelle’s friend being a CGI VP)?
4. Who cares? This extortion racket to suck money from the American citiens’ pockets and give them little to no health care for it was bound to collapse sooner or later.
The American middle class and poor can’t afford to keep the blod suckers in their trillion dollar profit style they’ve become accustomed to skimming off of the masses with our plummeting wages.
CGI is a Canadian company but is using an all-US workforce for the project. I believe the total contract is $500M but that is over some unspecified timeframe for ongoing maintenance as well as initial programming. This was per the Exec VP of CGI at House Committee Hearing. Both she and representative of CMS spoke like they were attorneys, referring to prior comments, though I don’t know their credentials for sure. In any case, they didn’t know much about software, nor much about the project.
As Healthcare.gov Falters, Silicon Valley Cringes
http://nymag.com/daily/intelligencer/2013/10/silicon-valley-reacts-to-healthcaregov.html?mid=facebook_nymag
“You could take any engineer on the street here and ask them, ‘I have a friend who works for a private company — don’t mention the government — who’s thinking about a five-year, $100 million Oracle installation, and they’ve hired an outsourced contractor to build it for them. It’s going to be proprietary, hosted in their own data center, Oracle-based, with waterfall management. What are the odds that it’s working on Day One?’ And everyone here will tell you: zero percent.”
…
“If the government is going to spend that much money on something, it should be open source,” [WordPress developer] Mullenweg told me in a phone call. “How cool would it have been if the site launched, didn’t work, and some passionate coders came in and said, ‘Oh, here’s a problem, and here, and here?’ But the people who want to help aren’t able to.”
All the discussion of the various reasons that the gateway process to evaluate and purchase insurance that adheres to the new ACA guidelines miss the point entirely. Failure was always the only possible outcome regardless of which boogie men were involved. Changes of this nature are always most difficult to make.
I spent 23 years as a successful agent of change in extremely stagnant and self-destructive social groups (including both governments and corporations). During those years I was responsible for over 200 major change initiatives directly effecting large numbers of corporate players and sometimes even larger groups of citizens. I was always successful, on time and under budget. My costs were several orders of magnitude cheaper than all of the others who sought to be involved and my time frames were always many times shorter than those required for others.
Here are some of my observations:
You can never change any social system by including those affected by the change in the discussion of the systems shortcomings and failures or more importantly the vision of a more effective system. They are completely invested in the current state of the system and often very delusional about the system, its purpose, its function and its failures.
You can never force change on any social system. Force only generates countermeasures.
Change induced by logic and rational discussion is almost never possible. If people could be rational and understand the logic, the change would have already occurred.
Social systems are so biased toward their current state that individuals in the system can almost never envision any alternative approach. Any concept of change generated from within a social system is therefore simply lipstick on a pig.
Over time all members of social systems lose perspective of their interests and goals and begin to behave superstitiously. Indeed, most human behavior is not a function of conscious thinking but simply rote responding to incorrect or unreliable stimuli. More importantly, almost all humans fail to honestly access the world and even when attempting to behave rationally will therefore make very poor decisions.
Members of social systems almost never understand their own motives or recognize the actual issues they face. They are confounded in their perceptions by their own prejudices ingrained by participation in the system.
Change almost never occurs as a reaction to the failure of a social system. The social system is incapable of self analysis and will always place the responsibility for failures on external structures. This means that the main function of crisis points is to reinforce the very processes and structures that created the crisis.
Agents of change must be constantly vigilant to accurately understand the real world. They must define accurate metrics to access a system and be relentless in controlling their own biases.
Change is an artistically creative process guided by the discipline forced by science. As such, it is very difficult to train individuals to become agents of change.
Successful changes occur when the new system is overwhelmingly more effective than the existing system. People only change when it is isolating and painful to remain stagnant and the change affects them in a completely positive way. Social inclusion and support are usually the major reasons people change – the logic of the approach, accomplishment of goals, ease of process, morality and monetary rewards are almost never the actual reason someone changes.
Almost all problems are perceived by those experiencing them to be complicated and intractable. The truth is most can be solved by extremely simple processes.
Simple solutions are generally found by an individual or a very small group.
Simple solutions are generally quick and cheap. If they are not quick and cheap, they are probably wrongheaded and will be ineffective.
Simple solutions are difficult for people to accept. Therefore, they need to be implemented with minimal discussion, quickly and without fanfare. By definition, simple solutions diminish the prestige and importance of individuals vested in complicated perceptions. If the solution is correct, people will be drawn to it as moths to a flame. Indeed, it is often the case that the more individuals understand about the nature and structure of the change being implemented; the more likely they are to actively oppose it. The slower change is implemented, the more forces opposing the change rally. The most effective changes are those that individuals never see coming.
Change agents are universally hated. Even the most successful changes that help members of a social group realize a much more effective and fulfilled lives will not mitigate the ire expressed to those individuals who caused the change. People just don’t like change. Worse, positive changes always shine a light on what was before illuminating just how far awry people had gone. In many ways successful change is an indictment of many individuals’ beliefs in their own honesty, rationality and free will. No one likes being demonstratively wrong. In the end, change agents are almost always sacrificed.
This is an excellent summary of my own experience in change management in a number of private industries. I would neither add nor deduct a single word.
The key takeaway is: “The truth is most can be solved by extremely simple processes.”
And that’s a fact, Jack.
Simple is not easy (hat tip, I think, Ian Welsh). Single payer is simple.
Johnson sounds like a nutcase.
I am not sure that Obamacare can work without open source. Consider the number of entities that have to plug into the network. Insurance companies, credit rating agencies, nearly all individual business who have to report their employee wages and the insurance options they are offering (which I note was suspended), the IRS, the NSA is in their some where I am sure, all the state agencies which will come with their own list, the federal exchange itself, as well as the state exchanges, and of course the front end for those surfing at home or on their phones. And I am sure that’s only a partial list.
How are all of these systems to be integrated if it isn’t open source? The problem with proprietary is that every one has to use the same system to be compatible. It’s a one size fits all solution for a very vast network system. That is an afoul lot of stuff that has to be integrated by the proprietor of the software. When ever some one has a problem, they have to take it to them, and THEY have to fix it. But as a general rule, each patch usually creates more bugs.
Open source allows every one to address their issues at there end, and not have to take their problems to the overlord and then start praying that they will get to it some time this year, or that their solution will actually work.
And open source will actually enable innovation, where as proprietary actually tends to stifle it. Look at cell phones as an example. Android platforms has basically taken over. While its not exactly open source, it is a lot more flexible and permits the users a lot more freedom in working with the software. Yes, you get silly little apps like the bro-stash or tickle-the-stripper, but its amazing the sort of innovations that these apps can do for business, corporations, and what ever. There are apps to report pot holes, apps to help old people track their pills and symptoms, rout planers for delivery people.
Where as before, you could only select from apps the proprietor saw fit to release. And they tended to only be your typical business applications, you know the rolodex, social planer, and so forth. In other words, apps that were use for the proprietor.
The problem I have observed with these observations is that they round counter to the Librarian world view which tends to reject the notion of public space. The tragedy of the commons is usually floated as an argument against it. And open source is part of the software version of the commons. So it doesn’t surprise me that Obamacare went with proprietary software.
That is not to say that it can’t work, but there are limitations to how well it can work. It’s as if Obama is trying to prove the Libertarians are right, by showing that government can’t do any thing. It kind of reminds me how his stimulus package “failed” so that we can never talk about it again.
“And open source is part of the software version of the commons.”
You have no idea what you are talking about.
Open source software is surely in common use, but it is not a perishable resource so there can be no “tragedy” of its over consumption by the community.
Nor is it publicly owned, controlled, or otherwise subject to change except by the small coterie of developers who put their own names on how — and how well — it functions.
>> Open source software is surely in common use, but it is not a perishable resource so there can be no “tragedy” of it’s over consumption by the community.
Nice straw man, dude.
I said that Libertarians will often argue that open source suffers from the tragedy of the commons. I did not say I held this view point.
By all means, help me to understand this better. But do at least address my real arguments please? So, what else did I get wrong?
“I said that Libertarians will often argue that open source suffers from the tragedy of the commons. I did not say I held this view point.”
I’m a very long-standing (lower case) libertarian of several decades and have never encountered another libertarian — inside or outside the tech community — who expressed that groundless opinion.
Your suggestion that libertarians “often” make this specious argument deserves citation of a publicly accessible example or two. Or a retraction.
Since open source software is invariably protected by exceedingly well drafted copyright protection, it is never at risk of becoming property “in the commons”, even if modified by others and re-published. No one with direct knowledge of open source practices could ever be in doubt about conditions of use subject to these rigidly defined ownership rights.
“The problem with proprietary is that every one has to use the same system to be compatible.”
Not necessarily, but the various systems have to be able to interface with each other. This can work but involves coordinating support from multiple vendors, who tend to fingerpoint at each other when things don’t work. It is easier to work with a single vendor if it has a product that meets everyone’s requirements.
I would like to add some support to the picture being drawn by Ed Johnson. In looking into the tech and government trade magazine web sites, you will see the belt way bandits in action. Long standing DC observers have pointed out that for decades, the Maryland and Virginia suburbs surrounding DC was smothered with successful companies that service the federal government. The federal government may have a lot of employees but it employs even more indirectly through the contracting and sub-contracting system. Relevant to Obamacare is the litany of major computer modernization initiatives from the IRS, to the FBI to the what is actually an operational Smithsonian history of computer technology, aka, the FAA’s Flight Control System. You could probably come up with other agencies that come to mind from over the years of 60 MINUTES reports and other investigative news pieces that were routinely, but not so much anymore, broadcast to the citizenry by major networks and widely consumed.
Here is one of the most telling items from my review of the more recent literature from the past 10 years or so. It involves The IRS. In what is the self licking political ice cream cone developed by the republicans, government waste and fraud is countered by running any agency as a business. This simply means firing or decimation by attrition the government employees, and outsourcing their tasks to government contractors. Any problems that arise, are blamed on the government not operating like a business, proceeding onto further ripping out of the wiring of the agency, leaving people who get paid by the taxpayer to have lunch with the contractors, just to see what’s up with all of the hoo ha going on in the newspapers. Here is a conclusion about what made things better for the government when it came to IT upgrades, new systems, modernization, take your pick of phrases here people.
———————————————————–
http://www.bloomberg.com/news/2011-06-24/irs-chief-tries-to-protect-crown-jewel-computer-system-from-budget-cuts.html
IRS Chief Tries to Protect Crown-Jewel Computer System From Budget Cuts………………..
History of Bungling
The agency has a history of problems with information technology projects. Since 1995, the IRS’s systems modernization efforts have been designated “high risk” and subject to extra scrutiny by the GAO.
In the late 1990s the IRS abandoned a technology upgrade, called the Tax System Modernization plan, after spending $2.5 billion, according to congressional testimony by J. Russell George, the Treasury Department’s inspector general for tax administration.
Early efforts to build a database in the current modernization program were plagued by delays, prompting sharp criticism of CSC, the main contractor at the time. In 2003 Commissioner Mark Everson, disturbed by missed deadlines, ordered an outside review of the contract.
A year later, the delays led the IRS to pull a $40 million chunk of the contract from CSC.
‘Lots of Challenges’
“There were lots of challenges and I think IRS and CSC have taken those on, resolved them and moved on,” said Kevin Kelley, a CSC vice president who has been involved in the modernization project since its early days.
A key shift for the IRS came in 2004, when officials began developing in-house expertise that allowed them to handle such elements of the upgrade as systems integration and project management. That change enabled the IRS to improve quality and cost control.
“There’s just been a huge improvement,” David Powner, director of information technology management issues at the GAO, said in a telephone interview. “The IRS is now an organization that does a lot internally. There’s less reliance on outside contractors.”
In 2008 the agency simplified the project, sidelining plans to include retirement plan and business tax records in the database and focusing on a new master file geared to individual taxpayers.
A preliminary version of the database has been in use for several years. For the 2010 filing season, according to a March 2011 GAO report, it processed 41 million “relatively simple” returns — about 30 percent of total individual returns received.
————————————————————
NOT TO BE OUTDONE, MY FINAL ILLUSTRATION OF GOVERNMENT CONTRACTORS WITH THE FBI.
I’ll only serve up a snippet from the very beginning of the piece, Lambert will be blowing all of his remaining gaskets, but in a few sentences, the FBI’s problem speaks for itself and covers almost every problem we scream about here and elsewhere about privatization, spying by contractors, even before Snowden!! and military contractors who do it all for the Federal government, your merchant of death and mine, Lockheed Martin.
————————————————————
FBI grapples with out-of-date computers
http://news.cnet.com/FBI-grapples-with-out-of-date-computers/2100-1028_3-6094070.html
Four years ago, a former FBI project manager lamented the state of the agency’s primitive electronic case-management system.
“There’s no mouse; there’s no icon,” the official told the U.S. Senate Judiciary Committee in July 2002, according to a recent government report. “There’s no year 2000 look to it. It’s all very keyboard-intensive.”
Not much has changed since then. According to recent reports, a string of managerial blunders, financial indiscretions and assorted snags have accompanied efforts to modernize the agency’s computer systems.
A former government contractor assigned to an earlier incarnation of the upgrades was sentenced Thursday to three years of probation, six months’ home detention and $20,000 in restitution after pleading guilty in March to “exceeding authorized access” to FBI records, the agency said. According to court filings, he abused his network administrator privileges and used free hacking software that’s readily available on the Internet to crack 30,000 agency user names and passwords.
Despite that latest embarrassment, the FBI says a turnaround is near.
The bureau in March sealed a six-year, $305 million deal with prominent defense contractor Lockheed Martin to start over. For the upcoming year, it’s requesting $100 million from Congress to launch the four-phase, 42-month overhaul, known as Sentinel, with the target completion date set for 2009.
“In the past few years we have struggled with our information technology programs,” FBI Director Robert Mueller told a Senate committee in May. “However, we have learned hard lessons from our missteps, and we are doing things very differently this time.”
Lambert: “… ObamaCare itself being a prime example of this, except that the rentiers are government contractors and not the health insurance industry…”
This is confusing to me since health insurance companies are government contractors. For example: Medicaid.
Also, since QSSI has been involved with the implementation of software for the HealthCare.gov website and has recently been named as the general contractor to fix the problems with the website, it should be noted that QSSI is owned by United Health, a health insurance company.
For all I know United Health is developing a drone for the government under the motto: if they’re dead they can’t make an insurance claim.
With all respect to Clay Johnson, his points are miss the issue. Reform of the government software procurement is long overdue and millions know it. Obamacare.gov Already exists in one form or another, now it has to be fixed.
Agile, open source are great concept with their own mountain of limitation, but they don’t belong here. They have absolute nothing to do with the problems of Obamacare.gov.
Let’s stop piling words on top the million nonsense words that media supplies us with the last two weeks.
I will take it a step further
Concepts like Agile can operate as snake oil where management or leadership does not want to be accountable or want to actually manage
Agile is only as good as the management team and problem trying to be solved
A complicated mess with poor managers is not going to stop being a hot mess with poor managers because Agile or any other technical consideration is involved
As another causal factor – when the Affordable Care Act was passed, it was assumed that most states would have their own exchanges. When the Republican governors and state legislatures blocked this, it expanded the anticipated load on the system by a significant amount.
“Blocked” it? What a curious phrasing.
What happened is that stuffed-shirt policy wonks dreaming their dreams at the federal level made grand assumptions without even bothering to ask around (cuz they’re just so damn prescient and all) to learn which states were unlikely to go along with the joke.
I’m sorry, I was being overly polite rather than factually accurate. I should have said “When the rabidly partisan stuffed shirt hacks infesting the Republican Party decided to do everything they could to prevent people without access to health care from obtaining affordable insurance, and when five members of the Supreme Court decided to overturn sixty-plus years of precedent regarding Congress’ power under the Commerce Clause, it greatly increased the prospective load on the Federal Government’s portal.
Perhaps I can clarify a bit.
The Administration erroneously assumed that conservative Republican governors, ideologically opposed to the expansion of the federal government, and advocates for the ceding of federal programs to the states to be administered as best suits the needs of their own individual states, would want to administer their own state exchanges/health insurance programs. Who’d have thunk they’d act contrary to their stated beliefs and cede their power right back to the federal government when given a chance?
With the infrastructure for a nation-wide system in place, one (significant) piece will already be in place for a single-payer system. Removing existing features, e.g. state-specific choices of private insurers and plans, is far simpler than adding new ones.
Wasn’t there a lesson to be learned here from the example of Thomas Drake with the Trailblazer (fiasco) Project?
not so sure that it was because it was a government contract (there are 100s of private business contracts) for IT services that fail. just usually nobody knows out side of the company. it there is just as much politics in corprate America as in the political realm. nor is lsdt minutre changes just a public contract problem. nor is short sighted planning not a problem just for public contracts. while Agile might have help, its really intended to make sure that function works. the problem hasnt been that so much as not enough recource to do the job. nor will open source help. it will generally end up costing you more (since no company supports your implemntation, except your self). maybe had they set it up so that for each state there were separate instances of the back end. so that as more states were added to the Feds role, there would have been a lot more smoother increase in demand. but thats want the choice made. might have been better to have one conractor responsible for all of it, as opposed to having what 40 contractors involved. but thats not just in the public arenea as it applied to business projects to. they both can make messes
Developers always think the solutions to every problem is technical
Most aren’t very good managers
And as others have imtplied – that’s the problem
Better IT doesn’t make a hot mess better
Smart management does
Ultimately Neoliberal public private systems are airways going to make management dumb
Which means the IT will be bad
Indeed open source would have made this worse
No, it would have made it impossible — quite a different result.
Indeed, while FOSS tends to do well at implementing strictly-defined standards for communication between machines, FOSS tends to work against institutional and/or legal constraints, and the enforcement of those constraints is critical to the mission of the Exchanges and every other web shopping cart.
This is the sort of multi-actor, multi-party system that needs to be engineered from the top down, not evolved from the bottom-up. It’s unsettling to imagine anyone who was around to watch Congressional staffer Mitch Glazier’s private lawmaking for the RIAA advocating that every individual should have the right to make binding policy in the form of code. Unbounded creativity and opportunistic optimization are neither desirable or welcome in bridge-building. Unfortunately, they were far too welcome in the design phase as long as they provided the outcome desired by those whose desires society deems worthy to support.
And that’s really the bottom line, isn’t it? In contemporary culture, livestock’s only proper seat at the table is on it, and the livestock enforce this amongst themselves. Another fine self-licking ice cream cone, innit?
Exactly right.
Wish I had read before posting
What I’ve seen is that development is about function, the user AND management’s desired outcome
Its not magic and no process is right for all circumstances
I can only speak from my experience as an attorney dealing with Agile and open source.
It may or may not have worked. You are right in that bc Agile really requires greater managerial commitment it would have failed here. It also requires IMO simpler projects rather than a massive project like ACA
Its also the case that open source would present greater chance for security breaches.
However from my POV the biggest issue would be the lack of real risk management even if it functionally produced a better outcome
Here there are many issues beyond functionality that constitutes effective
“Its also the case that open source would present greater chance for security breaches.”
No, that danger would be greatly reduced. Closed-source software is generally less secure because it is less thoroughly reviewed and tested.
What you apparently don’t “get” is that open source code gets reviewed by acknowledged experts world-wide. This CANNOT HAPPEN WITH CLOSED-SOURCE CODE. Else it would no longer be closed-source — right?
The people involved in the open source movement are a voluntary meritocracy which grants street cred to individual contributors based on their ability to produce good work. No corporation or government can afford to assemble that kind of collective “juice” or gain by the integrity of work motivated to satisfy critique from both users and other programmers.
The reason GPG is solid as a rock is that it’s been beaten to death by well established experts in cryptography which the USG could not hope to hire or afford to pay for. Many people prefer it to PGP because it’s unknown (and unknowable) whether PGP contains a back door for government spooks.
Any back doors in open source would be quickly detected and shut. Dwell on that thought for a moment, especially in light of recent revelations by Snowden. The fact is that NO CLOSED SOURCE CODE IS ALSO “TRUSTABLE”. We now know that as an indisputable fact. We also know NSA didn’t claim they could penetrate PGP. But we don’t know if they only chose not to admit it.
Open-source GCC is now pretty much the gold standard for C compilers because it’s also rock solid, freely available, and runs on just about any platform you can imagine. It’s therefore widely used by commercial companies to compile their closed-source applications. It’s preferred by virtually all open source applications and is arguably one of the most thoroughly proven packages in the world. (Eventually there was just nothing more found to make it more reliable.)
The people who develop and support open source software cannot be compromised by fear of retaliation by whoever is filling their rice bowls if they don’t pretend all is well. (O-care seems the obvious poster child for this problem.)
Oh, and then there’s this most vital merit of open source: the individuals who contribute also undersign it with their own names and professional reputations. It’s personal. Very.
I’m not going to argue the tech other than to
Say it depends on the circumstances
Back in the real world of humans, the more layers you add to a bureaucracy that’s already a lot of coordination the messier it becomes
On that human level the more insecure it becomes
This is philosophy or ideology
Practical realities
How do they write successful scalable software that obscures the fact that you are being fleeced or that government has your arm pinned behind your back in case you want to complain?
Exactly the way this administration is doing it; with great difficulty, lots of missed deadlines, absurd complexity and most definitely not with Open Source.
Opacity rules!
Is it really that CGI is just inept? I have just assumed that the general critique is wrong.
Given my experience with the Gov’t and web contracts, I think the root of the problem is the out-dated non-realistic spec guidance that the contract gives to the contractors. You know, like — you have use source-code that is compatible to these server parameters… the type which automatically puts you back 10 years in time — sort of thing.
And, since the Gov’t was the project manager (you know, 9-5, no weekends), basic incompetency. What kind of project manager on a large scale website gets those hours, lol.
Open-source is a bit of a pipe-dream, maybe baby steps with allowing cloud.
The long hours of IT are necessary due to the power and social structures in the professional workplace and are utterly unconcerned with actual productivity.
If you’re doing it right, software can be developed better and more quickly by 40-hour-a-week workers than 80-hour-a-week death marches. As best I can tell, those people have a psychological need to be violated and shortchanged.
I agree on that, but I’m was about the project manager, on a project of this scope.
Did your project succeed? If so, why? (Not snark, a serious question.)
IT’s venerable muckraker Robert Cringely published something interesting on the matter yesterday: Big Data is destroying the US healthcare system.
OT, but is Cringely still going out with Pammy?
Got favoritism?
First Lady Michelle Obama’s Princeton classmate is a top executive at the company that earned the contract to build the failed Obamacare website.
Toni Townes-Whitley, Princeton class of ’85, is senior vice president at CGI Federal, which earned the no-bid contract to build the $678 million Obamacare enrollment website at Healthcare.gov. CGI Federal is the U.S. arm of a Canadian company.
Townes-Whitley and her Princeton classmate Michelle Obama are both members of the Association of Black Princeton Alumni….
As reported by the Washington Examiner in early October, the Department of Health and Human Services reviewed only CGI’s bid for the Obamacare account.
I watched a portion of the House Committee hearing on the Obamacare IT fiasco this weekend. In general it was lots of legal-sounding evasions, “I don’t have that information”, and few answers, with instructions from Congressional reps to get the missing information to them by 9 am the next morning. What could be gleaned however was stunning.
The pieces to Obamacare involve many entities. CGI was responsible for most of the front end user interface (but not the user registration portion, at minimum). Each entity had tested their portions at the last Congressional hearings on Sept 10 and reported they were working as required, prompting the report that the IT was ready for launching. But the software had never been tested on the end-to-end integrated system…….. it was 2 weeks prior to launch that CMS first tested the system and discovered there were “glitches” (which they admitted were unrelated to traffic volume). Ideally there should have been months built into the schedule for testing. Why wasn’t it done? There was NO lead developer/contractor. CGI was hired to write the code for the web interface but nobody had been placed in charge of implementing the software across all the systems.
It appears they now have somebody in charge. The question is whether the problems can be fixed in the allotted time (Nov. 30).
I just see this as more evidence that our tax dollars are being looted by no-oversight government contracts.