Sunday, 30 March 2014 00:00

Case Study: $250 Million Failure, Who Are You Going To Sue?

Rate this item
(1 Vote)

In order to comply with the Affordable Care Act, the State of Oregon made the decision to build its own Health Insurance Exchange (ORHIX). An online portal to allow applicants was supposed to go live October 1, 2013. As of March 30, 2014 the site was not functional and all ORHIX applications must be processed from paper applications.

This is one of two Case Studies on recent government project failures that I was asked to review. I am providing this assessment as a public service. The other assessment is on the Massachusetts Division of Unemployment Assistance.


The in-depth analysis is restricted to the September 2012 Oracle contracts since that is the only set of original documents with enough data to make an analysis. Two audits reports were also read—one from Centers for Medicare & Medicaid Services (CMS) and the other from FirstData—and Maximus' two quality reports from November and December.

Discrepancy in PO Values (missing)

Figure 1: Discrepancy in PO Values

Lack of Clarity in POs

The September contract for Oracle was for $35.1MM with a very clear assignment to "assist" Oregon on a (T&M) basis to build the ORHIX system. This amount was for an estimated 118,477 hours of services totaling $35,094,504. There are, however, irreconcilable differences on this PO. The PO body has a narrative that defines the work is being contracted. The narrative list defines one extra area of work and the summary hours are inconsistent with those in the subsequent hourly estimates table (even though the grand total of hours for both sets are the same 188,477-hour total). When calculating the value of the PO using the labor rate and estimated hours tables found in the PO, the value should be $34,139,415.62—$955,088.38 less than what was paid to Oracle. Due to the lack of access to the Cover Oregon staff, this is unexplained.

Project Structure

The project had a number of root issues that gravely affected its ability to complete successfully. From my analysis, there were four major issues:

Oracle Contract page (missing)

Figure 2: September SOW with focus on "Assist"

  1. Hiring Oracle on a T&M basis. Time and materials contract placed all the risk for completing the project on the State of Oregon. Oracle's September 2012 contract has very little scope that they had to provide. A large majority of their service delivery lines start with the word "assist" (see Figure 2). They are clearly subservient to either the State of Oregon or a System Integrator (who ended up being the State). Oracle's contract should have been fixed-price for specific deliverables.
  2. Failing to restrict the scope to what was essential to meet the go-live requirements for October 1, 2013. The scope of the project, according to the two independent audits, was not only to develop the ORHIX, but also to include the DHS Modernization project. This created the MaX project. The decision should have been to minimize scope by trimming it to a minimally viable product for the ORHIX go-live in October 1, 2013. The DHS Modernization should have been postponed to a subsequent year and a roadmap developed to define the phase-in goals for it after a successful go-live and system stabilization.
  3. Failing to hire a system integrator with large-scale system integration experience. Projects to build systems of this size are rare. Their chances for success are low regardless of who builds them. Trying to be the integrator with little or no experience has almost no chance of success. A system integrator should have been hired on a fixed-price contract to run and manage the project.
  4. Inadequate project governance and oversight. Numerous issues raise questions on the governance of the project from only looking at the minimal contractual documents that were available. These include:
    • Lack of reaction to the Maximus reports. It is unclear how the Maximus reports could go unheeded. This underscores, however, the most serious flaws in communication, cooperation, and understanding of reality of the project.
    • Hiring Oracle on T&M. As mentioned above Hiring Oracle on T&M removed all accountability. With a contract that primary had Oracle in an assisting role gave them less accountability.
    • Use of the Dell Contract process. It is not understood why the contracts were processed through Dell (these were service contracts processed through a hardware purchasing agreement). Accord to the other audits, the practice of putting these contracts through Dell continued. This only added cost to the project.

The CMS and FirstData audits point out many other areas where governance and communication fell short. This is a major issue that must be resolved prior to moving forward with this project or others.

Maximus Quality Reports

Maximus Quality Summary (missing)

Figure 3: Maximus Quality Summary

Oregon hired Maximus to run monthly audits of the ORHIX project (Maximus refers to these as quality reports). These reports started in 2011, but only two reports were available to review—November and December of 2013. They show multiple areas where the project at high risk for protracted lengths of time. They clearly highlight and document issues early in the project. Figure 2 is one page from the December 2013 report that clearly indicates a history of issues on the project.

These reports underscore a lack of project control since they clearly show the project was in trouble from early 2013. Audits conducted by CMS and FirstData confirm that the project's oversight team failed to heed Maximus' warnings. According to FirstData, "multiple members of the [Legislative Oversight] committee told us they were completely unaware of the Maximus QA role and had not received any of the QA reports".Audit ReportsAs mentioned above, two different organizations performed audits on the project in 2014—after it was clear that the project was seriously missing its goals. These reports add to the findings here. Neither was commissioned to find root causes (CMS was a Federal audit to assess the state of the project, and FirstData was to answer seven specific questions). Both reports are very good, but stop at charting a course to prevent re-occurrence of the problem on this or other projects (this was not in their scope).No reports can be complete, and, although very valuable, FirstData's points out a couple inconsistencies that should be investigated. Three of these are:

  • They report that the decision was made to not use a system integrator in 2011. However, the contract for services with Oracle in September, 2012, clearly states:
    "[Oracle will] Assist the SI Vendor with Architecture, Design Guidelines to build interfaces to DHS systems, Federal Systems, and third-party systems for the following: "1. Real time integrations using the Oracle SOA Suite. "2. Web Services "3. Batch interfaces"
    If the decision had been made a year earlier, why would Oracle say this when the rest of the document clearly talks about work with the State?
  • The list of POs in their report does not include a PO for a Cloud Services contract signed on August 21, 2013. This is a three-year contract for $22MM of Cloud Services.
  • The completion of a complete financial audit on the project. Aside from the discrepancy noted above, FirstData enumerates 43 POs that were issued to Oracle. There are references, however, to another contracted amount prior to September 2012 that is not reflected in their list. Notably, the detail in the Dell Quote BPB11111715-R07 (Addendum Two of Oracle License and Service Agreement US-TERM-GMA-5772-30-NOV-2011) refers to an original contract in the value of $7,241,836. The POs listed by FirstData cannot be reconciled to this amount.


In my opinion, although Oracle is culpable in the failure, they were on a T&M contract and it was the State of Oregon and Cover Oregon's lack of governance and controls that is the biggest issue. The investigation into what happened needs to move on to find the root causes and solve them. That must to be done before any further money is spent on completing the ORHIX project or defining an alternative option. The two steps are:

Table 1: Documents Reviewed
Known or Referenced POs and Contracts
PO DateDescriptionPOContractValue
11/2011 Unidentified services. Document not seen. Ref US-TERM-GMA-5772-30-NOV-2011   X $7,241,836
9/2012 Services Contract. Addendum to US-TERM-GMA-5772-30-NOV-2011 X X $35,094,504
3/14/2013 Software Licenses   X Not Provided
8/21/2013 Cloud services (3-year)
  Core service   X $15,493,111
  Service Option Fees   X $3,528,848
  Enhanced security (8/1/13-7/31/14)   X $2,128,815
Total of POs listed: $63,487,114
  • A complete financial audit to show the actual and complete cost of the system. This should include money spent on ORHIX, DHS Modernization, and MaX (the combined projects). It should itemize all internal and external costs (not just Oracle).
  • A root cause analysis focused on finding solutions, rather than blame. The FirstData analysis, although well done, was commissioned to answers to seven questions, which appear to be looking for blame. Blame does not solve the problem. It is only political fodder. To find answers an audit needs to look at the following:
    1. What were the processes for each of these roles and what changes need to be implemented to make the role effective:
      1. Scope definition.
      2. Project planning (Schedules, project methodology, risk assessment, mitigation planning, assumption tracking, communications, etc.).
      3. Inter-agency and project communication.
      4. Issue resolution and escalation.
      5. System architecture and validation.
      6. System and interface development.
      7. Project Quality: System testing, test definition and approval, and regression testing.
      8. System deployment and training.
      9. Vendor selection, due diligence, and contract review.
      10. Task/resource allocation.
    2. What was the plan around project governance and what is an appropriate plan to make it work?

Constraints on Analysis

According to the March 19, 2014 Assessment Report completed by FirstData, there were 43 POs issued to Oracle with a value of $132MM. There were other vendors and subcontractors involved in the Cover Oregon project. The value of their POs and the State's internal costs are unknown. My analysis was conducted using a small subset of all contractual documents along with assessments from Maximus, CMS, and First Data. No one from the State or Oracle was consulted in making this assessment or its recommendations. This is a serious constraint on this analysis and hence it only represents are a small part of what needs to be done.The POs reviewed include Oracle's September 2012 time and materials (T&M) services agreement and the August 2013 Cloud Services purchase orders (POs). The value of these POs is $56.2MM. The former references an initial PO for $7.2MM, but it was not reviewed. No POs were seen for the software purchase; however, I reviewed the Oracle Software License agreement that listed the packages and seat counts. Following is a summary of these items.Conflict of Interest DisclaimerThis was a non-paid engagement and neither eCameron nor myself has received any form of remuneration for this effort. The genesis of the investigation based on a request from a reporter at the Oregonian who asked for an interpretation of the material that he provided. This assessment is based totally on those documents.With these items completed and addressed, this and other large projects can proceed.

Read 11755 times

Related items

  • Process Mapping

    Process is at the core of any business. It makes work predictable, repeatable, and transferable. Without it we cannot scale our businesses. However, process can be a bane to making progress. Processes that work for a $10 million company have difficulties supporting a $30 million company. Trying to scale them to a $300 million company will not only fail but not address the issues that larger companies have that were never dreamt of in a smaller organization. Processes need to be discarded, revamped, and built—all of that without creating an overburdening bureaucracy.

    Anytime you need to go someplace, you first have to know where you are. Processes are never static and your company's current state is probably far from where you think it is. Hence, the first step is mapping out you company's current state followed by defining the future state. This is more than a logical map of the process; it must also include physical maps. Whether your process is solely to provide a service (say, website development) or physical (say, manufacturing) there are logistical issues that complicate the process flow. Without fully understanding those nuances, future state processes will not reach the desired efficiencies.

    For more information about process mapping fill out the form to the left and we will get in touch with you.

  • Success vs Culture

    The other day a Latvian student contacted me for my views the connection between culture and success criteria—an important and intriguing topic. After working in Taiwan, Singapore, Korea, Japan, Israel, United States, and Canada, I wear many scars of both blatant and subtle cultural violations. I also know that within a culture one person's success is often another person's failure. So, after dispelling concerns about clicking on some random email link, I completed her survey (please feel free to take it yourself). In the process, I struck up a friendship with the student, Kristine Briežkalne, who is studying at Riga International School of Economics and Business Administration . She has some interesting views and presented me with a Venn diagram showing four frames to a project (business, client, project management, and growth perspectives) and how they intersected. As the diagram is part of her Master's thesis, I will let you ponder the how to label the overlapping areas (an eye-opening exercise).

  • Kill The White Knight

    There is a reason we do not teach classes on fixing failing projects. Many a cynic feels that we simply do not want to teach our trade, however, our reason is far nobler—we should be teaching prevention rather trying to create white knights to save the day. It is the same philosophy as building a fence at the cliff's edge rather than an emergency room at its base. Our language is replete with idioms telling us to look past the symptom and address problems at their root cause. 'An ounce of prevention versus a pound of cure' or 'a stitch in time saves nine.' Please, feel free to supply your own in the comments. Unfortunately, most of our businesses loathe this philosophy, waiting to address an issue until it is irrefutably broken.

  • Tales of an Expert Witness: Sex, Lies, and Video Tape (Part II)

    Trust relationships, certifications, and standards sound like such a safe harbor. These sound like such great words in a proposal or statement of work. How could you possibly go wrong building a trusted relationship with a customer by committing to follow a standard? In fact, this can burn you… in court.

    No one ever starts a project with the goal of ending up in court. In fact, litigation may never cross your mind; after all, you have built a trusted partner relationship. Taking a few cautionary steps, however, will make your life easier if you end up in that ill-fated litigious position. Your best chances for success come long before you enter the courtroom—even before the project starts.

  • Comparing Organizational Change Management Models

    A few weeks ago, I set out to write a post on the comparison of various organizational change management (OCM) methodologies and realized that would be a disservice to my readers. It would simply drag you down the path of implementation while failing to focus you on building the foundation. The pressure was too much and I have relented to numerous requests on making that comparison. The caveat is that juxtaposing these models is not comparing different varieties of oranges or even apples and oranges; we are surely comparing the peel to the fruit they contain. Hence, comparing methodologies like Kotter's model (the peel), Prosci's ADKAR (the core), and General Electric's Change Acceleration Process (the whole fruit) need a different approach.

Leave a comment

More Info on Project Recovery

Tell me More!

Please send me more information
on fixing a failing project.

Rescue The Problem Project

Internationally acclaimed

Image of RPP

For a signed and personalized copy in the US visit the our eCommerce website.

Amazon logo
Buy it in the United States Buy it in Canada Buy it in the United Kingdom
Buy it in Ireland Buy it in Germany Buy it in France
Buy it in Italy Buy it in the PRC
Buy it in Japan
Book sellers worldwide.

Upcoming Events

Other's References