Eight companies originally pursued the project to design the new Coast Guard offshore patrol cutter, smaller than the national security cutter. It is intended to replace its fleet of 210-foot and 270-foot medium endurance cutters.This short description of the GOA decision is pretty accurate so far as it goes, but a reading of the whole decision reveals a close attention to detail that renders the implication that the decision was based on arbitrary subjective criteria out of place. I also will not provide all of it, so you should read the whole decision at the link. I do hope, however, to provide enough information taken from the decision to suggest a more dispassionate judgment.
In a rare move, HII and another shipbuilder filed a formal protest with the federal government. Between Ingalls and Newport News Shipbuilding, HII is the government's most productive shipbuilder. But the Government Accountability Office recently denied the protest in a 21-page decision that sheds light on what factored into a highly competitive bidding war for a program presumed to be worth an eventual $12 billion.
A side-by-side comparison of the bids showed HII running even with the three eventual winners in factors such as sound design, its design approach, organizational management and production capability.
The key difference was past performance. In this case, the company's past performance on other programs came into play, including the San Antonio class amphibious transport dock ships, the GAO report showed. HII was rated "marginal" while the three winners were deemed "satisfactory."
Regarding the San Antonio class problems, HII spokeswoman Beci Brenton said the company "has vigorously applied lessons learned, markedly improving quality and efficiency with each successive ship, resulting in very successful trials and, we believe, a very satisfied customer." "We are seeing the benefits of lower costs, higher quality and increased learning which comes from serial production," she said.
"I think it was a big surprise to the industry that Huntington Ingalls was not included," said Loren Thompson, a defense analyst with The Lexington Institute, Arlington-based research group. "First of all, they make the national security cutter, which is widely considered to be a success." Thompson said these types of solicitations place a major emphasis on past performance. But in this case, "past performance isn't really indicative of current capability," he said.
Still, Thompson said Coast Guard's source selection authority "is in a legal strait jacket in terms of applying its selection criteria. If the solicitation places an emphasis on past performance, then Huntington Ingalls is stuck with that aspect of its credentials, even though the company is better run today and has better capability."
"You've got to stick with the rules," he said.
As usual, I cut, paste, rearrange, delete citations and paraphrase, so go to the source if you need reliable unedited authority.
Matter of: Huntington Ingalls Industries, Inc.; VT Halter Marine, Inc., File: B-409541; B-409541.2; B-409541.3; B-409541.4; B-409541.5, June 2, 2014 (pdf here)
Huntington Ingalls and VT Halter Marine both challenge the Coast Guard’s evaluation of Bollinger’s past performance, arguing that the agency unreasonably assigned Bollinger a satisfactory past performance rating despite receiving negative information about Bollinger’s performance on the Fast Response Cutter contract. In this regard, Huntington Ingalls contends that the Coast Guard unequally evaluated its and Bollinger’s past performance. Specifically, Huntington Ingalls contends that the Coast Guard ignored negative information with respect to Bollinger’s performance of the Fast Response Cutter contract, while focusing on negative comments in a PPIRS report in evaluating Huntington Ingalls’ performance of the LPD contract.
As a general matter, the evaluation of an offeror’s past performance is within the discretion of the contracting agency, and we will not substitute our judgment for reasonably based past performance ratings. Where a protester challenges an agency’s past performance evaluation and source selection, we will review the evaluation and award decision to determine if they were reasonable and consistent with the solicitation’s evaluation criteria and procurement statutes and regulations, and to ensure that the agency’s rationale is adequately documented. A protester’s disagreement with the agency’s judgment concerning the merits of the protester’s past performance does not establish that the evaluation was unreasonable.
The technical evaluation team (TET) determined that the offerors had corrected all identified weaknesses or deficiencies, and assigned all offerors superior ratings under the two concept design subfactors, and under the design approach factor.
With respect to past performance, the agency’s past performance evaluation team (PPET) reviewed the past performance questionnaires for contracts identified by the offerors for recency and relevance. The PPET also identified additional recent and relevant contracts through the past performance information retrieval system (PPIRS). Where the PPET found negative or questionable past performance information, the PPET had telephone conversations with references.
The RFP stated that the agency’s evaluation of offerors’ relevant past performance would be based on the following elements (in descending order of importance):
(1) Overall Customer Satisfaction – Would the customer select this firm again?
(2) Cost Control – Delivered within budget, provided current and accurate/complete billings, and relationship of negotiated costs to actuals.
(3) Quality of Product – Compliance with contract requirements, accuracy of reports, technical excellence, management responsiveness, appropriateness of personnel, and stood behind warranty.
(4) Timeliness of Performance – Met interim milestones, reliable, completed on time, including wrap-up and contract administration.
Here, the record shows that the Coast Guard’s evaluation of Bollinger’s past performance on the Fast Response Cutter was reasonable. The agency’s past performance evaluation team recognized that the most recent PPIRS report included four marginal ratings in the areas of schedule, management, management responsiveness, and logistic support/sustainment. The PPET also received, however, a more recent past performance questionnaire from the same individual (the contracting officer) that prepared the PPIRS report for this cutter. The questionnaire provided adjectival ratings that ranged from mostly exceptional to satisfactory; there were no marginal or unsatisfactory ratings. The questionnaire’s narrative comments indicated that Bollinger had made “substantial changes” to its organization and to its management, which resulted in improved performance.
With respect to Huntington Ingalls, the PPET determined that the offeror’s past performance was marginal. The PPET received past performance questionnaires for four contracts for the Coast Guard’s National Security Cutters and the U.S. Navy’s DDG 51 destroyer. All were found to be recent and relevant, and included mostly positive comments. However, the PPET noted that a PPIRS report for the National Security Cutter No. 5 contract indicated schedule problems. The PPET also identified three PPIRS reports for Huntington Ingalls’ contract for LPD-17 class amphibious transport dock ships. The PPET found that the PPIRS reports, which covered performance from October 2009 to September 2012, included adjectival ratings that ranged from exceptional to unsatisfactory. In this regard, the PPET noted that, although the customer that completed the PPIRS report stated that it “probably would” again award a contract to Huntington Ingalls, the PPIRS report included numerous adverse comments. For example, the PPET noted Huntington Ingalls’ unsatisfactory rating for cost control, under which the LPD customer commented that “[c]ost performance . . . has degraded” over the performance period. The PPET’s follow-up conversations on the LPD contract indicated that negative performance was continuing. In this regard, the customer for the LPD contract noted that Huntington Ingalls’ work had deteriorated prior to Hurricane Katrina, worsened after the hurricane, and had improved to be “stabilized at a poor level.” On this basis, the PPET concluded that Huntington Ingalls’ failure to meet contract requirements was the contractor’s fault.
The evaluation results were provided to the agency’s Source Selection Authority (SSA), who weighed the comparative merits of the offers based upon the final evaluation reports. In this regard, the SSA identified strengths and differentiating characteristics of the offerors’ proposals under each of the evaluation factors. With respect to the most important factor, concept design, the SSA concluded that all the competitive range offerors had strong technical concept designs and that no offeror stood out. The SSA recognized, however, that various offerors had strengths under the hierarchical requirements.
Under the design approach factor, the SSA concluded that all offerors provided a comprehensive design approach, with few discriminators, but noted that Bollinger offered an approach that increased the government’s confidence of success.
Under the organizational management factor, the SSA concluded that all offerors provided adequate descriptions of their organizational management and met the minimum requirements of the RFP, but noted that the proposals of Huntington Ingalls and Bath Iron Works were the only ones to receive strengths under this factor.
Ultimately, the SSA concluded that past performance provided the basis upon which to differentiate these proposals. With respect to Huntington Ingalls’ past performance problems, the SSA noted that the customer provided updated information that indicated that Huntington’s performance, though improved, remained at a poor level. The SSA agreed with the PPET that Huntington Ingalls’ and VT Halter Marine’s past performance should be assessed as marginal.
Based upon his review, the SSA concluded that, although Huntington Ingalls’ and VT Halter Marine’s proposals met all technical and management requirements and offered some unique strengths, the offerors’ recent and relevant past performance records did not provide confidence that they would be able to satisfactorily perform the contract. The SSA further concluded that Huntington Ingalls’ proposal included “essentially a threshold Concept Design” and otherwise had limited distinguishable benefits in comparison to other offerors. The SSA also noted that considering the firm’s marginal past performance rating, its proposal was the least advantageous to the government.
The protests are denied.
In contrast to the Huntington Ingalls Industries case, another case, Gaver Technologies, illustrates a winning protest when evaluation factors are questioned.
Here, the protest is sustained where record shows that agency’s source selection authority lacked a reasonable basis for failing to give weight to several source evaluation board findings pertaining to strengths in the protester’s proposal. The RFP sought services for computer science, computer software engineering, IT security, networking, application development, and web services. The protestor, Gaver Technologies, Inc., is referred to as GTI.
The RFP provided for award to the offeror whose proposal was considered most advantageous to the government based on the following three factors (of approximately equal weight): mission suitability, relevant experience/past performance, and cost. The mission suitability factor was comprised of three subfactors: technical requirements (worth 45% of the factor weight), management plan (worth 35%), and work management (worth 20%).
Matter of: Gaver Technologies, Inc., File: B-409535 June 3, 2014 (pdf here)
It is a fundamental principle that agencies must evaluate proposals consistent with the terms of a solicitation and, while the evaluation of offerors’ proposals generally is a matter within the procuring agency’s discretion, our Office will question an agency’s evaluation where it is unreasonable, inconsistent with the solicitation’s stated evaluation criteria and requirements, or undocumented. Further, while source selection officials reasonably may disagree with the evaluation ratings and results of lower-level evaluations, they are nonetheless bound by the fundamental requirements that their independent judgments be reasonable, consistent with the stated evaluation factors, and adequately documented.
Of relevance to this protest, the RFP explained that in evaluating offerors’ technical approaches, the agency would consider their overall understanding and approach to accomplishing the requirements of the SOW, taking into account the “adequacy, realism, effectiveness, quality assurance, risk reductions, efficiencies, and completeness” of the information provided in the following areas:
1. Proposed approach of disciplines, skills, and techniques the Offeror plans to use in the performance of this work.
2. Proposed innovative processes, systems, and technology trends the Offeror suggests for accomplishing and/or streamlining the tasks required in the SOW with supportive rationale.
Of significance to this protest and our discussion below, with regard to the technical requirements subfactor, three of the five significant strengths for GTI concerned its technical approach.
The Source Selection Board's (SEB’s) first significant strength was that GTI had “demonstrated a thorough understanding of the SOW [Statement of Work] by proposing the critical disciplines, skills and techniques needed to effectively implement the SOW requirements.” The evaluators found that GTI’s proposed initiatives, as summarized in the five bullets, would “significantly increase the quality and timeliness of contract deliverables,” which was “expected to result in greatly enhanced performance of IT services throughout the life of the contract.”
The second significant strength assigned GTI was based on the SEB’s finding that GTI’s proposed technical approach contained “a multitude of highly innovative processes and insightful technology trends.” This finding was supported by six bullets with explanatory details. The third significant strength was based on the finding that GTI’s technical proposal demonstrated “an in‑depth understanding and innovative approach to accomplishing the requirements in the SOW area of IT Security (ITS).” The evaluators supported this conclusion with four detailed bullets.
In comparing GTI’s and Peerless’s proposals under the technical requirements subfactor, the Source Selection Authority (SSA) noted that while she had initially agreed with the SEB’s observation that the protester’s proposal provided several innovations and insights, she concluded after extensive discussions with the SEB that “there existed offsetting considerations,” which led her to “question the overall impact of the findings.” Summarizing the technical requirements subfactor, I acknowledge the potential benefits stemming from the GTI proposal. However, considering the unknown implementation costs, I find them of relatively lesser value to the Government because I do not have high confidence that the innovations could be implemented. Based on this, I believe that the quantitative difference between the two Offerors is immaterial.
The SSA then compared the two proposals under the management plan subfactor. She noted that Peerless offered an organizational structure that mirrored the agency’s own structure, which, in her view, demonstrated a keen understanding of the GRC’s requirements and culture that would lead to successful contract performance. The SSA also noted that she was impressed with Peerless’s proposed customer feedback mechanism and viewed it as a discriminator in her selection decision. She went on to identify Peerless’s proposed phase-in plan as a second key discriminator, noting that Peerless’s approach left the final 30 days of phase-in for dealing with contingences and unexpected issues associated with start up of the new contract, which ensured that GRC would have “no systematic problems upon full contract initiation.” The SSA further noted that in comparing the proposals under the management plan subfactor, she had considered the significant strength assigned GTI’s proposal for subcontracting agreements, but had concluded that this significant strength was not “of the same qualitative merit” as the findings associated with Peerless’s proposal under the management plan subfactor.
As a result of the above findings, the SSA concluded that Peerless’s proposal had a slight advantage over the protester’s with regard to the mission suitability factor. She further found that while both offerors received ratings of high confidence under the relevant experience/past performance factor, Peerless’s proposal had a slight edge under the factor due to the highly relevant experience of its team members and “the direct experience and overall successful performance of its major subcontractor” on the predecessor contract to the contract here. Finally, the SSA noted that the probable costs of the two proposals were almost equal, with Peerless’s being slightly lower. She concluded that because Peerless’s proposal offered slight advantages under each of the three factors, it represented the best value to the government.
GTI complains that the SSA failed to adhere to the evaluation scheme set forth in the RFP in evaluating its proposal under the technical requirements subfactor by unreasonably failing to credit GTI’s proposal with any of the proposed innovations identified by the SEB, as contemplated by the solicitation. The protester further contends that Peerless’s phase-in plan should not have been viewed as a significant strength, and that the SEB and the SSA treated the two proposals unequally by crediting Peerless’s proposal, but not its own, for offering a management structure mirroring the structure of the OCIO.
As noted above, under the technical requirements subfactor, the SSA considered the protester’s proposed innovations, which formed the basis for the significant strengths assigned by the SEB, to be “of relatively lesser value to the Government,” and she gave them little weight in her source selection decision. In dismissing the weight of the SEB’s findings, the SSA concluded that she did not have high confidence that the innovations could be implemented due to their unknown implementation costs. In order to further shed light on the extent to which the SSA’s concerns regarding funding pertained to the innovations underpinning the significant strengths assigned GTI’s proposal by the SEB, our Office held a hearing.
In our view, the SSA’s explanation for why she did not attribute value to the innovative approaches proposed by the protester that were funded is not supported by the contemporaneous record. The SSA made no contemporaneous reference to insufficiently detailed descriptions in her SSS (she referred only to unknown implementation costs), nor is there any indication in the record that she directly reviewed the content of the proposals herself (rather, she based her decision on the information presented by the SEB). Moreover, there is no evidence in the record that the SEB questioned the value of proposed innovative approaches, such as horizontal protection, on the basis that the approaches were not adequately described, or that the SEB conveyed such a concern to the SSA.
In sum, we find that the record fails to support the SSA’s conclusion that the protester’s proposal should not be given any meaningful credit for the multiple proposed innovative approaches for which implementation funding either was included in the protester’s proposal or was not required. That is, while we recognize that the overall impact of the SEB’s findings pertaining to the protester’s proposed innovations is diminished when items with unknown implementation costs are excluded, we nonetheless fail to see a reasonable basis for the SSA to have given the innovations for which unknown implementation costs was not an issue--in particular, those pertaining to IT security--essentially no weight. Accordingly, we sustain GTI’s complaint pertaining to the evaluation of its proposed innovative approaches.
GTI further argues that the SEB unreasonably assigned Peerless’s proposal a significant strength for its proposed phase-in plan -- indeed, the SSA cited Peerless’s plan to complete phase-in within 30 days, rather than the required 60, as a key discriminator in her selection decision.
The record supports GTI’s position. While Peerless represented in its proposal that it “can complete transition in as little as 30 days because we are very familiar with all aspects of the contract,” it went on to indicate that “[w]e acknowledge a lower level of risk associated with a 60-day transition and planned our schedule accordingly.” Peerless Mission Suitability Proposal at 59 (emphasis added). Thus, while Peerless believed that it could accomplish the phase-in transition in as little as 30 days, it did not commit to do so; rather, it proposed a 60-day phase-in schedule to reduce risk. Because Peerless did not propose to complete the phase-in transition within 30 days, both the SEB’s finding to that effect and the SSA’s reliance upon that finding as a key discriminator in her source selection decision lacked a reasonable basis.
We recommend that the SSA make a new source selection decision, taking into account our findings above. If, as a result, the protester is selected for award, the agency should terminate the contract awarded to Peerless and make award to the GTI. The protest is sustained.
No comments:
Post a Comment