The Federal Aviation Administration’s troubled and potentially hackable $40 billion air traffic control upgrade is flying blind: The agency hasn’t made a comprehensive effort to identify potential threats or vulnerabilities in the system.

That finding comes in a recent report from the Government Accountability Office (GAO). The lack of a detailed risk assessment is particularly crucial given that the system must be interoperable on a global basis and work seamlessly with several other global air traffic control upgrades, specifically with a similarly ambitious air traffic upgrade being done by the EU.

The FAA has developed an international strategic plan that identifies several challenges to implementing its Next Generation Air Traffic Control System (NextGen), including those of interoperability. “However, FAA has not conducted a comprehensive risk assessment or analysis of threats and vulnerabilities specific to NextGen interoperability,” the GAO said.

“FAA has taken positive steps to better coordinate its NextGen interoperability efforts across the agency—through the development of an international strategic plan and establishment of two internal bodies to guide and monitor international activities—but the agency lacks a process for comprehensively assessing and managing potential NextGen interoperability risks on a routine basis,” the report said.

The lack of a comprehensive risk assessment by the FAA could create a cascade of failures and delays in the future due to a myriad of decisions that need to be made absent a complete risk analysis. “FAA’s lack of an approach for identifying and assessing potential risks makes it more challenging for FAA to develop a strategic approach for mitigating risks,” the GAO said.

“In all engineering projects, and particularly software engineering projects, this usually means understanding the consequences of risky decisions as early in the life cycle as possible, lest the costs of unwinding previous bad decisions become prohibitive, and the architecture becomes a source of change friction that burdens efficiency of execution,” says a report from the National Research Council (NRC) that also examined the FAA’s process around the NextGen system. “By contrast, an effective architecture can be a basis for risk assessment and mitigation and can be used as a tool to support decision making and the recording of decisions,” the NRC said.

Air traffic management on a global level doesn’t require identical systems because there is a wide variation in levels of traffic density and traffic complexity; systems and aircraft can accept and use information and services from each other for technical or operational purposes, the GAO said. Global standards bodies work out differences in systems and then the various air traffic service providers agree on and implement compatible standards, procedures and technologies.

Although the FAA’s international strategic plan “includes a high-level problem definition of the global trends and associated international engagement challenges affecting global interoperability” the agency “has not identified or assessed whether or how these or other factors might affect NextGen interoperability,” the GAO said.

Risk Management ‘Heavy on Process and Procedure’

Risk management inside the FAA is “heavy on process and procedure, but there is little insight inherent in the artifacts and outcomes of their risk management process,” the NRC said. The NRC requested a list of the top five risks to NextGen from the FAA and the agency failed to supply “any quantified representation of the top risks,” the NRC said. “[I]n an environment with an effective risk management process, the top several risks—whatever they were, and there will always be risks—would be well known and internalized by everyone,” the NRC said.

FAA officials themselves acknowledged to the GAO the lack of a comprehensive risk assessment. In addition, the FAA told the GAO that the agency lacks a mechanism for: comprehensively identifying or tracking interoperability risks to NextGen programs on a routine basis, evaluating potential consequences across its programs, and identifying potential impacts on global interoperability efforts.

In its defense, the FAA responded saying their existing processes, including participation in international working groups, demonstrates their commitment to addressing risk. The GAO countered those statements saying although those are important steps they don’t go far enough: “A comprehensive risk assessment entails identifying risks throughout the entity, considering different types of risks that might affect the entity, assessing the likelihood of those risks and incorporating an analysis of those identified risks to provide a basis for managing and responding to the risks.”

Despite the existence of an internal strategy, without a comprehensive approach to risk management the “FAA cannot develop an effective strategy—within FAA and with international partners—to mitigate risks and target and prioritize resources to best achieve its NextGen interoperability goals,” the GAO concluded. The GAO recommended setting procedures for a comprehensive risk analysis, periodically reevaluating those risks and documenting how it will mitigate the risks.

The FAA agreed with the recommendations, said the Department of Transportation in its response to the GAO report. “It should be clear exceedingly clear that failure to achieve international harmonization and interoperability is not an option, and the FAA’s international priorities reflect this fact,” DOT said.