On December 23, 2015, system operators at Ukrainian Kyivoblenergo (a regional electricity distribution company) noted the tripping of seven powerful substations. Soon, they were scrambling to manage the domino effect playing out in front of them, having realized that hackers had gained access to critical control systems designed to monitor and protect electricity networks—including getting kicked out of the system and being left powerless to initiate any response.

Over a three-hour period, three different distribution systems were attacked, causing approximately 225,000 customers to lose power. Ultimately, operators had no choice but to shut down the recently installed control system and revert to more traditional—indeed, manual—operations and system repair.

In one regard, Ukraine was lucky: It still had the knowledge and manpower to revert. Many systems that have relied on modern control systems for a longer time would find themselves less capable of dealing with the fallout of such an attack.

Investigation of the attack revealed that an email to the corporate side of Kyivoblenergo delivered malware, which was inadvertently transferred via a data key to the operations side. There, it lay dormant for months, quietly monitoring operational commands. Once the malware got in the “backdoor,” it was unstoppable. The breach was so extensive, Kyivoblenergo had to toss out its new control system and start anew.

“Utilities tend to think that with SCADA [supervisory control and data acquisition] systems they are isolated and protected,” says cybersecurity expert Frances Cleveland of Xanthus Consulting International. “The Ukraine attack and more recent attempts with Crash Override prove otherwise. In fact, companies in the industrialized world would have a harder time dealing with such an attack as they simply don’t have the experienced manpower to send out for the manual repairs that saved the system in Ukraine.”

Takeaways from the attack analysis, says Cleveland, underline a persistent naiveté in the sector, i.e., that IT experts know how to hack-proof systems and come to the rescue if an event occurs. She makes the case that while their expertise is vital, the unique features of electricity systems require early and ongoing engagement of operational experts.

Cybersecurity requires action by utilities around system resiliency and protection of cyber-physical assets.

“The typical IT response to a cyberattack is ‘Turn everything off!’ Operators know that is not an option in terms of providing essential service and avoiding damage to physical assets,” says Cleveland. “They also have a different mindset about what matters most, including requiring reaction times to threats within milliseconds.”

As smart grids comprising a wide range of distributed energy resources become more widespread, the question arises of whether they are more vulnerable to cyberattacks. After all, while utility companies remain responsible for power delivery, they no longer own or control all the elements—rooftop solar panels, electric vehicles, individual storage systems, etc.—which are now active players in generation and distribution.

“Introducing more [distributed energy resources] systems does not inherently increase the number of vulnerabilities on the grid,” says Galen Rasche of the Electric Power Research Institute. “It does, however, increase the attack surface in that systems rely on more communication channels, more automation and the application of more security controls.”

Cybersecurity, according to a report jointly produced by Xanthus and EPRI, requires that utilities plan strategic action on two levels: system resiliency and protection of cyber-physical assets. Resiliency speaks to the need for critical infrastructure to be designed to not only prevent malicious cyberattacks and inadvertent failures, but also to cope with and recover from such incidents. The cyber-physical aspect arises from the reality that both types of assets are tightly intertwined: As much as an attack on one can affect the other, it is also true that engineering and cybersecurity techniques can be used in combination to improve resiliency.

Tension Between IT and Operational Security

The potential disconnect between corporate/IT and operational security requirements can become an issue. For example, risk assessment typically calculates the likelihood of attacks and what their possible impacts would be, but from an operations perspective, the possible impacts of the mitigation actions on real-time operations must also be taken into account.

While IT specialists are used to the rapid development of software, cryptographics and certificates, operations experts better grasp the requirements for keeping the lights on. They may also have different views of exactly what needs to be protected in the event of an attack.

“A lot of utility executives are legitimately extremely concerned with confidentiality of their customer data,” says Cleveland. “Those data are important to business, but largely irrelevant for operations.”

To mitigate the likelihood of a successful attack, Cleveland emphasizes four elements related to operational data:

• Authentication—being certain the system is designed to know who is at the other end of every communication activity and can identify interlopers
• Authorization—ensuring that only the right people or systems are authorized to take action
• Data integrity—putting in place measures to ensure data about operations is not compromised
• Availability—designing the system, often with redundancy, so that the right data are available at the right time and in the right place

Suggesting that utilities hold sole responsibility for cybersecurity would be misguided: In fact, they can only protect themselves. Much of the shift to distributed energy resources is driven by aggressive national, state or municipal policies, but policymakers also play a critical role. In part, this relates to ensuring that standards, regulations and guidelines are in place for distributed energy resources manufacturers, installers and aggregators. EPRI is currently developing a white paper on cybersecurity for multi-party grids, while the International Electrotechnical Commission is updating its cybersecurity standards and guidelines to ensure their applicability to interactions between utilities and distributed energy resources.

“A key element of protection is ensuring that all these [distributed energy resources] never communicate directly with the utility, and vice versa,” says Cleveland. “The devices all need to share information, often extremely rapidly. But each device needs to follow the principles above to determine which communications it trusts and is willing to accept and how it will respond.”

As the Ukraine attack shows, in extreme cases, it may be necessary for operations to override IT security protocols, a scenario that insiders refer to as “breaking the glass.” Faced with a massive blackout and extensive system damage, the right mitigation action might be to bypass cybersecurity and focus on keeping electrons flowing.

“There is enough commonality in the attack attempts reported in different countries on different systems to stimulate collaborative action,” says Candace Suh-Lee, senior technical leader at EPRI. “Industry players have come together to look at the big picture and boost investment in research on both engineering and cybersecurity mechanisms to improve protection.”