Florida Water Treatment Plant Hacked

, ,

Over the last several years, we’ve seen utilities increasingly become a target for cybercrime. On Friday, February 5th, an unidentified assailant remotely accessed a computer that is a part of the SCADA system at the City of Oldsmar’s water treatment plant. In an attempt to poison the city’s water supply, the assailant briefly increased the amount of sodium hydroxide by a factor of more than 100. Thankfully, a supervisor noticed this increase and immediately reversed it, causing no harm to the water supply.

While beneficial to utilities, SCADA systems open the operation up to many cyber threats when the proper protocols and security are not in place. In 2019, we wrote the following article for the Tennessee Association of Utility District’s publication, Tennessee Utility News, outlining benefits, threats, and best practices for protecting SCADA systems.

Have questions?   Need help securing your system?

That’s what we’re here for! Reach out, and we’ll be happy to help!

Making Data Work for You:

SCADA for Utilities

Throughout the utility sector, SCADA (Supervisory Control and Data Acquisition) systems are becoming more and more prevalent. SCADA systems are a type of industrial control system that enables utilities to easily collect useful data, monitor operations, and automate many processes. For the utility industry, SCADA systems can have many different applications. SCADA systems can give operators and administrators instant access to information that was previously unavailable or difficult to obtain. SCADA systems can be used to monitor and control pump stations, monitor water in tanks and reservoirs, perform remote shut-offs and turn-ons, collect data for regulatory and financial reporting, among many other useful applications. These networks are not a one size fits all solution, however. They may be small or complex, only address one small portion of operations or encompass them all.  SCADA systems are becoming essential to water and wastewater system operations, monitoring, maintenance, and planning.

The heart of SCADA is the data it generates.

SCADA systems are typically made up of field equipment, data collection equipment, communication networks, and software.  Field equipment such as sensors or relays allow the utility to monitor and automate their system through the creation or consumption of data. This data is communicated via fixed or wireless communication networks through the use of Remote Telemetry Units (RTUs) or Programmable Logic Controllers (PLCs) that are connected to the sensors or relays in the field. In most cases, these RTUs or PLCs perform real-time, continuous data collection, giving the utility easy access to data that enables operators to create and maintain a high level of control.

We’ve established that SCADA systems communicate data, but where is this data going? How is it being turned into action? The final piece of any SCADA system is a host platform. The host platform is comprised of hardware such as servers or a cloud solution as well as software. It is here where the actual benefit of a SCADA system comes to fruition. It is here where data is received, integrated, analyzed, and turned into actionable items. The large amounts of data that are generated by SCADA systems can be onerous and overwhelming. Having software and solutions that ensure that this data is processed properly, organized, and stored is essential to the performance of the SCADA system.

Protecting SCADA data and connected systems.

Just as SCADA continues to increase in popularity, so does the threat posed by outside sources. Over the years, we’ve heard mention of the threat posed by terrorists, hackers, and less than satisfied customers and employees. How do you protect your system from these outside threats and ensure that you avoid any loss of service?

The threat of cyber-attacks and cyber warfare is growing each day, and we see every day that utilities are not taking the necessary steps to protect their network. This topic is one that is constantly being discussed, but many times, no changes are being implemented. The two most common reasons are that many utilities do not grasp the reality that it could happen to them or the cost seems prohibitive to protect and monitor their system.

Threats to SCADA systems include but are not limited to:

Employee Error: Employees are a common cause of network security issues. While it is possible to suffer the wrath of a disgruntled employee, most commonly, network security issues caused by employees are unintentional and can many times be attributed to poor training, carelessness, or a combination of both. With proper training for employees and an active culture of vigilance, you can help your organization reduce issues caused by employee error.

Malware: Malware includes viruses, spyware, and an array of other malicious programs. These programs may not necessarily target SCADA, but because the SCADA network is not separated or protected, they are able to traverse over the entire network, thusly affecting the SCADA system.

Hackers: Intentional, malicious individuals or groups that are intent on gaining access to the network. These hackers could also use this data against you. By manipulating data or gaining complete control of your SCADA systems, serious harm can be caused to services, customers, and the entire operation.

"Using a combination of security policies and controls to adequately secure today’s systems is critical to your organization’s operation"

So what can you do:

The first step utilities need to take is documentation of your network connections to the internet. Any and all internal networks must be properly documented. All hardware, software, firmware, and applications need to be part of that documentation. All users, including outside vendors that have access to these systems, should also be documented. It is vital that utilities create and maintain accurate and thorough documentation of all connections to, pieces of, and access to their network.

In most cases, SCADA systems lack adequate monitoring and detection systems, making them vulnerable to attacks from external and internal sources. Setting up monitoring and detection controls is the next step in protecting your SCADA system. There are many different types of monitoring and detection software on the market. Selecting the software that is best for your particular operation can be a difficult task. We suggest that you work closely with network professionals to evaluate your specific network needs.

Once you have documented and set up adequate monitoring and detection systems, you can then begin to segment the network. Segmenting the network should be used to separate other business systems that are running on the same network as the SCADA system. Due to the fact that attacks are increasingly exploiting both physical and cyber vulnerabilities, it is important to align physical security and cybersecurity processes. Application whitelisting, firewalls, and gateways are all ways to build a defensive perimeter around your SCADA systems.

Security is also something that is in continual motion. Rules, security checks, report monitoring, and standardized processes must be instituted and utilized by everyone who has access to the SCADA system and all other connected networks. Regular evaluations must also be performed for vulnerability, risk, and all assets in general.  These assessments should be conducted on a regular basis to verify that security measures are adapting to the changing threats on the IT landscape.

Using a combination of security policies and controls to adequately secure today’s systems is critical to your organization’s operation. Understanding common weaknesses, creating and implementing an action plan to bring security to an acceptable level, and employing standardized processes will minimize the risk posed by an increasingly hostile Internet environment.

Itron Utility Week 2020

, , ,

Over the last two weeks, our Sales & Marketing has had the privilege of participating in Itron Utility Week and the Itron Partner Conference in a virtual format. While these conferences looked different this year, we could connect with utilities, industry leaders, and partners from across the country with innovation at the center of many conversations.

We heard from Itron, Inc. President & CEO, Tom Deitrich on the importance of innovation in our industry and how Itron Inc. solutions and ideas can be harnessed to meet the challenges that the utility industry is facing today as consumer expectations change, cities become more interconnected, and sustainability needs increase.

“We can empower innovation by sharing fresh ideas with endless possibilities, and together, create a more resilient, more reliable, safer and more resourceful world.” – Tom Deitrich

Throughout the last two weeks, we had the opportunity to witness how Itron, Inc. solutions can allow utilities and cities to address these changing needs and create a better future for the industry. The use of intelligence and analytics offered by these solutions will enable utilities to increase customer engagement, reduce water loss, better manage energy, and respond to changing real-time conditions. With the ever-increasing amount of data at utility’s fingertips, it’s more important than ever that we turn that data into actionable items. Having information is the first step; turning this data into progress is the second. Many utilities struggle with managing and acting upon data. Many solutions and ideals discussed over the last two weeks give utilities the tools needed to ride into the future, better prepared, and ready to ensure long term sustainability.

We want to thank Itron, Inc., for the ability to participate in these thought-provoking and informational sessions. We look forward to sharing the valuable insights gained with our customers!

The Utility Industry Finds Itself in the Middle of a Major Workforce Transition

Issue:

The Utility Industry is experiencing a large shift in employment as many of today’s utility employees will soon be eligible for retirement. This imminent talent gap is further exacerbated by a lack of qualified talent to fill these positions as well as a need for utilities to adapt to changing technology. According to the US Bureau of Labor Statistics, the Utility Industry employees approximately 1,345,000 individuals in the US. Of those individuals, approximately 636,000 (27%) are at or nearing retirement1. As is evident by these numbers, utility employees tend to be “lifers.” Meaning, the industry as a whole has historically had a low turnover rate and as such, many current employees have been there 20+ years and gained 20+ years of experience.  This tendency for employees to maintain long term employment, while beneficial in some manner to the utility, has caused the industry to be ill-prepared for this shift in employment. Many utilities now find themselves unprepared to recruit qualified employees and unable to effectively offer the necessary training for new employees. Simply put, utilities have little to no experience with looking for and training new employees.

Due to a shift in population trends and newer generations’ desire for different types of career opportunities, there is no potential to fully replace the current utility workforce as individuals retire. As a whole, there are fewer skilled workers in the workforce with some estimates pointing out a 50% decrease in graduating engineers in the last 15 years. In addition, newer generations bring a different perspective to the workplace. They have a different set of expectations and values as well as the tendency to be more tech-oriented.  Many young individuals just don’t see utilities as a place to build a career. Newer generations are looking for challenging, forward-thinking work environments and aren’t afraid to quickly change places of employment.

Solution:

In order to adequately address the issues caused by an aging workforce, utilities need to look for alternative solutions to many of their current issues. Are there more efficient ways to accomplish tasks that may result in a need for fewer employees? View this as an opportunity to evaluate what you’re currently doing and determine what is the most cost, time and labor efficient. Look to implement talent management strategies and build active succession plans. Look to hire replacements long before highly skilled employees depart. Explore the idea of manual process reduction and embrace the ideas of automation and the reduction in labor associated with solutions that automate daily tasks like Automated Meter Reading or an interactive Customer Web Portal.

Utilities can also look at partnering with local schools and organizations to educate newer generations on the exciting opportunities working at a utility brings. Build strong programs locally and work on showcasing sustainability. The newer generation is no longer driven purely by profit. They seek to work for organizations that display environmental consciousness and seek to have a greater purpose in their positions. Embracing technological advancements can also help to keep these employees engaged and excited about their positions.

Issue:

If the idea of an almost irreplaceable workforce doesn’t scare utilities enough, they also have to face the loss of knowledge associated with the departure of tenured employees as well as a need to keep up with and adapt to ever-evolving technology. Utility workforce retirement not only equates to a loss of labor but also to a loss of critical knowledge. For years, the same individuals have been doing the same jobs in the same exact manner. Utilities are notorious for not documenting exact job functionality and steps for problem-solving of daily issues. On the job training has traditionally been performed via word-of-mouth and on-the-job interactions with tenured employees. Even so, it is impossible for a seasoned utility worker to efficiently and effectively communicate with a new employee how they successfully manage every aspect of their job. Much of the knowledge and skill needed to effectively manage the utility is stored in the heads and hands of the aging utility workforce and only passed down to newer employees when specific situations arise. Because of these nuances, utilities may have to take multiple shots at replacing a single employee or even replace one employee with several employees. The traditional word-of-mouth training methods utilized by utilities are no longer viable.

Solution:

It would be almost impossible to fully address the issue of knowledge loss. That being said, steps can be taken to set utilities up to educate and train the next generation and reduce risks associated with the loss of knowledge and skill. Utilities need to focus on documenting and outlining processes. Put initiatives in place to have employees document daily tasks and the rationale behind their solutions. Implement a program of rotation and cross-functional assignments so that employees can be trained and knowledgeable in a variety of areas within the utility in order to help reduce the knowledge loss created by a single employee. Work on making a list of jobs that are critical to the functionality of your operation and the skills needed for those positions. Prioritize succession plans based on when workers plan to retire and implement training plans accordingly. Develop knowledge sharing programs and partner with neighboring utilities to share knowledge and train new hires.