When the lights turn on, the faucets work, and the trains run on time--that's a good day. Dragos exists to safeguard civilization.
This post is a first in series that will describe hunting, discuss best practices and explain our approach and lessons. Because hunting in industrial infrastructure is important to all of us and with focus and effort we can accomplish it.
Industrial control system (ICS) networks are very defensible, yet it seems increasingly difficult to convert them from defensible to defended environments. There’s a sense, an intuition, that they are ripe for being hacked. For years, the internet device search engine Shodan has shined the light on internet-connected industrial devices. ICS-CERT has annual incident response engagements measured in the hundreds as well as a myriad of news stories articulating even more. And of course, there are a handful of partially understood campaigns such as Dragonfly and Sandworm that targeted industrial sites around the world leveraging HAVEX and BLACKENERGY 2 respectively. We've only had four cases of ICS tailored malware deployed against targets: STUXNET, HAVEX, BLACKENERGY 2, and CRASHOVERRIDE. And yet we understand we have an ICS threat landscape that is mostly unknown due to a lack of visibility into the environments and threats we face.
I submit that industrial environments have two fundamental security challenges: Lack of visibility into what devices exist (and their function) and the expertise to safely respond when something is found. Anecdotes of steel mills, power outages, and espionage are powerful but incomplete and unsatisfactory. Our MIMICS project tried to address this anecdotal problem by trying to measure public sources. Our project continues today, made available to our WorldView customers, and it is constantly evident that adversaries are active in this space.
Visibility is a hard problem. You cannot defend something that you cannot understand, measure, and be knowledgeable on. We keep trying to answer questions that are not fully articulated or are based on copy/pasting best practices from IT security and not necessarily finding the right solutions for ICS. This is combined with the knowledge gap of industrial incident response. There just isn't enough prior examples, lessons, tools, or practices to generate a community of industrial incident responders. We as a community have made traction in areas such as advocating for network security monitoring (though it's no magic bullet), focusing on firewalls and perimeter defense (though our perimeters are quickly eroding), and trying to structure active defense mechanisms such as incident response (but, again, with little experience).
Threat hunting can address these challenges. Threat hunting creates a proactive and iterative process to find threats and by doing so deeply understand the networks, processes in place, and gaps in security postures. It allows for a fine-grained understanding and reporting of these environments. If you have unknown opportunistic malware or a sophisticated actor operating in your environment, then you are best positioned to detect it. Many proactive analysts have successfully done this on an ad-hoc basis. Hunting is about moving the blue team from art to science.
There’s huge opportunity to take this concept to iteratively increase our visibility - to turn anecdote into risk level understanding. To make defending ICS environments doable.
Some examples that illustrate today's challenges:
Hunting is regarded as the latest trend in finding threats and reducing dwell times in IT networks, but it is simply the codification of what has worked over the years into formal approaches that can drive sustainable success. Hunting requires the assumption that breaches can and do happen. While ICS environments are definitely a target, and definitely have breaches, it becomes a hard sell to hunt in an ICS environment and routinely find zero threats. It is a high effort process, and if it is measured only in finding evil, it will have a very low return on investment.
But hunting in ICS requires accepting more assumptions to create a return on investment:
The assumption that your security program has unknown blind spots.
Threat hunting helps to prove your people, processes, and technology actually does what you say they can. Threat hunting first and foremost recognizes that red teaming is not nearly as effective as the blue team validating their systems work. As the last vignette suggested, hunting gives organizations the empirical data they need to escalate issues when a new deployment doesn't meet expectations.
The assumption that your staff requires hands-on experience to couple with the training they get.
Hunting, if done right, can create the framework for an analyst to learn the environment and skills needed to scope, troubleshoot, understand, and synthesize new information about the environment. It's how you can successfully move the knowledge out of one person and into the team. How you put into action the investments in training you make.
To conclude with three immediate benefits of hunting:
Dragos was founded on the knowledge that evil exists in the world targeting our civilian infrastructure and that human defenders can be empowered through process and technology to safeguard civilization.
Would you like to know more?