Hack the Plant

From the Archives

Episode Summary

“I started Hack the Plant in August 2020, driven by a deep conviction that we need to do something about the problem of critical infrastructure assurance: from education to action. Over 4 years and 40 episodes, we’ve hosted industry giants and practitioners, members of Congress and policymakers, and hackers and scientists; because with technology it’s the people, those on the front-lines that are making the difference.” - Bryson Bort Bryson closes out season four with a look back at the work we’ve done so far.

Episode Notes

For the final episode of the season, our host Bryson Bort reflects on four years and forty episodes of Hack the Plan[e]t, and picks a few favorites. 

Episode 8, DoD and Critical Infrastructure: https://hack-the-plant.simplecast.com/episodes/dod-and-critical-infrastructure

Episode 10, The Congressman, The Commission and Our Critical Infrastructure: https://hack-the-plant.simplecast.com/episodes/the-congressman-the-commission-and-our-critical-infrastructure

Episode 27, Managing Incident Responses to Critical Infrastructure Attacks: https://hack-the-plant.simplecast.com/episodes/managing-incident-responses-to-critical-infrastructure-attacks

Episode 28, Cyber Threat Intelligence Over the Past 25 Years: https://hack-the-plant.simplecast.com/episodes/cyber-threat-intelligence-over-the-past-25-years

Episode 36, Supporting Ukrainian Electrical Grid Resilience in Wartime: https://hack-the-plant.simplecast.com/episodes/supporting-ukrainian-electrical-grid-resilience-in-wartime-mxxhn2g3

Hack the Plant is brought to you by ICS Village and the Institute for Security and Technology. 

Episode Transcription

Bryson: I'm Bryson Bort, and this is Hack the Plant, season four. Electricity, finance, transportation, our water supply. We take these critical infrastructure systems for granted, but they're all becoming increasingly dependent on computers to function. We walk through the world of hackers working on front lines of cybersecurity and public safety to protect the systems you rely upon every day.

From the ransomware threats of Colonial Pipeline to the failure of the Texas power grid, it is clear our interconnectivity is also a significant source of risk. This season, we will continue to bring you a panoply of different insights across all of the different things happening in critical infrastructure.

In my day job, I'm the CEO and founder of Scythe and the co founder with Tom Van Norman of the non profit ICS Village, where we educate people on critical infrastructure security with hands on examples, not just nerd stuff. I founded Grimm in 2013, a consultancy that works at the front lines of these problems every day for clients all over the world.

I'm also an adjunct senior advisor at the Institute for Security and Technology, a 501c3 think tank dedicated to tackling technology driven emerging security threats. This is Scythe. Is Hack the Plant, brought to you by the Institute for Security and Technology and ICS Village. Subscribe wherever you find podcasts to get each episode when it drops.

Today, we're doing things a little differently. We're closing out season four at Hack the Plant, and as we look ahead to what we want to accomplish in season five, I find myself looking back on the work we've done so far, I wanted to share with you the. Five episodes that I feel are the most relevant to the challenges we're facing right now.

I started Hack the Plant in August, 2020. I know quite a while ago. Thank you for joining us and keeping with us. Driven by a deep conviction that we need to do something about the problem of critical infrastructure assurance from education to action. Over four years and 40 episodes, we've hosted industry giants and practitioners, members of Congress and policymakers, and hackers and scientists.

Because with technology, it's all about the people. Those on the front lines that are making the difference.

My first pick dates back to February 2022. Episode 8, DoD and Critical Infrastructure. It featured Lieutenant Colonel Douglas Fletcher, Chief Data Scientist, and Lieutenant Colonel Erica Mitchell, the research lead for Jack Voltaic, a program of the Army Cyber Institute out of the United States Military Academy.

Jack Voltaic was a series of exercises spearheading a major multi sector effort aimed at understanding critical infrastructure dependencies on force deployment. Congressional funding for the project has since run out, but the Army Cyber Institute is reportedly looking for ways to continue the program.

I still continue to serve with this project on the West Point Cyber Science Advisory Board. Every now and then, we get to see how small the world is. Erica was a cadet a couple of years behind me at West Point, where I lent her my car once. And the world can look pretty small for force projection too.

The U. S. military is the most capable in the history of humankind. But it has to get there. I like to say the best tank on the battlefield fight is the one you don't have to fight. Our forces are heavily dependent on civilian critical infrastructure. And Jack Voltaic was the first project to look at this with posts, bases and ports across the country.

I personally helped broker some of the agency relationships. And my dual hat as a board advisor to the Army Cyber Institute and a senior advisor for critical infrastructure to then CA director Chris Krebs. 

Erica: So Jack Voltaic is a research series that has started to look at the interdependencies between civilian critical infrastructure and the DOD.

And so the first iteration was born from a, a cyber mutual assistance workshop. So the energy sector for many, many years has had the concept of mutual assistance. If there's a big storm in one part of the country, you'll see teams being dispatched from all over the country to get those lines back in place and restore service in the affected areas.

So, uh, CW3, uh, Judy Esquibel had the idea that what if we could do the same thing with cyber? And that was back in 2016. And so she put together a cyber mutual assistance workshop that brought in some leading industry players, uh, and worked on developing these public private partnerships and really bringing everybody to the table to say what would it look like if we had to respond to cyber?

How would our interdependencies affect that cyber response? How could we possibly assist each other? And so after they had the first, uh, cyber mutual assistance workshop, they decided to try and exercise. So JV1, uh, was conducted in New York City and focused on a physical attack coupled with an attack on the financial industry, as well as the, uh, the subway.

And what that showed was, one, there are a lot of silos of excellence, as we like to call them. Within each of the critical infrastructure sectors, there are communication pipelines where people report on incidents, but there was not a lot of cross talk between those different critical infrastructure sectors.

And so after going through that, it actually led to the development of the New York City Cyber Command in order to kind of have some type of unified cyber response and be able to minimize the amount of information that stayed within these silos. And, uh, we've maintained a relationship with New York City and, and continue to do workshops with them, you know, even to the present day.

And after that, we looked at JV2. 0. Um, we decided to take it a little bit further and do something a little bit bigger and looked at a hurricane coupled with a, an opportunistic cyber attack. And with that hurricane scenario, one thing we noticed was we kind of took the ports out of play. Uh, we wanted to know what would happen, uh, at our Surface Distribution and Deployment Command, which is an army, um, transcom Battalion that kind of focuses on the movement of army equipment.

They participated, but because the hurricane closed down the port, it kind of took a lot of the cyber off of the table for them. And so, we had a lot of good findings come out of Houston, um, some of the similar findings from New York City that we still have a lot of these silos of excellence, um, by moving to another large city.

They're very, very responsive to the physical. Their reaction to the hurricane is, you know, on point, they, they know how to react to that. They've hosted several large events. They know how to react to any type of physical issue. But when it comes to cyber, one, it's just, it's hard to know that it's cyber at first, you know, the first instinct is, hey, I have a glitch in my system.

Let me restart it. Or, hey, You know, I'm experiencing problems. I wonder if anybody else is, and let me just kind of wait and see what's going on. And so that is still an issue. 

Bryson: Next up is episode 10, the congressman, the commission, and our critical infrastructure. We recorded this episode with former Wisconsin representative Mike Gallagher.

In April of 2021 to learn more about the cyberspace solarium commission. The commission was a bipartisan intra governmental body whose goal was to help create a strategic approach to defending the United States from cyber attacks of significant consequence. The commission. Was sunsetted in December, 2021, but continues today as a nonprofit led by Mark Montgomery and is featured in episode 37.

I was fascinated with the cyberspace solarium commission from the first time I'd heard about it. Like many of us working in the space for years, it can feel disheartening looking at the overall system and the incremental progress that feels both too slow. And not enough. The Commission was a whole of nation effort to do something about it, and enjoyed bipartisan support.

Mike's character really shines in this episode. He's not just another politician, but someone who really cares about the problems facing our country. The latest Cyberspace Solarium Commission 2. 0, a non profit constituted out of it, continues the work at cybersolarium. org. In this clip, Mike is answering my question about the role of government versus private industry in cybersecurity efforts like the Commission.

Mike: Well, I think we have to recognize the fact that, you know, When it comes to cyber, 80 percent of the critical infrastructure is owned by the private sector. And that's, that's not going to change, right? And so that requires, first and foremost, I think, a paradigm shift for how the national security bureaucracy approaches the problem, uh, and, and a recognition that in some ways they are not.

They're not the main effort. They are the supporting effort and they culturally, they have to sort of change from this, this posture of, of need to know to, you know, a duty to share and add value to the private sector. You don't want the private sector constantly suspicious of working with the federal government, either cause it's going to compromise their internal information or, or hurt their, their bottom line.

So I think there's an overall cultural shift that, um, needs to occur. Um, and I think. The federal government then needs to distinguish itself in certain key areas where the private sector simply can't, um, compete or, or, or just isn't involved in, right? I mean, there, you know, are a very sensitive intelligence streams that the federal government, um, has that, um, it could do a better job proactively sharing with the, the private sector if they're, you know, um, infrastructure has been compromised.

Um, you know, there, there are, you know, you know, specialized personnel that work in the private sector that can add value to the private sector. And then I think if you go back to, you know, incentivizing the private sector to step up, we really want the, that the, the culture of cybersecurity to permeate.

Um, through our companies in the United States. So the question is, how do you incentivize things like 1 10 60 reporting? 

Bryson: What is 1 10 60? It's about breakout time. The measurement of the amount of time it takes an adversary to take action on their objectives during an attack. It calculates the time from initial access when the adversary first gets in of a given incident.

To them being able to successfully move laterally within the victim organization, ultimately landing on the asset or assets they are targeting during their campaign. It consists of time to detection organization has set a goal of allowing only one minute to detect an incident or intrusion time to investigation.

The length of time it takes to find out. If the incident is legitimate, next steps, organizations should aim for 10 minutes, and time to remediation, the amount of time it takes to kick them out and restore operations, which should be about 60 minutes. We're jumping ahead to April 2023, Episode 27, Managing Incident Responses to Critical Infrastructure Attacks, which featured Leslie Carhart.

The Dragos Director of Incident Response for North America. Leslie and I took a deep dive into the inner workings of incident response and vulnerability assessments. Most don't realize it, but the majority of enterprise cybersecurity defense is built on detection and response. And how do you ensure breakout time is contained?

Your incident response isn't your last resort. It's the largest contributor to minimizing the inevitable breach through visibility and time to response. Leslie and I went on to speak together at the RSA conference in 2024. Preparation is key for incident response, so we talked about the good, the bad, and the ugly of tabletop exercises.

Leslie shared their experiences as a threat researcher in an ever changing environment. 

Lesley: Yeah. I mean, we see it all. We see it all in this field. It's like working in an ER kind of, but I mean, like worse, like working in an ER in a mining town or something, you know, like we see, we see everything in, in OT and center response.

And, you know, that's a mix. of categories of incidents. We see insider cases, both intentional and unintentional insider cases. We see, um, we see a lot of crime where, so crime actors are getting smarter about where they're doing things like ransomware attacks. They're less haphazard. There's probably less overall attacks now, but they're more smartly performed.

So they're targeting more critical industries. They are targeting people who they think will have to pay less defended industries. So that leaves. industrial kind of wide open for a lot of that type of criminal activity, uh, financially motivated activity. And then there's still adversaries out there who are adversary groups who are more state style, who are building their capabilities to launch attacks in the future and conducting espionage, preparing to do sabotage.

And that's still happening and they're getting better at it. Like there used to be like this security through obscurity thing with ICS and they're still kind of is like. If you want to do a specific thing to a process, like tamper with a water supply or turn off the power to a city, you know, that takes a lot of knowledge of the industrial process and systems.

Cause there's a lot of controls in place, human and otherwise to prevent that from happening. But if you spend 10 years in a system. Because you've got the funding to do as a, as an army from a country or an intelligence agency, you're going to know how to do that. You're going to have the experts on staff who know how that system works, and you're going to have built up the knowledge to be able to do the bad thing.

And even those criminal activity actors, like the ransomware actors and things, They're building up their knowledge base too, because they're becoming like, that's a billion dollar industry now. Like they're very well resourced criminal groups. And if they want to do something bad to make money, they're pivoting their operations away from again, from haphazard ransoming, like they're, they're going to be able to have those types of capabilities soon too.

So um, that security through obscurity thing, isn't going to work much longer. 

Bryson: My next pick is the episode we released after Leslie. I sat down with senior research scholar at Columbia university school for international and public affairs and cyber risk and conflict expert, Jason Healy for episode 28, cyber threat intelligence over the past 25 years.

Jason was here to discuss his October 2023 Lawfare article looking back at 25 years of White House cyber policies. Those who cannot remember the past are condemned to repeat it. For those of you who don't know Jason, he has been one of the stalwart leaders of government cybersecurity and policy for decades.

Cybersecurity is a young discipline, but with a dense history of success, failure, and rapid innovation. We tend to forget or not heed the lessons of the past in our relentless pursuit of improvement. Perhaps a retrospective from someone who has driven a lot of it, studied more of it, and done the homework would help.

Jason explains what he gets into in his article. 

Jason: Yeah, so the White House has been trying to get their arms around these solutions for 25 years. If you look back at the very earliest White House document, Presidential Decision Directive 63, it came out. In 1998, and they don't really mention operational technology, right?

It's very focused on I. T. Um, they're focused on critical infrastructure, but they don't really make any differentiation about about I. T. N. O. T. And it's they're so optimistic back then, Bryce, and it's so cute to see. They say, you know, within five years, we're gonna have, you know, most of the solved with 10 years.

Americans. Critical infrastructure will be secure as if it was a one off as if we could just Get it right once and then it would just be secure. But of course we have intelligent adversaries and we keep inventing new technology. So even if we could get it to a state of security, we would move off. So over the last 25 years, you've seen a lot of trends stay the same.

White House doesn't give themselves public deadlines like that anymore to have the whole thing secure. But they've continued to talk about things like information sharing. They've continued to talk about, um, a lot of these themes where you see the biggest difference, Bryson is first off is in regulation.

They started 25 years ago in saying, look, the market, you know, our solution is going to be through the market. And we saw that all the way through, um, Uh, the Obama reports, uh, strategies and the strategies from the Trump administration. The biggest difference in this one, at least in, in, in the major content is this new strategy coming out and saying that we need to regulate, that the market has failed.

They, they, they talk about market failure at least five or six times and that. We need regulation and they push regulation a couple different areas from, um, software liability or liability for software manufacturers to baseline regulation for critical infrastructure. There's a lot more in here about operational technology, so it was nice to see these changes because it's like if you've seen if you if you like sports, right?

I've been watching a lot of American football lately and you see teams that continue to lose. Mhm. And you say, okay, well, we shouldn't be having consistency anymore between coaching, right? If you, if you've been at this for 25 years, your strategy shouldn't keep covering the same things year after year, right?

You've got to mix up your coaching style if you want to succeed. And we are suffering the same things from 25 years ago or even 50 years ago. So I am glad that the strategy made a break and started to go out in these new areas. 

Bryson: The last episode is from season four, senior IOT security strategist at Cisco Talos Intelligence Group, Joe Marshall.

He joined us for episode 36, supporting Ukrainian electrical grid resilience in wartime when Russia invaded Ukraine in 2022. Joe helped coordinate a multinational multi company coalition of volunteers and experts to find a technological solution. Joe has the honor of being the first guest invited back on the show.

His first episode, we looked at cyber security and big agriculture. A year later, he came to me and asked if I could solve a timing problem in the electrical grid. I didn't have all the details, but I certainly was stumped by the challenge. Well, he wasn't, and he doggedly pursued a solution that would help Ukraine.

Maintain its grid for its people with the help of his company, Cisco. It's not hyperbole to note his actions saved innocent lives. In this clip, Joe explains to me how he got involved with the project. 

I was like, I was so enraptured by the stories. Of hardship and innovation and things they were doing to be able to just keep the lights on and they mentioned something really offhand like it was just like item one of 50 that they brought up to me and it's actually the least sexy of all the things they were mentioning had nothing to do with.

Guns or missiles or whatever they were like, yeah, we can't even get accurate timing to work on our transmission grid because of jamming that is interrupting GPS communications and this little thing in the back of my head went off and it goes, Hey, that's really important for synchro phaser management and for, you know, being able to monitor the health of the grid.

My time in the utility had taught me that, but I haven't thought about it in eight years. And I was like, Oh yeah, that's, that's kind of important. And I, I kind of, I said to him, I, uh, I said, Hey, uh, why don't you guys just go buy time at clocks? You know, that way you don't have to worry about GPS timing.

And the guy looked across me from the table. He's like, cool, you're going to cut me a check. Those are 30 to 40, 000 a piece. And I need well over 40 of them. And I'm like, man, unless you're asking for pesos, I can't help But even then, I don't know if I can afford that. So. But it was a really interesting problem because I was like, how do you do grid synchronization and timing when you don't have a discipline clock that everybody steps to, or at least can measure and timestamp to?

I was like, man, I don't really know. But then I made a very naive assumption. I went, dude, I work for the largest hardware manufacturer in the world. I am stone cold handsome and brilliant. I got this. I'll figure something out. And my little neurodivergent brain just wouldn't let go of it. And it was just this weird, one off thing.

And I went, nah, we gotta have something. Answer is, we did not. And we would have to go make that thing. But the journey to get there was emotional highs and lots of lows, 

Bryson: but we got it. That's all folks. We'll be back here with you on March 11th for season five, where we're tackling four of our most vital lifeline sectors, electricity, healthcare, food, and water.

We know that our interconnectivity makes us vulnerable to our enemies, but what can we do about it? The first episode of season five. Features my good friend, Josh Korman, founder of I am the Cavalry, an executive in residence for public safety and resilience at the Institute for Security and Technology.

Josh leads the Undisruptable 27 Initiative, a program of Craig Newmark's Cyber Civil Defense Initiative that works to drive more resilient lifeline critical infrastructure for our communities. Josh was also one of Hack the Plant's first guests. You might remember him from episode two, Where is the Cavalry?

We discussed Josh's idea for experts devoted to improving the security of medical devices, transportation, connected homes, and infrastructure. Look, the world's getting crazy. The good news is, we've been paying attention. The bad news is, it's about to get a lot worse. But we'll be ready, together.