- Welcome to The CYBER5 where industry experts and leaders answer five burning questions on one hot topic in cyber. I'm your host, Landon Winkelvoss, co-founder of Nisos and industry leader in managed intelligence and security investigations. In this episode, I talk with director of Red team, cyber threat intelligence, security and cloud engineering for Intuit, Shannon Lietz. We talk about our passion for securability in the world of DevSecOps and dive into details around the concepts of external threat hunting and active defense. We also discuss important metrics to capture in the world of security and how to make cyber threat intelligence actionable. Stay with us. Welcome to the show Shannon. Would you mind sharing a little about your background for our listeners?

- Yeah, hi there Landon and all of the listeners. This is Shannon Lietz. I wanted to state that these are my own views and not that of my employer. Anything that I say here should be taken in that context. I have been in the industry for quite some time. I lead the adversary management capability at Intuit and have been there for quite some time. In my day-to-day journeys, I work on making sure that Intuit keeps customer data safe and ensures that our security controls are working effectively.

- I really appreciate you taking the time. I know you're really busy and you have a lot certainly on your plate as you cover threat intel, Red team, threat hunt, and kind of put all that under adversarial management. So I think this is going to be a fantastic conversation kind of diving into external threat hunting and active defense. What does active defense and external threat hunting mean to you? I think when a lot of people hear that term, they think quote unquote, hack back, which is completely wrong.

- Yeah, absolutely. And it's interesting, you know, when you really kind of look at what the landscape looks like for ensuring that companies have resilient capabilities and software and hardware. Active defense has been a term of art for a little bit of time, but hunting, for a lot longer. And I'd say, you know, as we've been working through our journey with things like DevSecOps and security and adversity management now, active defense is a great term of art. I think it has been defined in a variety of different ways. Some folks have said hack back, others have said deception. I've heard varieties of things and I'd say being mindful and not necessarily putting all of your capabilities into reactive defenses is helpful. Active defense to me is being on top of the fact that you have adversaries that you need to work through understanding and defending against. So I think that's part of it. And then on threat hunting, as we think about active defense, going out, understanding your company's capabilities from the outside in, really ensuring that those things work effectively is part of that journey.

- How do you use intelligence to profile adversarial actors' infrastructure such as analyzing outbound and inbound traffic and how you augment that with what's already been ingested into the SIM?

- You know, traditional threat intelligence has allowed us to have a variety of information as a threat sharing community that tells us when an IP address is doing something not quite right across the entire industry. And when you think about how that applies to your organization, there's a discipline that has to be about what is happening in the industry versus what is happening to your organization and discerning the difference between those two things is really important as part of what the SIM does for most organizations, basically sifting down, what is a security incident, what is a security event, so that folks can respond quickly to any issues that may come up. That discernment of threat intelligence, basically pooling it together and weeding through it is, I think, a critical step. And so there's, I think, having enough threat intelligence about the IP addresses that you might be encountering, being able to do things like attribution, all of those are critical steps in really understanding the adversaries that your applications might have. One of the things that I've said in the past while doing DevSecOps work has been that, I think that building something through the lens of resilience really requires understanding which adversaries you're going to encounter from your application's point of view so that you can do something about it.

- So Shannon, to kind of dive into a little bit of these use cases, you mind giving even a few examples from your career of where this was applied?

- Yeah, absolutely. Looking back on my career, which has been long, everything that I've done through my research has been to really narrow in on the ideal state of security. And so use cases that may crop up along the way are anything from the typical email security issue, malware, some of those, to things like fraud and other types of application specific security issues such as cross-site scripting and XSE. As you kind of look into the dimensions of use cases, my experience with it has been really about trying to discern the specifics of those use cases so you can distill them from maybe log data or event data. But also as you distill them, another area and the reason why emulation is so important is to really help to emulate adversaries and make sure that as your controls are operating, that you're pushing traffic through to determine their level of effectiveness. And by creating this grid, if you will, of use cases, what I believe has really helped in the past has been attributing to persona categories and some of those things, because if you can imagine the use cases, the amount of data, the determination of what a resource is and how that applies in an environment, that level of detail and complexity can be quickly challenging and you have to be able to look at a very complex environment as quickly as possible to really be able to react in the near real time that these events are happening. So when you're looking at like an email security issue as an example of one of those, that email security issue may not need the level of speed that a fraud issue may require. So really, I think that what has helped in the use cases has been understanding timing and periodicity and some of those things. Some of the other measures that are out there, volume, those have all been very critical in being good at these use cases. So if you can, example, compare the email security issue to say the fraud related issues, you know, as you kind of work your way through them, an email security issue, you go back and look at the metrics that are out there from places like Symantec and McAfee and some of the other vendors that are really measuring in the space and you can see that basically one in about 250 emails is the common practice that you'll see something problematic. And that's really going to be independently determined across an organization based on the user's experience with email. So the frequency there is going to be a lot less than say when you're looking at a website that may be encountering a volume of traffic where adversaries are constantly going after it. And those require different ways and rates of speed, which is why you're seeing more and more that a lot of the active defense type of capabilities and controls that are shifting for faster use cases are starting to do things like ML. And I know I say ML and people say, oh is there really ML in the space? But machine learning is really all about how you increase your speed against the things that are coming in.

- I think that, as you know, that there's a lot of noise in threat intelligence. It's pretty critical for security operations teams to have a level of actionability really against the security stack. How do you action that type of information to determine if bad traffic is detected by your security stack?

- Yeah, that's a really good question too. I think we're all in a race in the security industry for high fidelity event data. And, you know, that's going to be true for quite some time. And the reason why is because adversaries have interesting opportunities and capabilities that they work through. When you're establishing your understanding of data, not even a few years ago, it was too expensive to have large volume data as a security professional. SIMs were taking in essentially cooked data and allowing you to sift between alerts. These days with the cloud, you now have things like alert data and raw data which would be your log data that may not necessarily have been sifted through a security event processor. And I think what really allows you to get down to high fidelity is really about your data modeling capability, how you measure the adversaries that are trying to accomplish their objectives and really discern what bad looks like. I know that's kind of a wiggly answer, but as we all move towards that, I think the biggest challenge I've seen in the industry around this particular space is high fidelity is only going to be able to be determined by us picking out a target state metric and capability answer or outcome to this which includes more than just compliance controls. There has to be more emphasis in the security industry around things like belt and suspenders type of approach to controls. Thankfully, in the environments I've worked in, there's been a lot of application of belt and suspenders, but that can mean that any one person in an organization doesn't necessarily get to see the entire picture of what's happening in a complex environment. So to me, that also means that you have to take in your data, you may have high fidelity at some point in your processes but it may not necessarily be apparent. And there's a lot of folks that really make up a security community that's going to be effective in dealing with organizational issues that are going to be security related.

- We've talked before around metrics and I think that there's a lot of people in security that get upset by trying to having to show metrics, but I haven't talked to a mature security team that is not showing some type of metrics and how those are very critical to communicate return on investment. What are some useful metrics that you've found useful to use around budget season?

- Yeah, you know, I've talked a lot in the industry around securability, in particular, being able to look at the attack surface of your organization and determine, in particular, what's happening in that space, right? securability is one of those things that I think is really about creating a target state of adversary resilience. And then if you break down adversary resilience into its component part measurements, input metrics, if you will, really, you have the notion of adversary opportunities. So that's the things that happen in terms of the risks that an organization takes. And those commonly can be a mixture of risks that they take deliberately and risks that they are unaware of. Escapes, so ensuring that you reduce and control escapes that might happen. So essentially, if you have controls in place, making sure that they're effective. And I believe that measuring escapes is really critical, which a lot of organizations already do. The term escapes can be a collection of things like vulnerabilities and incidents and some of those things because effectively very, systematically, you can determine that those are the control losses that you might have, the things that are actually, what I would term an adversary conversion of an issue. And then the last one I think is really interesting and one of the things that I think most companies are starting to understand which is adversary dwell time. Companies do measure things like customer dwell time on their applications. Adversaries also dwell and they look for things to do and they're looking for their opportunity to be met. And that can be a really meaningful metric towards understanding whether or not you're putting enough emphasis on your security controls, if you're investing them appropriately. I think those are some critical measures that the industry itself needs. And as I've looked through and done a lot of research in measurement, I'd say the security industry's been trying to solve security measurement almost as long as most of my career. I can look back on two decades now of conversations around security metrics. And so for me, it's been decades of a question that I'm very passionate about. That's allowed me to distill these down. And then I'd say the next piece of that puzzle is really, what lever metrics are you leveraging to be able to determine that the day-to-day work that you're doing is really going to move the needle on those input metrics for things like securability.

- I got to ask out of pure curiosity, how would you define securability?

- The way that I look at securability and define it is really, if you were to essentially establish a mathematical model, the ultimate goal for all of us is establishing a resilience level to adversaries and their threats at a significantly high level. So if we could all be a hundred percent adversary resilient and our applications could be a hundred percent adversary resilient, that would be ideal. The realistic research that I've seen so far is that, looking again, back at things like some of the reports that have come out from Symantec and McAfee, it looks like around most of the controls are effective at a 98.6% perspective. So the ultimate definition for me around securability has been escapes over exploitable opportunity. And so if you think about it, that resilience number can be attributed to a zero or a one, you either have it or you don't, but to really get to the dimension of understanding of how much you have it, you would basically take one and subtract away the exploitability that occurs within that resilience measure. And so ultimately, exploitability is a pretty small micro number and somewhat challenging for most organizations to reconcile when they're looking at bigger numbers and to get it to the point where it can be a number that most folks can reconcile. You need to make it so that the things that they do every day are applicable. So for me, securability is the five nines of security and it essentially tells you how much resilience and security you're applying to your space. And that means that every time you see an adversary, they have odds associated with whether they will be effective and it allows you to do the roll-up that's very similar to things like availability. So that was why we built it was to really try and have a measure that allowed us to have almost a companion to availability.

- Shannon, we appreciate your nuanced point of view today and thank you for your time. For the latest subject matter expertise around managing intelligence, please visit us at www.nisos.com. There, we feature all the latest content from Nisos experts on solutions ranging from mitigating advanced cyber actors, batting disinformation, mitigating insider threats and reducing threats around third party risk management and mergers and acquisitions. Special thank you to all Nisos' teammates who engage with clients and solve some of the world's most challenging security problems. Without the value the team provides day in, day out, this podcast would not be possible. Thank you for listening.