- Welcome to "The Cyber5," where security experts and leaders answer five burning questions on one hot topic on actionable intelligence to enterprise. Topics include adversary research and attribution, digital executive protection, supply chain risk, brand reputation and protection, disinformation, and cyber threat intelligence. I'm your host, Landon Winkelvoss, co-founder of Nisos, a managed intelligence company. In this episode, I talk with senior director of cyber intelligence strategy A.J. Nash. We talk about building holistic intelligence programs within enterprise, including how to leverage cyber threat intelligence beyond the confidentiality, integrity, and availability of data and networks. We also talk about the skillsets to hire and the metrics and frameworks that are useful for measuring success. Stay with us. A.J., welcome to the show. Would you mind sharing a little bit about your background for our listeners?

- Yeah. Thanks, Landon. Happy to be here and happy to talk a little bit about building programs. So my background, currently, I'm the senior director of cyber intelligence strategy for Anomali. Where I come from, I came out of the government space. So I'm not... There's a lot of folks in our industry, I'm not really a technologist. I'm not a ones and zeros kind of a guy. My time was spent in the government space doing traditional intelligence. Counter-terrorism, counter-insurgency, countering trafficking in persons, chasing war criminals. Almost all that time was spent at Fort Meade. So either NSA or U.S. Cyber Command. And then when I transitioned out of the government, I moved into the private sector, hm, about six years ago now. I started at a financial company, a large bank, working to build an intel program there from the ground up. I then moved on to a large technical company, where I did a lot of work there, helping folks build intel programs while we also delivered some intelligence to them. And then went inside that company and helped build a new intel program there as well. We realized we needed some improvements. I've been at Anomali now for about a year and a half. And my role still is very much focused on helping people build intel programs, mature their intel programs, understand the foundations of intelligence, and at the same time, I oversee our intel team and what we do to to contribute to the overall knowledge base of the intel community in the private sector.

- I appreciate you joining us, and you kind of teed up exactly what we're going to be talking about. So what's the first steps to building an intelligence program that supports enterprise?

- That's a good question. That's the tough piece, right? So, you know, when you're going to build an intel program, and there's a couple of things people they really need to focus on. So I'm very much a fundamentals kind of a person. So I talk a lot about the intelligence cycle. In fact, most things I talk about I'm not trying to brand something or reinvent something. Yeah, mostly it's about taking best practices out of the government space and bringing them into the private sector. So the intel cycle is a good example of that and it starts with planning and direction. So where most organizations really struggle is taking the time to plan properly, to have a reasoned discussion, to have some investigation to understand who are our stakeholders, who are we serving, what are we trying to accomplish. Spend more time figuring that out and then start to document what you believe your intel requirements are going to be based on those stakeholders. I think those are two fundamental pieces that a lot of organizations skip over. They move from, "We don't have an intel program" to, "Boy, we really need intelligence." And then they jump right into the vendor space. And how do we buy intelligence? How do we buy tools and technologies without taking the time to figure out who they're serving and what those needs really are. And fundamentally, I think the biggest problem you have, the reason that happens is because we're hiring the wrong people to build our intel program. Most organizations I run into don't have intel people, and when they decided to build an intel program, what they do is they take someone who's been very successful in their security role, a lot of times it's incident response, and they put that person in charge of the intel program. And it's understandable. They want to take somebody who's already a proven leader or already a proven performer in the security space, but not understanding that intelligence is a bit different, it is its own career field, sets you up with a mismatch. And from there, I think a lot of things cascade off. So to me, the way you start is first of all, you need to identify the right person, somebody who understands intelligence, the fundamentals of intelligence, and what organizations need. And then that person has the ability, the latitude as well, to take the time to understand what stakeholders you're going to be serving and what their needs are. Then you can start having discussions about what collection do we need. You know, what capabilities do we need to bring in in terms of accesses and tools and who do we need to hire. Starts from there. So I'd say those are the big fundamentals. So hire the right person first, give them the opportunity to build out an understanding of your stakeholder assessment, and then what intel requirements go with that.

- Do you see any industries that particularly do this well? Or is it really just on the size of the company that ultimately gets to a certain maturity in their security program?

- Yeah, so I think ultimately right now what I'm seeing is the financial sector is leading the way. You know, and it makes some sense. I think some of that's cultural. When we talk about the financial sector, it's just understood that people, people have tried to attack them, right? As long as banks have had money, people have been trying to rob banks. So it's built into the culture that there's threat. Therefore, there's a need to understand that threat and how to counter it. I think that put them at a giant advantage. So the areas where I think people struggle, again, are industries that aren't used to that. So if you think of that healthcare industry, historically people have not been breaking into healthcare organizations or research foundations and trying to steal from them. So they're less security-focused and therefore they're a little bit less intelligence-focused. So I certainly think the financial sector leads the way. I think other industries are starting to catch up with that, but I think they could take a lot of lessons learned from what finance has been able to do in building those programs. You know, the other half of what you asked, yeah, the size of an organization certainly plays a role. It plays a role in the budget and the maturity level a lot of times, but it isn't a single indicator of whether somebody will be mature or successful. There are some very large Fortune 500, Fortune 100 companies that are not really succeeding in this space. And there are some smaller companies that are doing quite well. So I think it has more to do with the mentality and the focus than it does necessarily your budget or the size of your brain.

- What kind of a combination of this kind of skillset is probably most appropriate to build an intelligence program.

- Yeah, that's a good question. So you know, obviously, as I mentioned, I think you start with somebody with intel background. I will say, there's no one single answer, but if I were just going to build somebody the ideal candidate, I would want somebody who's got 10 plus years experience in the intelligence space at one of the three-letter agencies. And I could go into some biases. There's a couple three-letter agencies I happen to think are better than others. But also, hopefully that person has spent time in the private sector first and cut their teeth in somebody else's organization because there's a culture shift that goes with that. The government space and the private sector work differently. Their politics are different, their timelines are different. Their objectives are different. So there can be a culture fit there. So if I had my choice, I'll take somebody with 10 plus years in the government space, proven leadership, who's hopefully cut their teeth someplace else in the private sector either working on an intel team or maybe has had a chance to even run and build one themselves. But then when you talk about the rest of the team, I believe in a pretty good mix. My personal opinion is I don't try to look for unicorns. I don't think people should. I think you can find people who are very technical and strong researchers who may not be interested in writing finished intelligence products. They may not be interested in getting on stage, might not be prose communicators, right? And then you have other folks that are very good at putting together the puzzles, taking all those pieces and making a story out of it, making sense of it, and being able to communicate that to others. And that isn't to say you can't find someone who does both, but I've often found with organizations if you split up the responsibilities that way, they connect really well. From a talent standpoint, I definitely want people who have intelligence experience, who understand the fundamentals of intelligence. That means, again, looking for people who've spent time in the IC. There can be exceptions, certainly. I've seen people be very successful who never spent time in the intelligence community, but they're strong at technical skills, they have inquisitive minds. They've done some studying that might fill the gap of not having been in the intelligence community, something like the SANS course, GCTI, or there are some universities that put out some good folks with this skillset. So you want to have that. And you know, you could have some folks who have incident response backgrounds with maybe some network security background, is really valuable. But I think the key piece if you're gonna go into the intel space is, again, that inquisitive mind and really understanding what intelligence is, and the structures and analytic techniques, and the communication that goes along with that. I'm absolutely a proponent for intelligence community directives and a lot of the standards of intelligence because they're proven to work. So I wouldn't want a team that's filled with just technical folks. You know, if you take a team and you fill it with your best incident responders, you're going to build another incident response team. They'll be very good at what they do, but they just don't think the same way as intel folks. It's not their background. So, so it can be a lot of different types. You know, I've looked at journalism graduates, people like that who are really good at communicating, again. I've heard stories of people talking about people hiring folks who are good at math or who are good at music even, artists. I do want a good mix on the team. I recommend that for folks. So you don't get into group think. So you have folks that think from different points of view. That includes, gender. That includes race and culture. And again, the technical backgrounds. So there isn't one pure answer, but I do lean heavily on folks who understand traditional intelligence practices and processes. Otherwise, you're going to have a very hard time developing an intelligence team.

- So let's kind of get into the problems. 'Cause I think you just kind of alluded to some of the issues that you get into, right? So somebody coming out of CIA, NSA, they're probably going to be looking at intelligence requirements. They're probably going to be heavily focused on producing some type of daily brief or daily report. And that might not be the quite the answer. And just like you said, the incident response person is going to ultimately be building another incident response program. I've talked to other folks that come out of the financial industry and they say like risk is what you have to do. So therefore you have to gear an intelligence program around what you have to do, which ultimately, I guess, leads to my question. You know, what are the business problems that intelligence addresses, both with inside confidential integrity and availability of data systems, and frankly, even with outside of CIA data systems?

- Yeah, no, I think, I mean, you hit a key point, right, when you're talking about risk? When I talk about intel programs, again, as you do that stakeholder assessment, I tell people to think big, right? We're not just talking about the SOC. We're not talking about just defensive cyber operations or offensive cyber operations. When you think big, you're gonna be the executive level, you start talking about what not just even the CISO needs but the CIO, the C-suite, the board. You know, the two languages are risk and profit, right? And everybody in the company at some level has to be able to solve for those. Either you're making the company money, you're saving the company money, you're reducing the company's risk, or you're a liability. Like, that's pretty much where you land, right? And I think the intel team is a big part of that. So most intel organizations, if you do take the time to do the stakeholder assessment, you get those requirements, you're going to find where that fits in. I think intelligence needs to be driving security. And it's always a discussion about risk. Here's what we understand that's going on outside of the wire and this is the level of threat that this presents to our company based on our understanding of the adversary's intentions and their capabilities versus what our landscape looks like. What's our footprint look like? What's our risk look like? What are our defensive capabilities? And the goal is to be able to communicate risk to the CISO, so the CISO can make the right decisions on what we need to focus on, what we need to back-burner, because ultimately the CISO ends up owning that risk. So they need to be informed. Intelligence's job is to allow leadership to make informed decisions. They won't always make the decision the intel team thinks they should make, but ultimately, they own that risk and they need to know what they're dealing with. So that's our job. And the other side of it is is connecting that external knowledge, so the internal knowledge can be done proactively, hopefully. No, but also, reactively. If something bad happens, our job is to work with the incident responders, help them understand who might've been responsible, how they might've accomplished it, what they might be looking to do next to help cut down on time to remediate, which, again, is also a risk factor, that there's calculations for costs related to remediation. So yeah, absolutely. I think risk is at the front of what we talk about doing. I think intel organizations have to be focused on how do we make things less risky, and how do we make it more predictable, and how do we make leadership aware of what risks they're taking on so they can communicate? Bad things are still going to happen, but when they do, you want to have your leadership in a position to explain to the board this was a risk that we all understood and we accepted and we were knowledgeable of. When bad things happen and nobody's aware that this was even a possibility, they hadn't accepted the risk, that's when you have significant problems. And that's when people lose jobs.

- Have you seen intelligence programs go beyond the CISO and really expand to physical security, fraud, trust and safety, executive protection? Where have you seen it kind of blossom outside of just what's considered traditional cyber threat intelligence?

- Yeah, absolutely. And I'm a big proponent of that. I think one of the flaws we've seen, as the private sector is beginning to really take on intelligence as a function and is really growing it, the flaw we're starting to see is where people are putting their intel programs. A lot of times the intel program's built, it's almost always built inside the SOC still. And a lot of times it's built under either your SOC manager or even your director of defensive cyber operations, something like that. And that's really limiting. So when I talk to people about building programs and that stakeholder engagement, a lot of the orgs you just mentioned, I'm a big believer in they're all customers. Governance, risk and compliance, physical security, executive protection, insider threat, which can even lead you down the HR path a little bit. Certainly the CISO and all the CISO's requirements are going to be in there. I think all of those are really important and they're interconnected. And when you do that, you elevate your intel team outside of the SOC, but for the same expense, give or take, you get a lot more value because now you're talking about a unified support to security across the enterprise. Because these threats cross these areas, right? Physical security sometimes is a pretext for cyber attacks or vice versa. Insider threat is an issue that crosses over. When you start seeing something inside your networks, it's great to have an intel team that can also take a look at what's going on outside the wire and maybe check into somebody's social media or look at what they're seeing in the deep dark web, perhaps, in chatter or where other discussion may be, Pastebin, et cetera. So absolutely, I believe the intel team should be supporting enterprise-wide, up to and including the board. And I think that's where you start to be able to really deliver metrics on value, which is another big piece we're talking about, right? We're talking about risk, we're talking about what you're doing for the bottom line. There's a lot of discussion about where the value is. What are you actually providing? And if you're going to spend, and you are, if you're going to do this right, you're going to spend a couple of million dollars or more to build a really good intel program, you might as well build it as high as you can because you're really not adding a whole lot more expense at the higher level but you're getting a lot more value out of it. Just supporting your SOC is just not really getting enough value out of the technology and the people you're going to hire and the vendors you're going to pay for all the accesses. It makes sense to start there. But I think the roadmap and the vision has to be to be an elevated organization that serves all those other functions you were just referring to.

- So let's talk about . Based on that complex answer, let's get into a more complex question around metrics, right? So what are the metrics of success for an intel program? Let's start with the cyber functions of the data systems and then think about more broadly how that could be applied to other issues of the business.

- Yeah, metrics is... In my opinion, metrics is the hardest part of all of this. A lot of folks, almost everybody, still talks about actionable intelligence. Everybody's really focused on actionable intelligence, which is good, and of course, it's really important. And it helps drive some metrics. If intelligence provides tippers or written products, or whatever it might be, or even automated solutions, machine and machine intelligence, feeding one machine to tell another machine to make a change, et cetera. Those are all things that generate metrics. And you can see proactive defense, changes in signatures and changes in protocols are all things that can be measured. One of the challenges we don't get solved through that though is the value of informational intelligence. And that's a tougher one to draw a metric for. And it's a little more strategic and it's why people haven't quite gotten there yet. But the example I often give when you talk about informational intelligence and the value of it is the goal of intelligence is to be proactive. You want to know about things before they happen, so you can make adjustments so that things don't happen to you. The problem with that is sometimes you can never measure whether it would have been bad or not because you prevented the problem. And that's what happens when you have informational intelligence. For instance, if you understand about an adversary that's not a threat to your industry or to your geography, but based on some research, you determined that there's a potential they could become one and you make a bunch of changes, understand their tactics, techniques, and procedures, and a month later or six months later or a year later, there's an attack against another company that used those tactics, how do we know would that have mattered to us or not? What would the costs have been? So there's always some challenges there. But to answer your question of what metrics exist and how they've been working, right? Again, with the actionable piece, I think it's relatively easy to draw some conclusions based on what actions were taken based on the intelligence. We provided X number of bits of intelligence to different organizations. These are the actions they took. You can certainly document that. So anything that was actioned upon is helpful. I think another metric we use ties back to those intel requirements. If organizations have requirements, then the intelligence that's being produced should match against those requirements. And you can use that as a metric to say how much intelligence that was produced and delivered matched against which intelligence requirements and what the prioritization of those were. And when you talk about intelligence, bring in incident response, again, you get a good metric there when you're trying to reduce the time to detect or reduce the time to remediate. Intelligence can support those functions. And that creates some measurements that can be used. So those are just a few examples of where you can go with this. Metrics is still a really hard part when it comes to intelligence, again, because of the proactive piece to it. If you make proactive changes that nothing ever happens, that's a good thing, but sometimes it can be hard to explain to folks why this really mattered.

- Have you seen decision metrics or a decision tree be successful and used as a metric? Example, if there's information in the wild saying X actor is going to attack in a certain way and that information makes it to the, let's say, the adversary emulation or red team, and then they ultimately replicate that type of attack, will the security engineers then take that type of information and ultimately may increase the controls to ultimately reduce dwell time? Have you seen even just that process tree take place, be an appropriate metric if we're just talking about CIA of data systems?

- Yup, I have. That's a really good example. I'll be honest with you, that's a pretty mature one you don't see too often, but it is a perfect example of where a red team is a really good customer of an intel organization. You know, red teams are trying to develop cases that can work, that are as reflective of the real world as possible. So if the red team's a good customer of your intel organization, yes, they're able to emulate what's going on in the real world, identify your challenges and your risks, and make those changes before bad things happen. So it's a perfect use case as long as those organizations are working well together and the documentation is in place to show, hey, the reason we made these changes and lowered our risk was because the intel team informed us that there were adversaries that were specifically doing these things and we were able to emulate that and find that those are things we would not have been resistant to as an organization. So we had to make those changes. So that's a really good example of where you can generate some metrics off of this. I will be honest, I don't see a lot of organizations quite that mature yet. They should be driving that direction. And there's a lot of folks that can help them get there. Because that is a really good metric for where intel supports the red team specifically in this case.

- We've kind of glossed over, and there's probably no shortage of frameworks in the industry, sort of talking big picture, do you think that there's a need for a new framework in terms of developing intelligence programs? Are we still in early days and you can kind of mishmash different frameworks together? How would you ultimately try and frame this? If you're sitting there pitching a board, how would you discuss frameworks? We talked about reducing threats in terms of reducing attack surface as just one example among many. And of course, the board ultimately always comes back and says, "What are you leveling that against? "What are you applying that to in terms of our framework?" And sometimes there are not, you know, outside of compliance, that's a hard question to certainly answer sometimes.

- Yeah, it's another really tough, challenging position we're trying to work through, right? So I do think as an industry we haven't solved this problem. And I've said before that the first organization that solves this in a meaningful way and can communicate it is probably going to do really, really well in that business. But you know, there are several frameworks out there, right? So we've got the MITRE ATT&CK framework, can help people understand threats and risks and how they work together and how adversaries are operating and how you can map that against your own organization's defenses. I think that's really, really valuable, but it's very, very technical. I don't know that it boils up particularly well when you get into the senior leadership who need to understand this thing. Don't want to get into the granular. You know, again, a couple of other good frameworks that we use as intel folks, you know, we still talk about kill chains, the Diamond Model. Those are really valuable. They don't boil up particularly well. So when you get in the higher levels, the NIST framework is something that people understand. There's compliance frameworks you've mentioned. People have talked about the FAIR framework. I'm not as big a fan of that from this standpoint. FAIR is redesigned to help you measure the likelihood of damage after you've been compromised. And I think intel is really meant to be more proactive. So what you want to do is show the value of intel is lowering risk. So the ability to tie your intel organization, your intel spend and production, to lowering of risk, to changing your risk score in a defensible way is, I think, the golden goose of this industry. And I don't think anybody's quite solved it yet. There's some good organizations out there. And I'll be honest with you, this is a project I've had on my mind for the last couple of years and I haven't made any progress, frankly, either. I'm still working on it. There's a few organizations out there that do the risk piece really well. And I think being able to tie intel into that and really have a discussion about a justifiable, defensible metric that you can tie in and say, "If we put intel into this piece and we make these changes, "it lowers our number by this much," is something we're still all striving for. I think customers really need it. I think it helps them understand the value of what they're getting and understand their risk posture. It helps them also drive budget discussions up the chain and be able to say to people why we're spending this much money and what it's doing for us. And certainly vendors need to solve this problem because it helps us explain to people the value of the products and services that vendors are selling. So I don't see an answer yet, to be honest with you. I don't see one that's caught the market and has really become a standard. So I think a lot of us are still fighting to try to find that right piece. I'm hopeful. I know some folks working on it, myself included. I'm hopeful one of us, or several of us, can crack that nut, but as of today, I haven't seen one where everybody's brought it all together. I'm actually leaning back into the government space again. There's a couple of frameworks there. I mentioned this. There's another framework that was developed, a cybersecurity framework, a few years back that I'm actually right now taking a look at and trying to see if there isn't a way to to make good use of that. So not there yet. I hope this year we get there a little bit better than we are today.

- A.J., we'll keep hustling, and ultimately, we will crack this nut as an industry, for sure. Appreciate your time today. For the latest subject matter expertise around managed intelligence, please visit us at www.nisos.com. There, we feature all the latest content from NISOS experts on solutions ranging from supply chain risk, adversary research and attribution, digital executive protection, merger and acquisition diligence, brand protection and disinformation, as well as cyber threat intelligence. A special thank you to all NISOS teammates who engage with our clients to conduct some of the world's most challenging security problems on the digital plane and conduct high state security investigations. Without the value the team provides day in, day out, this podcast would not be possible. Thank you for listening.